Job Summary
The Data Engineer will transition the Energization Timelines pipeline from SAS to SAP Datasphere and establish reusable views that can also support Palantir Foundry. The role focuses on identifying and integrating source data, rebuilding the Energization and Easement Tracking (EET) step mappings across all eight phases, and implementing transformation logic for regulatory reporting. The pipeline supports CPUC-mandated reporting, requiring accurate calculations, validation with Regulatory SMEs, and continuity across existing dashboards. The initial production release is targeted within four months, followed by prioritized Work Management dataflow remediation and optimization.
Key Responsibilities
• Identify and integrate required source fields from Design Manager, SAP, Click, and related systems.
• Rebuild EET step mappings across all eight phases, documenting ownership and clock treatment for each phase.
• Implement transformation logic to distinguish SCE-controlled time from customer-driven time, including pause-the-clock treatment.
• Handle overlapping and repeated task intervals accurately.
• Validate pipeline outputs with Regulatory and business SMEs and resolve outstanding interpretation questions.
• Productionize the pipeline in the CPUC-required reporting structure.
• Repoint dashboards currently sourced from SAS and reconcile their outputs against SAP Datasphere.
• Establish reusable Datasphere views for Foundry consumers.
• Document data lineage, validation procedures, and handover materials to support ongoing maintenance by the permanent team.
• Inventory and rationalize existing Work Management dataflows, consolidating duplicated transformations and retiring redundant or legacy flows where appropriate.
• Remediate fragile joins and dependencies, standardize business logic, and improve refresh and validation controls.
• Migrate appropriate transformations and logic into the Datasphere and Foundry backend.
Required Qualifications
• Demonstrated experience in production data engineering and pipeline development.
• Strong data integration and transformation skills across multiple source systems.
• Experience implementing complex time-based business rules, including interval overlaps, repeated task intervals, and pause-the-clock logic.
• Ability to validate data outputs with business and regulatory stakeholders and resolve data interpretation questions.
• Experience reconciling dashboard outputs and maintaining reporting continuity during pipeline migrations.
• Ability to document data lineage, transformation logic, validation processes, and operational handover procedures.
• Ability to remediate dataflow architecture, consolidate duplicated transformations, and improve data quality and pipeline maintainability.
Preferred Qualifications
• Experience with SAP Datasphere and Palantir Foundry.
• Experience modernizing legacy data pipelines and migrating reporting dependencies from SAS to modern data platforms.
• Familiarity with regulatory reporting processes and compliance-sensitive data environments.
• Experience with Work Management dataflows, complex joins, and reusable data products.
Deliverables
• Initial production release within the four-month delivery timeline, with the Energization Timelines pipeline running on SAP Datasphere and the SAS dependency retired.
• EET mappings rebuilt across all eight phases, with phase ownership and clock treatment documented.
• SCE-controlled versus customer-driven time logic implemented, including overlapping and repeated intervals, and signed off by Regulatory.
• All dashboards previously sourced from SAS repointed and reconciled, with no interruption in reporting continuity.
• Reusable Datasphere views available to Foundry consumers.
• Documented data lineage, validation approach, and handover package.
• Following the initial delivery, prioritized Work Management remediation, including dataflow rationalization, standardized business logic, improved refresh and validation controls, and migration of appropriate logic into the Datasphere and Foundry backend.
