5-7 years of experience in data engineering or related field
Proficiency in SQL, Spark SQL, PySpark, and Python
Experience with cloud environments, particularly Databricks
Strong understanding of ETL/ELT processes
Ability to develop reproducible and auditable code
Familiarity with data integrity and validation techniques
Experience in creating technical documentation and training materials
Responsibilities
Build, test, and maintain data pipelines in cloud environments
Design bidirectional data movement for operational systems
Develop extraction and transformation logic using SQL and Python
Implement patient-matching logic to ensure data integrity
Harmonize datasets across various sources and facilities
Create data flow maps detailing inputs and outputs
Deliver clean data for analytical reporting platforms
Develop training materials for data administrators
Benefits
Remote work flexibility
Opportunity to work with cutting-edge cloud technologies
Engagement in meaningful data projects impacting healthcare
Collaboration with government informatics teams
Potential for long-term project involvement
Professional development through training material creation
Full Job Description
Responsibilities:
AUTOMATED APPLICATIONS AND DATA PIPELINES The Contractor shall: Build, test, implement, enhance, maintain, and support data pipelines within the cloud workgroup (Databricks) and other analytical/operational systems that extract data from the enterprise CDW cloud environment and deliver reliable, analysis-ready datasets.
Design and maintain bidirectional data movement between the enterprise CDW, the cloud workgroup, and operational systems in support of NTSP workflows, processes, initiatives, programs, or projects, including source-of-truth logic, conflict resolution, and synchronization integrity.
Develop and maintain extraction and transformation logic (ETL/ELT) using SQL/Spark SQL, PySpark, and Python, producing reproducible, version-controlled, auditable code rather than manual one-off pulls.
Implement patient-matching and record-linkage logic using multiple identifiers, with auditing of prior matches, to prevent data-integrity failures.
Harmonize datasets across facilities and sources (consistent keys, date ranges, formats, and naming conventions) and implement automated validation checks (missingness reporting, range and impossible-value checks) at extraction and delivery.
Develop data flow maps identifying inputs, outputs, data sources, update frequency, and internal controls.
Deliver clean, well-modeled data in a form the Government's in-house Informatics team can consume for Power BI or other analytical reporting platforms.
Develop administrator and technical documentation for all Contractor-developed pipelines and code, and facilitate transfer to the Government upon TO conclusion. •
Develop data administrator training materials and presentations describing, at minimum: purpose, background, design, data sources and flow, and step-by-step administrator and end-user instructions. •
Provide a transition plan detailing frequency and competencies necessary to ensure continuity of NTSP efforts and support of deliverables. •
Provide ongoing technical support, maintenance, enhancements, and updates to all Contractor-developed and Contractor-modified applications and associated documentation, and facilitate transfer of management to the Government upon TO.
