Senior Data Engineer

OnMed

Healthcare
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's degree in Computer Science, IT, or a related field.
  • 7+ years of experience in data engineering, architecture, or enterprise integration.
  • Deep expertise in Databricks, including Delta Lake and Unity Catalog.
  • Advanced knowledge of Azure services like Data Factory and API Management.
  • Strong proficiency in SQL and Python; knowledge of C# is a plus.

Responsibilities

  • Architect and design scalable enterprise data solutions with Databricks and Azure.
  • Own and optimize the Databricks environment for performance and security.
  • Define a long-term data architecture strategy aligned with healthcare compliance.
  • Serve as the lead integration architect for system interfaces and API exchanges.
  • Design integration patterns using Azure and ensure data interoperability.

Benefits

  • Unlimited PTO and paid holidays.
  • Comprehensive benefits package.
Full Job Description
Who You Are

You are a senior-level Data Engineer and integration architect experienced in designing and scaling enterprise data platforms in cloud-first environments. You bring deep expertise in Databricks and Microsoft Azure and have led the architecture, optimization, and governance of modern data ecosystems.

You think beyond pipelines - you design resilient, scalable data platforms that power analytics, interoperability, and operational systems. You are comfortable operating at both the strategic architecture level and the hands-on engineering level. You thrive in high-growth environments and are motivated by leveraging data to expand access to quality healthcare.

Role Responsibilities

Enterprise Data Architecture & Platform Ownership
  • Architect, design, and scale enterprise-grade data solutions using Databricks and Azure cloud services.
  • Own and evolve the Databricks environment, including performance optimization, cost management, governance, and security controls.
  • Define long-term data architecture strategy aligned to business growth and healthcare compliance, and the evolution of OnMed's CareStation and HomeStation platforms.
  • Establish data modeling standards, metadata management practices, and scalable ETL/ELT frameworks.


Integration Architecture & Interoperability
  • Serve as lead integration architect across all system interfaces and API-driven data exchanges.
  • Design and optimize integration patterns using Azure Data Factory, Azure API Management, and event-driven architectures.
  • Ensure reliable, secure, and scalable interoperability between internal systems, third-party platforms, and healthcare data sources - including HL7/FHIR-based integrations with EHR and clinical systems.
  • Oversee monitoring, observability, and fault tolerance across all integration touchpoints.


Data Engineering & Pipeline Development
  • Build and optimize high-performance, scalable data pipelines in Databricks using Python and SQL.
  • Design and implement advanced data transformations and complex query optimization.
  • Implement automated testing, validation frameworks, and data quality controls to ensure pipeline reliability.
  • Drive CI/CD practices and infrastructure-as-code principles for data platform management.


Governance, Security & Compliance
  • Establish and enforce data governance standards using Databricks Unity Catalog, including lineage tracking, auditing, and role-based access controls.
  • Ensure compliance with healthcare data privacy and security regulations including HIPAA, HITRUST, and SOC 2 requirements.
  • Implement role-based access, encryption standards, and secure API management, and access controls for PHI/PII data handling.


Cross-Functional Leadership
  • Partner with BI, analytics, product, and engineering teams to translate business needs into scalable data solutions.
  • Provide architectural guidance and mentorship to junior engineers and analysts.
  • Influence data strategy at the leadership level through proactive recommendations and roadmap planning.
  • Communicate technical concepts clearly to non-technical stakeholders including clinical, operational, and executive audiences.


Knowledge, Skills & Abilities

  • Experience operating in regulated industries; healthcare (HIPAA, HITRUST, SOC 2) strongly preferred.
  • Deep expertise in Databricks: Delta Lake, Unity Catalog, performance tuning, cost optimization, job orchestration.
  • Advanced knowledge of Azure services including Azure Data Factory, Azure API Management, Azure Storage, and related cloud infrastructure.
  • Strong proficiency in SQL (complex query optimization, performance tuning, indexing strategies) and Python; working knowledge of C# a plus.
  • Extensive experience designing scalable ETL/ELT frameworks and data warehousing solutions.
  • Experience implementing event-driven or API-based integration architectures.
  • Familiarity with healthcare interoperability standards (HL7, FHIR) is a strong advantage.
  • Proven ability to balance architectural vision with hands-on execution.
  • Strong communication skills with the ability to influence technical and non-technical stakeholders at all levels.
  • Demonstrated ability to mentor engineers and drive engineering best practices across a team.

Requirements

Education & Experience

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • 7+ years of progressive experience in data engineering, data architecture, or enterprise integration.

Benefits

Benefits

OnMed provides a competitive salary and benefits package, including unlimited PTO and paid holidays. The base salary range for this role is up to $170,000 commensurate with the candidate's experience, plus an annual discretionary performance bonus.

#LI-HYBRID

Similar Jobs

More Jobs at OnMed

More Healthcare Jobs

Find similar Senior Data Engineer jobs: