We are currently looking for a Principal Data Platform Engineer (Databricks) to join a modern data platform initiative for healthcare and life sciences clients. This role will focus on owning end-to-end architecture and delivery of scalable data solutions, working closely with data architects, analysts, and client stakeholders.
Location: Remote, United States
Key Responsibilities:- Own end-to-end architecture and delivery of scalable data solutions, with strong emphasis on Databricks-based platforms and modern cloud ecosystems
- Lead the design and implementation of data pipelines, data models, and transformation frameworks supporting analytics, reporting, and advanced use cases
- Serve as the primary client-facing technical lead, building trusted relationships and guiding stakeholders through complex data decisions
- Translate ambiguous business requirements into clear technical architectures and delivery plans
- Establish and enforce best practices across data engineering - ingestion, pipeline orchestration, testing, optimization - and DevOps
- Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks
- Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support
- Collaborate with cross-functional teams - data architects, analysts, client stakeholders - to ensure alignment and value delivery
- Identify risks and proactively address challenges to ensure high-quality, on-time delivery
- Contribute to internal capability building: reusable frameworks, accelerators, and thought leadership
- Support business development by shaping technical solutions and contributing to proposals and client discussions
Requirements:- 7+ years of data engineering experience, with clear progression into technical leadership and architecture ownership
- Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing
- Advanced SQL and Python skills, with strong experience building and optimizing large-scale data pipelines
- Hands-on experience with cloud platforms (AWS, Azure, or GCP), including data services and infrastructure design
- Solid understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing
- Experience with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt)
- Proven ability to lead technical delivery while staying hands-on
- Strong client-facing experience: requirements gathering, solution design, executive communication
- Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments
Nice to Have:- Healthcare data experience (e.g., Epic, HL7, FHIR, claims data)
- Experience with CI/CD, DevOps practices, and infrastructure-as-code tools
