Job Description

Overview

The Lead Data Platform Engineer is the highest individual contributor engineering role on the EDAP (Enterprise Data Analytics Platform) team. You will serve as the primary technical authority for the platform, owning the end-to-end architecture and driving the largest, most complex engineering initiatives across the organization. This is not a management role - you will not carry a headcount budget or run performance reviews - but your technical leadership will shape how an entire team of engineers do their work.

You will set the architectural vision for the platform, define engineering standards, and hold functional oversight over technical decisions made across the team. You will lead multi-quarter, cross-functional programs from inception to production, partner directly with engineering leadership and senior stakeholders, and represent the platform function in organizational-level planning. Engineers at every level will look to you for guidance, direction, and mentorship.

This role requires technical depth and leadership qualities. You are the person the team turns to when the problem is hard, ambiguous, or high stakes.

Key Responsibilities

Architecture & Technical Vision

• Own the technical architecture of the data platform end-to-end: ingestion, storage, transformation, orchestration, serving, and observability layers.
• Author and maintain the platform architectural vision document; lead quarterly architecture reviews to assess alignment with organizational goals and technology trends.
• Define and evolve the target-state architecture for the platform, establishing a multi-year technology roadmap in partnership with engineering leadership.
• Evaluate emerging technologies, frameworks, and make evidence-based adoption recommendations.
• Serve as the final technical escalation point for complex design questions, cross-team conflicts, and build-vs-buy decisions affecting the platform.

Large-Scale Project Leadership

• Lead the planning, execution, and delivery of multi-quarter platform initiatives involving multiple engineers, cross-functional dependencies, and significant organizational impact.
• Break down large, ambiguous programs into scoped workstreams; assign technical leads per workstream, define milestones, and manage cross-team dependencies and risk.
• Drive initiative kick-offs with clear problem framing, success criteria, architectural constraints, and delivery phasing - from 0-to-1 exploration through production hardening.
• Maintain stakeholder alignment throughout delivery: proactively communicate status, surface trade-offs, and escalate blockers to leadership before they become risks.
• Lead post-mortems and retrospectives for large initiatives; document and socialize lessons learned to raise the organizational bar on delivery excellence.

Functional Oversight & Engineering Standards

• Exercise functional oversight across the EDAP team: review technical designs, set quality gates, approve architectural decisions, and ensure consistency of implementation patterns.
• Define, document, and socialize platform engineering standards including coding conventions, testing requirements, CI/CD practices, schema design guidelines, and SLA frameworks.
• Establish and own the platform's technical review process (design review): triage incoming projects, chair review sessions, and ensure decisions are documented and traceable.
• Identify and drive resolution of technical debt, redundancy, and architectural drift that impede platform reliability, developer productivity, or scalability.
• Partner with security, compliance, and infrastructure teams to ensure platform systems meet governance, data privacy, and regulatory requirements by design.

Mentorship & Organizational Influence

• Serve as a primary technical mentor for junior and senior data platform engineers across the platform; provide structured coaching on system design, technical communication, and engineering judgment.
• Conduct and lead design reviews, architecture critiques, and technical deep-dives that strengthen the overall capability of the engineering team.
• Define what technical excellence looks like at each level of engineers and participate in calibration discussions.
• Represent the data platform in engineering-wide forums, all-hands, and external venues (conferences, open-source communities, recruiting events).
• Build a culture of documentation, reliability, and platform-as-a-product thinking across all data platform engineering functions.
• Participate in goal planning cycles as a technical voice; define engineering-led goals that improve platform reliability, developer experience, and data quality.

Qualifications

Required Qualifications

• 8-14 years of professional software or data engineering experience, with at least 3 years in a technical lead, principal, capacity on a data platform or distributed systems team.
• 6 Plus years of Vizient experience
• Demonstrated track record of leading large-scale, cross-functional data infrastructure projects from conception through production with measurable business impact.
• Deep expertise in distributed data systems: data warehouse/lakehouse architecture, streaming platforms, large-scale batch processing, and cloud-native data infrastructure.
• Expert-level proficiency in Python and SQL; strong working knowledge of at least one JVM language (Scala or Java) for Spark or Flink development.
• Deep knowledge of modern data warehouse and lakehouse platforms (Snowflake, BigQuery, Redshift, Databricks, or equivalent) including storage optimization, compute management, and cost governance.
• Strong command of open table formats (Apache Iceberg, Delta Lake, or Apache Hudi) and their trade-offs in production lakehouse architectures.
• Experience architecting and operating large-scale streaming pipelines with Apache Kafka, Kinesis, or Pub/Sub, including schema management, consumer group design, and exactly-once semantics..
• Exceptional communication skills: ability to write clearly, present confidently, and adapt technical depth to a wide range of audiences from individual contributors to high level leaders.
• Experience establishing engineering standards, review processes, and quality gates across multi-team engineering organizations.

Preferred Qualifications

• Experience designing and implementing federated data platform architectures at scale- like data mesh.
• Background in ML infrastructure, feature engineering platforms, or MLOps pipelines that intersect with the data platform layer.
• Familiarity with data governance, data cataloging, and lineage tooling (DataHub, OpenMetadata, Collibra, or Alation).
• Prior experience working at a company with a mature, scaled data platform serving hundreds of internal consumers or petabyte-scale data assets.
• Exposure to formal program management methodologies or experience driving large initiatives using structured delivery frameworks.

About the Team

Atlantic Health System is at the forefront of medicine, setting standards for quality health care powered by a workforce of more than 18,000 team members and 4,800 affiliated physicians dedicated to building healthier communities, the system offers more than 400 sites of care, including eight award-winning medical centers. Specializing in cardiovascular care, cancer care, orthopedics, neuroscience, pediatrics, women's health and rehabilitation medicine. Atlantic Medical Group, comprised of 1,000 physicians and advanced practice providers, represents one of the largest multi-specialty practices in New Jersey and includes finance, legal, marketing, human resources, talent acquisition, ISS and more. Caring for our patients, our team members and the communities we serve is our central mission.