Position Summary

The Senior Data Specialist provides technical leadership in designing, developing, and optimizing cloud-native data solutions on the Azure Data platform, with a primary focus on Azure Databricks. This role is responsible for building enterprise-scale data pipelines, implementing modern lakehouse architectures, and enabling trusted, high-quality data products that support analytics, AI/ML, and business decision-making.

The ideal candidate brings deep hands-on experience with Databricks, PySpark, advanced SQL, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Delta Lake, Delta Live Tables (DLT), and Change Data Capture (CDC) frameworks. This individual will play a key role in designing and supporting robust Medallion Architecture patterns while ensuring data quality, performance, security, and governance across the enterprise.

Working closely with architects, product teams, analytics stakeholders, and governance teams, the Senior Data Specialist will deliver scalable, reusable, and production-ready data engineering solutions in a highly regulated environment.

Key Responsibilities

Azure Databricks & Data Engineering

• Design, build, and optimize large-scale data pipelines using Azure Databricks, PySpark, and advanced SQL.
• Develop and maintain batch and near real-time data ingestion frameworks using Azure Data Factory (ADF), Delta Lake, and CDC methodologies.
• Build and support modern Lakehouse solutions leveraging Medallion Architecture (Bronze, Silver, Gold layers).
• Design and implement Delta Live Tables (DLT) pipelines to improve reliability, maintainability, and data quality.
• Develop highly scalable and resilient ETL/ELT frameworks that process high-volume enterprise datasets.
• Optimize Spark workloads, partitioning strategies, job orchestration, and SQL performance for large-scale processing environments.
• Implement robust monitoring, alerting, troubleshooting, and performance-tuning practices across data pipelines.

Data Quality & Governance

• Establish and maintain data quality validation frameworks, lineage tracking, metadata management, and governance controls.
• Implement automated testing and reconciliation processes to ensure data accuracy, completeness, and consistency.
• Support regulatory, security, and compliance requirements through documented controls and audit-ready processes.

Solution Delivery & Leadership

• Collaborate with architects and stakeholders to translate business requirements into scalable technical solutions.
• Drive engineering best practices, code reviews, CI/CD adoption, and reusable framework development.
• Mentor junior engineers and provide technical leadership on Databricks and Azure-based implementations.
• Evaluate emerging Azure and Databricks capabilities and recommend enhancements to improve platform performance and operational efficiency.

Minimum Qualifications (Knowledge, Skills & Abilities)

Required Technical Skills

• Expert-level hands-on Azure Databricks experience designing, building, and supporting enterprise-grade data solutions.
• Strong expertise in PySpark and advanced SQL for large-scale data processing and optimization.

Extensive experience with:

• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS Gen2)
• Delta Lake
• Delta Live Tables (DLT)
• Change Data Capture (CDC)
• Medallion Architecture
• Deep understanding of Spark architecture, job optimization, partitioning strategies, and performance tuning.
• Experience architecting and supporting end-to-end ETL/ELT pipelines in cloud-native environments.
• Strong knowledge of data modeling, data warehousing, metadata management, and governance best practices.
• Experience implementing CI/CD, source control, automated testing, and production deployment processes.
• Ability to troubleshoot complex pipeline failures, data quality issues, performance bottlenecks, and integration challenges.

Preferred Technical Skills

• Experience with Snowflake and modern cloud data ecosystems.
• Experience with Kafka, streaming architectures, and event-driven data processing.
• Exposure to AI/ML enablement and advanced analytics workloads within Databricks.
• Azure certifications and Databricks certifications are highly preferred.

Business Experience

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline; Master's degree preferred.
• 7+ years of experience in data engineering and enterprise data platform development.
• 3+ years of deep hands-on Azure Databricks engineering experience in production environments.
• Proven track record building and supporting enterprise-scale data pipelines and Lakehouse implementations.
• Experience working with large volumes of structured and semi-structured data.
• Healthcare, pharmaceutical, life sciences, or other regulated industry experience strongly preferred.

Working Conditions

Flex & Connect model requiring 2 days per week in office.

We are proud to offer a competitive compensation package at McKesson as part of our Total Rewards. This is determined by several factors, including performance, experience and skills, equity, regular job market evaluations, and geographical markets. The pay range shown below is aligned with McKesson's pay philosophy, and pay will always be compliant with any applicable regulations. In addition to base pay, other compensation, such as an annual bonus or long-term incentive opportunities may be offered. For more information regarding benefits at McKesson, please

Our Base Pay Range for this position