Bachelor's or Master's in Computer Science, Data Science, or related field
7+ years of experience in data engineering with cloud platforms, particularly Azure
Advanced expertise in Azure (5+ years) and Databricks with Apache Spark (5+ years)
Strong SQL (10+ years) and Python (5+ years) skills; familiarity with Scala is a plus
Experience in building real-time data pipelines and a strong understanding of data governance
Responsibilities
Design, build, and optimize data pipelines in Azure and Databricks
Implement scalable ETL/ELT frameworks using Azure Data Factory and Spark
Optimize data structures for performance and reliability
Drive data quality initiatives and manage metadata
Collaborate with teams to define and implement data models
Maintain documentation and enforce engineering best practices
Ensure compliance with security and data privacy standards
Benefits
Mentorship opportunities for career growth
Engagement in diverse projects with cross-functional teams
Focus on establishing engineering best practices
Emphasis on a diverse and inclusive workplace
Support for continuous learning and staying current with technology trends
Full Job Description
We are seeking an experienced Staff Data Engineer to join our Unified Data Platform team. The ideal candidate will design, develop, and maintain enterprise-scale data infrastructure leveraging Azure and Databricks technologies. This role involves building robust data pipelines, optimizing data workflows, and ensuring data quality and governance across the platform. You will collaborate closely with analytics, data science, and business teams to enable data-driven decision-making.
Duties & Responsibilities:
Design, build, and optimize data pipelines and workflows in Azure and Databricks, including Data Lake and SQL Database integrations.
Implement scalable ETL/ELT frameworks using Azure Data Factory, Databricks, and Spark.
Optimize data structures and queries for performance, reliability, and cost efficiency.
Drive data quality and governance initiatives, including metadata management and validation frameworks.
Collaborate with cross-functional teams to define and implement data models aligned with business and analytical requirements.
Maintain clear documentation and enforce engineering best practices for reproducibility and maintainability.
Ensure adherence to security, compliance, and data privacy standards.
Mentor junior engineers and contribute to establishing engineering best practices.
Support CI/CD pipeline development for data workflows using GitLab or Azure DevOps.
Partner with data consumers to publish curated datasets into reporting tools such as Power BI.
Stay current with advancements in Azure, Databricks, Delta Lake, and data architecture trends.
Technical Skills:
Advanced proficiency in Azure 5+ years (Data Lake, ADF, SQL).
Strong expertise in Databricks (5+ years), Apache Spark (5+ years), and Delta Lake (5+ years).
Proficient in SQL (10+ years) and Python (5+ years); familiarity with Scala is a plus.
Strong understanding of data modeling, data governance, and metadata management.
Knowledge of source control (Git), CI/CD, and modern DevOps practices.
Familiarity with Power BI visualization tool.
Minimum Qualifications:
Bachelor's or Master's degree in Computer Science, Data Science, or related field.
7+ years of experience in data engineering, with significant hands-on work in cloud-based data platforms (Azure).
Experience building real-time data pipelines and streaming frameworks.
Strong analytical and problem-solving skills.
Proven ability to lead projects and mentor engineers.
Excellent communication and collaboration skills.
Preferred Qualifications:
Master's degree in Computer Science, Engineering, or a related field.
Exposure to machine learning integration within data engineering pipelines.
Don't meet every single requirement? Studies have shown that women and people of color are less likely to apply to jobs unless they meet every single qualification. At Press Ganey we are dedicated to building a diverse, inclusive and authentic workplace, so if you're excited about this role but your past experience doesn't align perfectly with every qualification in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.
