Data Engineering Lead

Coates Group

• $145K — $165K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years in data engineering with expertise in scalable system design.
  • Proven experience in building or modernizing enterprise data platforms.
  • Deep hands-on experience within the AWS data ecosystem, including data lake or lakehouse.
  • Strong proficiency in Python and SQL for pipeline development.
  • Experience with workflow orchestration solutions, preferably Airflow.
  • Knowledge of infrastructure-as-code practices using Terraform or AWS CDK.
  • Familiarity with modern data warehousing concepts and platforms, such as Snowflake or Databricks.

Responsibilities

  • Architect a scalable, cost-efficient AWS data platform for analytics and AI.
  • Design reliable data pipelines for batch and real-time processing.
  • Define enterprise standards for data architecture and platform performance.
  • Lead technical decisions across tooling and operational efficiency.
  • Implement CI/CD pipelines and automated deployment for the data platform.
  • Enable high-quality, governed data products through curated datasets.
  • Provide hands-on leadership and mentorship to a team of data engineers.

Benefits

  • 3 days in-office in the West Loop; 2 days remote work flexibility.
  • Opportunity to lead the evolution of state-of-the-art data practices.
  • Mindshare within a cross-functional partnership framework for analytics.
  • Exposure to cutting-edge technologies in data and AI.
  • Mentorship opportunities in a growing data engineering team.
Full Job Description
Design, build, and lead the evolution of Coates' next-generation enterprise data platform on AWS, establishing scalable and secure foundations for how data is integrated, governed, transformed, and consumed across the organization. This role enables business intelligence, analytics, operational decision-making, and future AI capabilities through hands-on technical leadership, architecture design, and cross-functional partnership. Responsibilities • Architect and evolve a scalable, cost-efficient AWS data platform that enables enterprise analytics, reporting, and future AI capabilities • Design and implement reliable, production-grade data pipelines and processing frameworks for batch and near real-time data • Define enterprise standards for data architecture, modeling, observability, reliability, and platform performance • Lead foundational technical decisions across tooling, infrastructure, data design, scalability, and operational efficiency • Implement and maintain infrastructure-as-code, CI/CD pipelines, and automated deployment practices across the data platform • Enable high-quality, accessible, and governed data products through scalable semantic layers, curated datasets, and transformation standards • Provide hands-on technical leadership, mentoring, and architectural direction for a growing team of data engineers and cross-functional stakeholders Capabilities • Ability to design scalable, secure, and cost-efficient data architectures that support enterprise analytics and future AI initiatives • Strong technical problem-solving and decision-making skills, including balancing tradeoffs between scalability, complexity, performance, and cost • Ability to establish engineering standards, operational best practices, and reliable platform governance processes • Strong understanding of CI/CD pipelines, modern software engineering practices, and production support models • Experience enabling accessible, high-quality data products for analytics, reporting, and downstream business consumers • Ability to work collaboratively across technical and business stakeholders while providing hands-on technical leadership and direction • Exposure to streaming technologies (e.g., Kafka or Kinesis), transformation frameworks such as dbt, multi-cloud environments, and/or ML and data science workflows is beneficial Qualifications • 5+ years of experience in data engineering with strong expertise in scalable system and platform design • Proven experience building, modernizing, or significantly evolving enterprise data platforms and architectures • Deep hands-on experience within the AWS data ecosystem, including production-scale data lake or lakehouse environments utilizing technologies such as S3, Glue, Spark, Athena, and/or Redshift • Strong proficiency in Python and SQL with experience developing production-grade data pipelines and transformation workflows • Experience implementing workflow orchestration solutions, preferably Airflow • Experience with infrastructure-as-code and automated deployment practices using tools such as Terraform or AWS CDK • Familiarity with modern data warehousing and lakehouse concepts, including dimensional modeling and platforms such as Snowflake, Redshift, or Databricks *This role will be 3 days in office in the West Loop and 2 days remote. $145,200 - $165,000 a year Full Salary Range: $145,200/year (minimum), $155,100/year (midpoint), $165,000/year (maximum). Pay is based on relevant experience, skills, education, internal equity, and market data. Well-qualified candidates can generally expect offers around the midpoint. Candidates who meet the minimum qualifications but have more limited directly relevant experience for this specific role are typically placed nearer the minimum, while highly experienced candidates with strong role alignment may be placed closer to the maximum.

Similar Jobs

More Jobs at Coates Group

More Information Technology Jobs

Find similar Data Engineering Lead jobs: