Data Engineering Lead

Coates Group

$145K — $165K *
Enterprise Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years in data engineering with a focus on scalable design
  • Experience modernizing enterprise data platforms
  • Proficient in AWS data tools such as S3, Glue, and Redshift
  • Strong coding skills in Python and SQL for production data pipelines
  • Knowledge of workflow orchestration, preferably with Airflow
  • Familiarity with infrastructure-as-code using Terraform or AWS CDK
  • Understanding of data warehousing concepts and related platforms

Responsibilities

  • Architect a scalable AWS data platform for analytics and AI
  • Implement reliable data pipelines for batch and near real-time processing
  • Define enterprise standards for data architecture and performance
  • Lead technical decisions on tooling, infrastructure, and design
  • Implement CI/CD pipelines for data platform automation
  • Enable quality governed data products via semantic layers and curated datasets
  • Provide technical leadership and mentoring to data engineers

Benefits

  • Flexible work schedule with 3 days in office and 2 days remote
  • Opportunities for professional development and team mentoring
  • Access to cutting-edge technologies and data capabilities
  • Involvement in strategic decision-making for data governance
  • Collaborative work environment with cross-functional partnerships
Full Job Description
Design, build, and lead the evolution of Coates' next-generation enterprise data platform on AWS, establishing scalable and secure foundations for how data is integrated, governed, transformed, and consumed across the organization. This role enables business intelligence, analytics, operational decision-making, and future AI capabilities through hands-on technical leadership, architecture design, and cross-functional partnership. Responsibilities • Architect and evolve a scalable, cost-efficient AWS data platform that enables enterprise analytics, reporting, and future AI capabilities • Design and implement reliable, production-grade data pipelines and processing frameworks for batch and near real-time data • Define enterprise standards for data architecture, modeling, observability, reliability, and platform performance • Lead foundational technical decisions across tooling, infrastructure, data design, scalability, and operational efficiency • Implement and maintain infrastructure-as-code, CI/CD pipelines, and automated deployment practices across the data platform • Enable high-quality, accessible, and governed data products through scalable semantic layers, curated datasets, and transformation standards • Provide hands-on technical leadership, mentoring, and architectural direction for a growing team of data engineers and cross-functional stakeholders Capabilities • Ability to design scalable, secure, and cost-efficient data architectures that support enterprise analytics and future AI initiatives • Strong technical problem-solving and decision-making skills, including balancing tradeoffs between scalability, complexity, performance, and cost • Ability to establish engineering standards, operational best practices, and reliable platform governance processes • Strong understanding of CI/CD pipelines, modern software engineering practices, and production support models • Experience enabling accessible, high-quality data products for analytics, reporting, and downstream business consumers • Ability to work collaboratively across technical and business stakeholders while providing hands-on technical leadership and direction • Exposure to streaming technologies (e.g., Kafka or Kinesis), transformation frameworks such as dbt, multi-cloud environments, and/or ML and data science workflows is beneficial Qualifications • 5+ years of experience in data engineering with strong expertise in scalable system and platform design • Proven experience building, modernizing, or significantly evolving enterprise data platforms and architectures • Deep hands-on experience within the AWS data ecosystem, including production-scale data lake or lakehouse environments utilizing technologies such as S3, Glue, Spark, Athena, and/or Redshift • Strong proficiency in Python and SQL with experience developing production-grade data pipelines and transformation workflows • Experience implementing workflow orchestration solutions, preferably Airflow • Experience with infrastructure-as-code and automated deployment practices using tools such as Terraform or AWS CDK • Familiarity with modern data warehousing and lakehouse concepts, including dimensional modeling and platforms such as Snowflake, Redshift, or Databricks *This role will be 3 days in office in the West Loop and 2 days remote. $145,200 - $165,000 a year Full Salary Range: $145,200/year (minimum), $155,100/year (midpoint), $165,000/year (maximum). Pay is based on relevant experience, skills, education, internal equity, and market data. Well-qualified candidates can generally expect offers around the midpoint. Candidates who meet the minimum qualifications but have more limited directly relevant experience for this specific role are typically placed nearer the minimum, while highly experienced candidates with strong role alignment may be placed closer to the maximum.

Similar Jobs

More Jobs at Coates Group

More Enterprise Technology Jobs

Find similar Data Engineering Lead jobs: