Lead Cloud Data Engineer
Position Summary
We are seeking a highly skilled, hands-on Lead Cloud Data Engineer to lead the design, development, delivery, and operational excellence of our enterprise data platforms on Google Cloud Platform (GCP). This role serves as a critical bridge between enterprise architecture and solution delivery, translating strategic data architecture into scalable, secure, reliable, and production-ready data solutions that support both operational and analytical business needs. The successful candidate will provide hands-on development support, technical leadership across the full data lifecycle, including data ingestion, integration, streaming, transformation, orchestration, governance, retention and production operations. Combining deep technical expertise with strong leadership and mentoring capabilities, this individual will lead a team of data engineers, establish engineering best practices, drive delivery excellence, and ensure the performance, scalability, reliability, security, and cost efficiency of enterprise data solutions.
As a data engineer, you will partner closely with architects, product owners, business stakeholders, and platform engineering teams to deliver high-impact data products and capabilities that enable strategic business outcomes. The role requires exceptional problem-solving skills, a passion for engineering excellence, and a proven ability to drive architectural consistency, code quality, automation, and operational maturity across the data ecosystem. The ideal candidate will also stay at the forefront of emerging AI, Generative AI, and agentic technologies, identifying opportunities to improve engineering productivity, automation, data quality, and overall platform efficiency.
Duties & Responsibilities
- Design, develop, test and deploy large scale enterprise data engineering pipelines and data processing ETL/ELT routines in Google Cloud data platform
- Lead, mentor, and develop a team of data engineers, fostering a culture of technical excellence, collaboration, accountability, innovation, and continuous learning.
- Drive the design, development, and deployment of scalable batch, real-time, and streaming data pipelines aligned with enterprise architecture standards, cloud well-architected principles, and business objectives.
- Translate enterprise data architecture patterns into practical, secure, scalable, and production-ready solutions that support operational, analytical, and data product workloads.
- Establish, enforce, and continuously improve engineering best practices, including solution design reviews, code reviews, reusable frameworks and design patterns, modular development, testing standards, CI/CD practices, release management and deployment governance.
- Design and implement enterprise workflow orchestration, scheduling, and automation solutions to enable reliable, efficient, and resilient data processing across the data ecosystem.
- Define and promote reusable architectural patterns, frameworks, and standards that improve solution consistency, maintainability, scalability, and operational efficiency.
- Ensure all solutions adhere to enterprise data governance, data quality, security, privacy-by-design, regulatory compliance, and Data-as-a-Product principles.
- Drive the evaluation, adoption, and implementation of emerging technologies that create measurable business value and improve the effectiveness of the data engineering platform.
- Own and lead production operations, including release management, monitoring, observability, incident response, root cause analysis (RCA), problem management, and continuous improvement.
- Partner closely with support and operations teams to ensure platform stability, reliability, resiliency, and compliance with enterprise change management and operational processes.
- Collaborate with enterprise architects, product owners, business analysts, BI developers, data engineers and cross-functional stakeholders to align data engineering solutions with business priorities, governance requirements, and data product delivery goals.
- Optimize platform performance, scalability, reliability, and cost efficiency through effective use of cloud-native technologies, automation, and engineering best practices.
- Evaluate and leverage emerging AI, Generative AI, and agentic AI capabilities to enhance data engineering productivity, automate operational processes, improve data quality and data cataloging, and accelerate data solution delivery.
Qualifications
Education:
- Bachelor’s degree in computer science, Engineering, MIS, or related field (Master’s preferred).
Experience:
- 5+ years of experience in data engineering, data platform development, and cloud-based data solutions, including demonstrated technical leadership, & team mentoring responsibilities.
- 5+ years of hands-on experience designing, building, and delivering large-scale enterprise data platforms and pipelines on Google Cloud Platform (GCP) or similar.
- 4+ years of hands-on experience with BigQuery, including SQLroutines performance tuning, query optimization, workload management, partitioning, clustering, materialized views, and query optimization.
- 4+ years of hands-on experience with Dataflow (Apache Beam), Dataproc (Managed Apache Spark), Python, PySpark, SQL, and Pandas for large-scale batch, streaming and real-time data processing.
- Proven experience managing cloud infrastructure consumption, capacity planning, environment provisioning, and cloud cost optimization, including reservation and usage optimization strategies.
- Deep expertise with Cloud Storage, Cloud SQL, Datastore, DataStream (CDC), Pub/Sub, and event-driven architectures supporting operational, analytical, real-time, and streaming data integration.
- Hands-on experience with Cloud Run, Cloud Functions, Compute Engine, App Engine and serverless computing architectures supporting batch, real-time, and streaming data processing solutions.
- Advanced experience designing and implementing workflow orchestration and automation solutions using Cloud Composer (Managed Airflow DAG), Cloud Workflows, Cloud Scheduler, and serverless execution frameworks.
- Experience implementing Infrastructure as Code (IaC), CI/CD pipelines, and automated deployment practices using Terraform Cloud and GitLab.
- Demonstrated experience implementing Data Governance, Data Quality, Metadata Management, Master and Reference Data Management, and Data-as-a-Product principles within enterprise data ecosystems.
- Experience working with enterprise data ecosystems and related technologies, including Oracle EDW, PostgreSQL, Kafka, Power BI, and Looker.
- Strong analytical, problem-solving, and troubleshooting skills, with the ability to resolve complex technical challenges in large-scale, production-grade environments.
- Excellent communication and stakeholder management skills, with the ability to influence technical and business teams and effectively communicate complex concepts with diverse audiences.
- Working knowledge of cloud security, networking, and governance, including IAM, VPC networking, firewall management, observability, logging, Data Loss Prevention (DLP), Dataplex (Knowledge Catalog), and enterprise data catalog’ing practices.
- Working knowledge of AI and agentic development capabilities, including Vertex AI, Google Gemini Enterprise, Agent Studio, Model Garden, Agent Development Kit (ADK), and MCP integrations a plus.
Working Conditions
Hybrid work environment with onsite presence required in Newton MA/or St Louis, MO
Physical Requirements
Regularly required to sit, stand, talk, hear, and use hands and fingers to operate keyboards and devices.
Direct Reports
No. Train and Mentor Junior engineers, and contract workers
Competitive salary $127,461 - $163,660 annually.
The actual pay offered will be determined by multiple factors, including but not limited to the candidate’s relevant experience, job-related knowledge, skills, and geographical location. Individual compensation decisions are dependent upon the facts and circumstances of each position and candidate.
Newton Support Center
