Technical Lead Data Engineering
Role Overview
We are looking for a highly skilled Tech Lead Data Engineering to guide our engineering team in building and scaling a robust, modern data platform. In this role, you will be the bridge between architectural blueprints and engineering execution. You will remain deeply hands-on with our core stackDatabricks, dbt, and Apache Airflowwhile mentoring engineers, conducting rigorous code reviews, and ensuring the delivery of high-quality data products.
Key Responsibilities
💻 Technical Leadership & Delivery
• Lead a team of data engineers to execute sprint goals, managing code quality and delivery timelines.
• Act as the subject matter expert (SME) for the team on Databricks, dbt, and Airflow best practices.
• Enforce engineering standards, including code modularity, documentation, and version control.
• Conduct comprehensive code reviews to ensure scalability, security, and performance.
âš™ Hands-on Engineering & Optimization
• Build and maintain production-grade data pipelines using PySpark, Delta Live Tables (DLT), and Spark SQL on Databricks.
• Develop complex dbt models, custom macros, and tests to transform raw data into analytics-ready layers.
• Author and schedule sophisticated Apache Airflow DAGs, utilizing dynamic task mapping and custom operators.
• Optimize pipeline performance by troubleshooting bottlenecked Spark jobs, Z-Ordering Delta tables, and tuning Airflow schedulers.
🚀 DataOps & Governance
• Implement and manage CI/CD pipelines for data deployments using toolsets like GitHub Actions or Azure DevOps.
• Enforce data governance policies, access controls, and lineage tracking via Databricks Unity Catalog.
• Integrate automated data quality testing and alerting mechanisms across dbt and Airflow workflows.
👥 Mentorship & Collaboration
• Mentor and upskill junior and mid-level data engineers through pair programming and workshops.
• Translate architectural designs into actionable, granular engineering tasks and JIRA tickets.
• Collaborate closely with data architects, product managers, and downstream data analysts.
Required Qualifications:
• 10+ years of professional experience in data engineering and backend software development.
• 2+ years of experience in a technical lead, team lead, or mentoring capacity.
Technical Proficiencies
• Databricks: Strong hands-on experience with Delta Lake, Unity Catalog, and optimizing PySpark workloads.
• dbt: Proficiency with dbt Core or Cloud, including advanced macros, packages, and custom testing.
• Airflow: Solid experience writing complex, rel iable DAGs, handling task failures, and managing dependencies.
• Languages: Advanced proficiency in Python and expert-level SQL writing.
• Cloud Platforms: Hands-on experience deploying data solutions on at least one cloud provider (AWS, Azure, or GCP).
Leadership Skills
• Proven ability to guide engineering sprint velocity and resolve technical blockers for a team.
• Excellent verbal and written communication skills to articulate technical trade-offs.
Salary Range- $130,000-$200,000 a year
