OverviewWe are seeking a Lead Data Platform Engineer to design, build, and optimize scalable data platforms and pipelines that support enterprise analytics and data-driven solutions. This role combines hands-on engineering with technical leadership, driving best practices in data architecture, platform performance, and governance while mentoring engineering teams.
Key Responsibilities- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
- Build and optimize data platforms using Hadoop, Databricks, and cloud-based technologies.
- Integrate structured and semi-structured data into reliable, high-quality data solutions.
- Partner with cross-functional teams to translate business and analytics requirements into scalable engineering solutions.
- Lead technical design discussions and promote best practices in data modeling, performance optimization, and governance.
- Mentor data engineers and contribute to engineering standards, architecture, and platform scalability.
- Support innovation through proof-of-concepts, automation, and continuous platform improvements.
- 8+ years of experience in data engineering, including 2+ years in a technical leadership role.
- Strong Python skills (Pandas, NumPy, PySpark) and experience with Impala.
- Hands-on experience with Hadoop, Databricks, and large-scale data processing.
- Advanced SQL and experience with relational and distributed databases.
- Experience with cloud platforms such as Azure or AWS, including Databricks or Snowflake.
- Strong knowledge of ETL/ELT tools such as Apache Airflow, Apache NiFi, or Azure Data Factory.
- Experience with CI/CD, DevOps practices, and enterprise data platforms.
- Understanding of data modeling, governance, and performance optimization.
Nice to Have- Experience supporting AI/GenAI solutions through scalable data pipelines.
- Knowledge of machine learning workflows, feature engineering, and model serving.
- Experience processing unstructured data and implementing data governance, privacy, and security best practices.
- Strong analytical and problem-solving skills with the ability to communicate effectively across technical and business teams.
