5-7 years of experience with Databricks and Apache Spark
Proficient in Python and SQL programming
Familiarity with Delta Lake, PySpark, and Databricks Workflows
Strong background in ETL/ELT and data pipeline development
Experience with cloud platforms like Azure, AWS, or GCP
Knowledge of Data Lake/Data Warehouse architecture
Understanding of Unity Catalog, Databricks SQL, and data governance
Experience with CI/CD, Git, and DevOps practices
Strong data modeling and performance tuning skills
Proven ability to lead technical teams and engage with business stakeholders
Responsibilities
Design and implement scalable Databricks data solutions
Lead data engineering and migration projects
Develop and optimize data pipelines using PySpark and SQL
Define data architecture and technical standards
Optimize Databricks jobs and Spark workloads
Collaborate with data engineers, architects, and business teams
Provide technical leadership and mentor team members
Benefits
Flexible work environment
Opportunities for professional development
Access to cutting-edge technology
Collaborative team culture
Health and wellness programs
Full Job Description
We are looking for an experienced Databricks Lead Engineer / Architect to design, develop, and implement scalable data engineering and analytics solutions using Databricks and cloud technologies.
Required Skills:
Strong experience with Databricks and Apache Spark
Strong programming skills in Python and SQL
Experience with Delta Lake, PySpark, and Databricks Workflows
Strong knowledge of ETL/ELT and data pipeline development
Experience with Azure, AWS, or GCP cloud platforms
Experience with Data Lake / Data Warehouse architecture
Knowledge of Unity Catalog, Databricks SQL, and data governance
Experience with CI/CD, Git, and DevOps practices
Strong understanding of data modeling and performance tuning
Ability to lead technical teams and work with business stakeholders
Responsibilities:
Design and implement scalable Databricks data solutions
Lead data engineering and migration projects
Develop and optimize data pipelines using PySpark and SQL
Define data architecture and technical standards
Optimize Databricks jobs and Spark workloads
Collaborate with data engineers, architects, and business teams
Provide technical leadership and mentor team members
