Experience with ETL/ELT and data pipeline development
Understanding of cloud platforms (Azure/AWS)
Basic knowledge of Machine Learning and AI concepts
Exposure to Generative AI, MLflow, or Vector Search is an added advantage
Responsibilities
Develop and maintain data pipelines using Databricks and PySpark
Work with Python, SQL, and Apache Spark for data processing
Build and optimize ETL/ELT workflows
Work with Azure Databricks or AWS Databricks
Handle data integration, transformation, and performance tuning
Work on AI/ML-related use cases and Generative AI solutions
Collaborate with data engineers, architects, and business teams
Benefits
Opportunity to work with cutting-edge AI/ML technologies
Collaborative environment with cross-functional teams
Exposure to cloud platforms and modern data architectures
Potential for professional development through certifications
Engagement with innovative projects in data engineering
Full Job Description
Designation: Associate Consultant
Experience: 3-6 Years
Job Summary
We are looking for a skilled Databricks Engineer with experience in data engineering, cloud technologies, and AI/ML concepts. The candidate will be responsible for developing and maintaining scalable data pipelines and supporting AI-driven data solutions using Databricks.
Key Responsibilities
Develop and maintain data pipelines using Databricks and PySpark.
Work with Python, SQL, and Apache Spark for data processing.
Build and optimize ETL/ELT workflows.
Work with Azure Databricks or AWS Databricks.
Handle data integration, transformation, and performance tuning.
Work on AI/ML-related use cases and Generative AI solutions.
Collaborate with data engineers, architects, and business teams.
Required Skills
3-6 years of experience in Data Engineering.
Hands-on experience with Databricks and PySpark.
Strong knowledge of Python and SQL.
Experience with ETL/ELT and data pipeline development.
Understanding of cloud platforms (Azure/AWS).
Basic knowledge of Machine Learning and AI concepts.
Exposure to Generative AI, MLflow, or Vector Search is an added advantage.
Good to Have
Databricks certifications.
Experience with Delta Lake and Lakehouse architecture.