Role description

Principal Azure Databricks handson Architect

Azure Databricks Data Engineering Lakehouse Architecture

Must Have Skills

Strong experience in designing and developing data pipelines and ETLELT workflows using Azure Databricks

Expertise in Apache Spark PySparkScala and SQL for largescale data processing

Handson experience with Delta Lake and Lakehouse architecture

Experience in building scalable optimized and performant data pipelines

Strong understanding of data modeling data integration and big data processing frameworks

Experience with Azure ecosystem ADLS ADF Synapse

Knowledge of data governance security and data quality frameworks within Databricks

Ability to monitor troubleshoot and optimize Databricks workloads and performance

Experience working with crossfunctional teams for integrating data solutions

Proven experience in architectural design and data engineering strategy planning

Secondary Nice To Have Skills NOT MANDATORY

Experience with machine learning workflows on Databricks MLflow model deployment

Familiarity with Unity Catalog data lineage and governance frameworks

Exposure to CICD DevOps and orchestration tools Airflow Azure DevOps

Experience with streaming frameworks Kafka Structured Streaming

Knowledge of BI tools such as Power BI

Domain exposure in Insurance Financial Services

Experience in evaluating and adopting emerging data engineering tools and technologies

Soft Skills

Strong technical leadership and mentoring abilities

Excellent stakeholder management and requirement translation skills

Effective communication and collaboration across business and technical teams

Problemsolving mindset with focus on performance optimization and innovation

Ability to drive best practices code quality and continuous improvement initiatives

Strategic thinking with ability to contribute to enterprise data architecture decisions

Qualifying Questions

1Can you describe your experience in designing endtoend data pipelines using Azure Databricks BronzeSilverGold architecture What scale and complexity did you handle

2How have you optimized performance cost compute query efficiency in Databricks workloads Please share a specific example

3Have you led architecture decisions or mentored teams on Databricks best practices Can you elaborate on your role and impact