div:has([data-free-thinking-preview-answer=true])+:is(.text-message,.relative:has(>.text-message))]:-mt-2 grow">
Job Summary
The Azure Databricks Architect will lead the design, development, and optimization of enterprise-scale data platforms and analytics solutions using Azure Databricks. The role will drive data engineering best practices, mentor team members, and collaborate with business stakeholders to deliver scalable, secure, and high-performance data solutions. The position requires strong experience with Azure-based cloud data transformation programs and enterprise-scale Databricks implementations, preferably within the Oil & Gas industry.
Key Responsibilities
• Lead the development of scalable data pipelines and ETL/ELT frameworks using Azure Databricks and PySpark.
• Architect and optimize data ingestion, transformation, and consumption layers.
• Design data models, data dictionaries, and reusable data products.
• Implement data governance, quality, security, and compliance standards.
• Optimize Spark workloads, Databricks jobs, and overall platform performance.
• Collaborate with business users, data scientists, and cross-functional teams to translate business requirements into technical solutions.
• Mentor and guide data engineers and junior team members.
• Drive CI/CD automation and DevOps best practices for data platforms.
• Support Data Lakehouse architecture and enterprise analytics initiatives.
• Lead technical governance and support data platform delivery across enterprise-scale initiatives.
Required Qualifications
• 10+ years of experience in data engineering and data platform delivery.
• Strong hands-on experience with Azure Databricks.
• Strong experience with PySpark and Apache Spark.
• Strong programming experience with Python.
• Strong experience with SQL and T-SQL.
• Hands-on experience with Azure Data Factory (ADF).
• Hands-on experience with Azure Data Lake Storage (ADLS).
• Experience with Azure Synapse Analytics.
• Experience with Delta Lake.
• Experience with Databricks Workflows.
• Strong knowledge of data modeling and data warehousing.
• Experience with Azure DevOps.
• Strong understanding of data governance, security, and compliance.
• Experience with performance tuning and optimization.
• Experience implementing CI/CD and automation frameworks.
• Strong experience leading Azure-based cloud data transformation programs.
• Proven experience delivering enterprise-scale Databricks implementations.
• Experience mentoring technical teams and driving technical governance.
• Strong stakeholder management and client-facing communication skills.
• Experience in the Oil & Gas industry or related energy domain.
Preferred Qualifications
• Experience with Unity Catalog.
• Experience with Apache Airflow.
• Experience with Power BI.
• Strong understanding of Lakehouse architecture.
• Exposure to DataOps or MLOps practices.
