Job Summary
We are seeking an experienced Azure Databricks Architect with strong Oil & Gas domain experience to lead the design, development, and optimization of enterprise-scale data platforms and analytics solutions using Azure Databricks. The role will drive data engineering best practices, mentor technical teams, and collaborate with business stakeholders to deliver scalable, secure, and high-performance data solutions. The position will support Azure-based data analytics ecosystems and enterprise Data Lakehouse initiatives.
Key Responsibilities
• Lead the development of scalable data pipelines and ETL/ELT frameworks using Azure Databricks and PySpark.
• Architect and optimize data ingestion, transformation, and consumption layers.
• Design data models, data dictionaries, and reusable data products.
• Implement data governance, quality, security, and compliance standards.
• Optimize Spark workloads, Databricks jobs, and overall platform performance.
• Collaborate with business users, data scientists, and cross-functional teams to translate requirements into technical solutions.
• Mentor and guide data engineers and junior team members.
• Drive CI/CD automation and DevOps best practices for data platforms.
• Support Data Lakehouse architecture and enterprise analytics initiatives.
• Lead Azure-based cloud data transformation programs and ensure alignment with enterprise architecture standards.
• Provide technical leadership for enterprise-scale Databricks implementations and data platform modernization initiatives.
Required Qualifications
• 10+ years of experience in Data Engineering and Data Platform delivery.
• Strong experience leading Azure-based cloud data transformation programs.
• Proven expertise in enterprise-scale Azure Databricks implementations.
• Strong hands-on experience with Azure Databricks.
• Strong experience with PySpark and Apache Spark.
• Proficiency in Python.
• Strong SQL/T-SQL skills.
• Experience with Azure Data Factory (ADF).
• Experience with Azure Data Lake Storage (ADLS).
• Experience with Azure Synapse Analytics.
• Strong knowledge of Delta Lake.
• Experience with Databricks Workflows.
• Strong understanding of data modeling and data warehousing.
• Experience with Azure DevOps.
• Strong understanding of data governance, security, and compliance.
• Experience with performance tuning and optimization.
• Experience implementing CI/CD and automation frameworks.
• Experience mentoring technical teams and driving technical governance.
• Strong Oil & Gas domain experience.
Preferred Qualifications
• Experience with Unity Catalog.
• Experience with Apache Airflow.
• Experience with Power BI.
• Strong understanding of Lakehouse architecture.
• Exposure to DataOps/MLOps.
• Strong stakeholder management and client-facing communication skills.
