Full Job Description
We are seeking a Senior Data Software Engineer with Databricks to lead the design, development, and maintenance of scalable data integration pipelines across cloud and enterprise systems. The role focuses on building a Databricks Lakehouse on Azure, driving data ingestion, transformation, orchestration, and automation initiatives across the broader Azure data ecosystem. Responsibilities Lead the design and development of a Databricks Lakehouse on Azure Research source systems, then analyze and document their data structures, data flows and business entities Build batch and near-real-time ingestion pipelines with Databricks Auto Loader and Azure Event Hubs Develop dbt transformations following the medallion architecture Design and maintain an operational data store (ODS) on Databricks Lakebase Implement integrations with Dataverse and the Power Platform Define data engineering standards, code reviews and best practices Ensure data quality, testing and documentation across the platform Requirements 5+ years of data engineering experience, including a technical lead role Expertise in Databricks, Delta Lake and Apache Spark (PySpark) Skills in dbt and Databricks ingestion tools Proficiency in Unity Catalog Advanced SQL and strong Python skills Solid data modeling skills, including dimensional modeling Proven ability to analyze unfamiliar source systems and produce clear data documentation Familiarity with the Azure data ecosystem: ADLS Gen2, Key Vault and Entra ID Experience with Git and CI/CD for data pipelines Flexibility to use AI tools English proficiency at B2 level or higher Nice to have Background in Databricks Lakebase or PostgreSQL Understanding of data governance, Lakehouse and ODS concepts Experience with Dataverse, Power Apps or Power Automate integration Familiarity with Terraform or Databricks Asset Bundles Databricks certifications AI certifications
