Azure DevOps Engineer with Databricks

Purple Drive Technologies

• $90K — $120K *
Louisville, KY 40214In-Person
Information Technology
Less than 5 years of experience
More than 3 months ago
Job Overview by Ladders

Qualifications

  • 5-7 years of experience with Azure DevOps
  • Hands-on expertise with Azure Databricks
  • Proficient in Python, SQL, and PySpark
  • Experience with Terraform or similar IaC tools
  • Familiarity with Git workflows and branching
  • Knowledge of Azure services like Data Lake and Key Vault
  • Experience with CI/CD pipelining for data platforms

Responsibilities

  • Design and implement CI/CD pipelines for Databricks using Azure DevOps
  • Manage source control integration between Databricks and Git
  • Automate deployment of Databricks clusters and jobs with IaC tools
  • Monitor Databricks environments for performance and cost optimization
  • Collaborate with teams to optimize development and deployment
  • Ensure compliance with security standards in Azure environments

Benefits

  • Opportunity to work with cutting-edge data technologies
  • Collaborative and innovative team environment
  • Focus on professional growth with skill development
  • Flexible work arrangements to support work-life balance
  • Potential to influence broader data architecture and strategy
Full Job Description
Role: Azure DevOps Engineer

Location: LOUISVILLE, KY

Responsibilities

  • Design and implement CI/CD pipelines for Databricks notebooks and workflows using Azure DevOps.
  • Manage source control integration between Databricks and Git repositories.
  • Automate deployment of Databricks clusters, jobs, and libraries using Infrastructure as Code (IaC) tools like Terraform or ARM templates.
  • Monitor and optimize Databricks environments for performance and cost efficiency.
  • Collaborate with data engineering and analytics teams to streamline development and deployment processes.
  • Ensure compliance with security and governance standards in Azure and Databricks environments.


Required Skills

  • Strong experience with Azure DevOps (Pipelines, Repos, Artifacts).
  • Hands-on experience with Azure Databricks (clusters, jobs, notebooks).
  • Proficiency in Python, SQL, and PySpark for data processing.
  • Knowledge of Terraform or other IaC tools for Azure resource provisioning.
  • Familiarity with Git workflows and branching strategies.
  • Understanding of Azure services (Data Lake, Key Vault, Storage Accounts).
  • Experience with CI/CD automation for data platforms.


Nice-to-Have

  • Experience with MLflow for model tracking and deployment.
  • Knowledge of Azure Monitor and Log Analytics for observability.
  • Exposure to data governance tools like Unity Catalog.
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Purple Drive Technologies

  • Data Modeler
    $100K — $130K *
    Los Angeles, CA 90011 (Los Angeles County)
    Reposted Today
    Finance & Insurance
    In-Person
  • ServiceNow Architect
    $120K — $150K *
    Malvern, PA 19355 (Chester County)
    Today
    Enterprise Technology
    In-Person
  • Guidewire Lead Developer
    $100K — $130K *
    Plano, TX 75025 (Collin County)
    Today
    Finance & Insurance
    In-Person
  • Procurement Engineer
    $70K — $95K *
    Loveland, CO 80538 (Larimer County)
    Today
    Manufacturing & Automotive
    In-Person
  • Data Modeler
    $100K — $130K *
    Los Angeles, CA 90011 (Los Angeles County)
    Reposted Today
    Finance & Insurance
    In-Person

More Information Technology Jobs

Find similar Azure DevOps Engineer with Databricks jobs:

NationwideLouisville, KY