Position: DevOps EngineerThe
DevOps Engineer's roles and responsibilities are multifaceted, requiring technical agility and operations mindset simultaneously. It is essential to have communication and coordination skills to successfully integrate various functions in a coordinated manner and deliver the responsibilities to the company's needs.
Responsibilities- Develop and maintain CI/CD pipelines in Azure DevOps to deploy notebooks, jobs, and workflows to Azure Databricks.
- Integrate Databricks Repos with Azure Repos (Git) for source control of PySpark/R notebooks.
- Automate job runs in Databricks using Databricks CLI, REST API, or DevOps tasks.
- Maintain infrastructure-as-code practices (e.g., using Terraform).
- Monitor and troubleshoot pipeline failures, notebook issues, and deployment inconsistencies.
- Collaborate with Data Engineers and Platform teams to maintain development, staging, and production environments.
- Implement best practices for version control, testing, and notebook promotion in a multi-environment setup.
- Store and manage Databricks secrets, tokens, and configuration securely within Azure Key Vault and DevOps variable groups.
- Understand and apply DevOps practices in a data platform context.
Position Requirements- Bachelor's degree in computer science, Information Technology, or related or an equivalent combination of education and experience.
- Minimum of three (3) years of experience in DevOps engineering or a related role, with a focus on Microsoft Azure, including other cloud-based data integration and associated technologies.
- Proficiency with data analytics and AI platforms and automating notebook execution.
- Proficiency in scripting languages for automation and various programming.
Benefits- Comprehensive Benefits - Medical, dental, and vision insurance, employee assistance program, employer-paid and voluntary life insurance, disability insurance, plus health and flexible spending accounts
- Financial & Retirement Support - Competitive compensation, 401k with employer match, and financial wellness resources
- Time Off & Leave - Paid holidays, flexible vacation time/PSSL, and paid parental leave
- Wellness & Growth - Work life assistance resources, physical wellness perks, mental health support, employee referral program, and BenefitHub for employee discounts
