Category/Area of Expertise: Data-Science & Analytics
Job Requisition: 561382
Address: USA-NC-Salisbury-2110 Executive Drive
Store Code: Arch/Data - Business Intell & Analy (5119291)
Primary PurposeThe Software Engineer III (Senior Data Platform Engineer) is responsible for designing, engineering, automating, securing, and supporting enterprise data platforms built on Microsoft Azure and Azure Databricks. The engineer will drive platform modernization, security, reliability, automation, observability, and cost optimization across Azure and Databricks environments. The position requires strong technical leadership and collaboration with Data Engineering, Cloud Operations, Architecture, Cybersecurity, and other technology teams to deliver scalable, secure and highly available platforms.
Our flexible/hybrid work schedule includes 3 in-person days at our Salisbury, NC office and 2 remote days. Applicants must be currently authorized to work in the United States on a full-time basis.Duties & Responsibilities- Design, deploy, configure, and support Azure infrastructure and data-platform services, including Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS), Azure Key Vault, managed identities and related Azure services.
- Engineer and troubleshoot Azure networking, including VNets, subnets, NSGs, Private Link/private endpoints, DNS, routing, firewalls, Network Security Perimeters (NSP), and secure connectivity between Databricks, storage, applications and enterprise networks.
- Administer and deploy Azure Databricks environments, including workspace configuration, compute, serverless capabilities, Unity Catalog, storage integration, identity and access management, workspace security, policies and platform troubleshooting.
- Develop reusable Infrastructure as Code (IaC) using Terraform and/or Azure Bicep to automate provisioning and configuration of Azure infrastructure, Databricks workspaces, networking, storage, security controls and supporting platform services.
- Implement DevSecOps and CI/CD practices using GitHub, GitHub Actions and/or Azure DevOps, including automated infrastructure deployment, configuration management, testing, security controls and environment promotion.
- Implement and maintain security and governance controls across Azure and Databricks, including Microsoft Entra ID, RBAC, managed identities, service principals, OAuth, Azure Key Vault, Unity Catalog, secrets management, access controls and audit logging.
- Support Databricks/ Unity Catalog capabilities including metastore, catalogs, schemas, workspace bindings, external locations, storage credentials, permissions, and secure access patterns while partnering with data-domain owners responsible for data products and data-access groups.
- Establish monitoring and observability across Azure and Databricks using Azure Monitor, Log Analytics, Databricks system tables and audit logs, Datadog and related technologies, with proactive alerting and operational dashboards.
- Drive platform reliability through incident management, root-cause analysis, vulnerability remediation, patching/upgrades, capacity management, high availability, BCDR, performance optimization and automation of operational activities.Identify and implement FinOps and cost-optimization opportunities across Azure and Databricks, including infrastructure utilization, compute sizing, serverless workloads, storage lifecycle management, stale-resource remediation, tagging and cost attribution.
- Provide technical leadership through IaC/code reviews, platform standards, technical documentation, mentoring, troubleshooting and collaboration with engineering teams and managed-service partners.
Required Qualifications- 7+ years of experience in cloud, infrastructure, DevOps, or platform engineering, with strong hands-on experience designing and supporting enterprise Microsoft Azure environments and meaningful hands-on experience with Azure Databricks platform engineering / administration.
- Strong knowledge of Azure infrastructure and networking, including VNets, NSGs, private endpoints, Private Link, DNS, routing, Entra ID, RBAC, managed identities, Key Vault, ADLS Gen2 and ADF.
- Hands-on knowledge of Azure Databricks, including workspace administration, Unity Catalog, compute, storage integration, identity and access management, networking, security, monitoring and troubleshooting.
- Demonstrated experience with Terraform and/or Azure Bicep, Git-based development, CI/CD pipelines, DevSecOps practices, and automation using PowerShell, Python, Bash or similar technologies.
- Experience operating enterprise platforms with an understanding of security, governance, observability, reliability, incident management, BCDR, performance optimization and cloud cost management.
- Strong communication and problem-solving skills.
Preferred Qualifications- Microsoft Azure and/or Databricks professional certifications.
- Experience managing multiple Azure Databricks workspaces and implementing enterprise standards for workspace provisioning, Unity Catalog, security, governance, networking, and platform administration.
- Advanced experience with Terraform, Azure Bicep, Databricks Terraform Provider, GitHub Actions, Azure DevOps, Databricks CLI/APIs, and automated deployment patterns.
- Experience with Databricks capabilities such as Databricks SQL, Delta Lake, Lakeflow, Asset Bundles, serverless compute, AI/BI, Genie, MLflow, or Mosaic AI.
- Experience implementing enterprise observability using Datadog, Azure Monitor, Log Analytics, Databricks system tables, and audit logs.
Salary Range: $125,040.00 - $187,560.00
Actual compensation offered to a candidate may vary based on their unique qualifications and experience, internal equity, and market conditions. Final compensation decisions will be made in accordance with company policies and applicable laws.
