OverviewInabia is seeking a MLOps Engineer (Databricks/AWS) to design, develop, and maintain end-to-end MLOps pipelines on Databricks running on AWS. This is a senior-level role requiring 10+ years of experience across the full ML lifecycle, from feature engineering and model training through deployment, monitoring, and retraining. The position is fully onsite and is open to candidates local to Columbus, OH; Dallas, TX; or Atlanta, GA.
Responsibilities- Design, develop, and maintain end-to-end MLOps pipelines on Databricks running on AWS.
- Build and automate machine learning workflows covering data preparation, feature engineering, model training, evaluation, deployment, and monitoring.
- Deploy and manage ML models using MLflow Model Registry and Databricks Model Serving.
- Develop and maintain CI/CD pipelines for ML solutions across development, staging, and production environments.
- Collaborate with Data Scientists to productionize machine learning models and ensure reliable deployments.
- Monitor model health, prediction quality, data drift, and system performance, and implement automated retraining strategies where required.
- Optimize Databricks workloads for performance, scalability, and cost efficiency.
- Implement Infrastructure as Code using Terraform for provisioning and managing Databricks and AWS resources.
- Ensure platform security, governance, and compliance using Unity Catalog and AWS IAM.
- Troubleshoot production issues, perform root cause analysis, and continuously improve platform reliability.
- Document MLOps processes, deployment standards, and operational best practices.
Key Qualifications- 10+ years of experience in MLOps, data engineering, or machine learning engineering roles.
- Strong hands-on experience with Databricks Machine Learning.
- Proficiency in Python, PySpark, and SQL for developing and operationalizing machine learning solutions.
- Hands-on experience with MLflow for experiment tracking, model registry, model versioning, and model lifecycle management.
- Experience deploying and managing ML models using Databricks Model Serving and batch inference pipelines.
- Strong understanding of the end-to-end ML lifecycle, including feature engineering, model training, validation, deployment, monitoring, and retraining.
- Experience building scalable ML pipelines using Databricks Workflows and Delta Lake.
- Hands-on experience with AWS services including S3, IAM, EC2, Lambda, ECR, ECS/EKS, CloudWatch, and Secrets Manager.
- Experience implementing CI/CD pipelines for Databricks and ML workloads using Git, Bitbucket, Jenkins, and Databricks Asset Bundles (DAB).
- Experience with infrastructure automation using Terraform (Infrastructure as Code).
- Strong understanding of Apache Spark architecture, optimization, and distributed data processing.
- Experience working with Unity Catalog for governance, security, and access management.
- Knowledge of model monitoring, data drift detection, model performance monitoring, and automated retraining strategies.
- Understanding of MLOps best practices including reproducibility, versioning, testing, and governance.
- Strong collaboration, analytical, problem-solving, and communication skills.
Preferred Qualifications- Experience working cross-functionally with Data Scientists, Data Engineers, and Platform Engineering teams in an enterprise environment.
- Familiarity with additional cloud-native or container orchestration tools (e.g., EKS, ECS).
