Senior ML / AI Operations Engineer

Role Overview

We are seeking a Senior ML / AI Operations Engineer to architect, deploy, and scale our core AI compute platform. Operating at the unique intersection of large scale machine learning and AI and critical energy infrastructure, our team builds the intelligent platform that powers renewable energy interconnection, autonomous grid dispatch and optimization, -long-term grid planning, and energy-domain foundation models.

In this role, you will bridge the gap between applied research and industrial grade production systems. You will be responsible for orchestrating distributed GPU/TPU training clusters, high-throughput inference serving pipelines, and the agentic execution environments for state-of-the-art agent workflows. You will ensure that our intelligent energy models deliver high reliability, low latency, and operational excellence at petabyte scale.
Key Responsibilities:
Agentic Infrastructure and Tool Execution Platforms

• Architect secure, scalable, and isolated runtime environments (sandboxes, microVMs, or container pools) that allow autonomous AI agents to safely execute code, invoke APIs, and run complex simulations.
• Build orchestrations for multi-step agent workflows (LangGraph, Vertex AI Agent Builder / Reasoning Engine, Ray) with state management, durable execution, and human-in-the-loop (HITL) intervention capabilities.
• Implement deterministic safety guardrails, structured output enforcement, and policy verification to ensure agent actions adhere to strict physics constraints and security perimeters.

High-Performance Training and Inference Serving Pipelines

• Maintain and optimize scalable distributed training pipelines (Ray, PyTorch, Kubernetes) for model training and evaluation.
• Design, deploy, and operate high-throughput, low-latency model serving clusters (e.g., vLLM, Triton Inference Server, TensorRT-LLM) handling real-time grid orchestration and continuous load forecasting.
• Optimize resource allocation, job scheduling, and fault tolerance across heterogeneous hardware fleets to maximize compute efficiency, leveraging automated distributed state recovery and high-throughput data pipelines capable of ingesting gigabytes per second of streaming telemetry.
• Manage model lifecycle management (registry, versioning, canary deployments, model routing, and prompt caching).

Modern LLMOps & GenAI Ecosystem

• Implement end-to-end LLMOps infrastructure for fine-tuning (LoRA, QLoRA, DeepSpeed, Megatron-LM), alignment (RLHF, DPO), and evaluating domain-specific foundation models.
• Build high-performance Retrieval-Augmented Generation (RAG) platforms utilizing distributed vector databases (Milvus, Qdrant, Pinecone) integrated with real-time energy domain knowledge bases.
• Implement runtime guardrails, prompt caching, token cost governance, continuous evaluation, and hallucination monitoring for production GenAI applications.
• Develop automated regression and evaluation frameworks targeting agent trajectories, tool-call fidelity, task completion benchmarks, and domain hallucination rates.

Observability, Reliability & Governance

• Deploy holistic ML and Agent Ops observability stacks (OpenTelemetry, OpenInference, Prometheus/Grafana) to trace hardware (GPU/TPU) metrics, inference latency distributions, data drift, concept drift, and model performance decay, as well as complex multi-step agent reasoning, token economics, and inference latency bottlenecks.
• Enforce rigorous security, data governance, and compliance standards for sensitive infrastructure operational data.
• Lead incident management, root-cause analyses (RCA), and continuous reliability improvements across all production AI services.

Basic Qualifications

• Industry Experience: 5+ years of software engineering or DevOps experience, with 3+ years dedicated to MLOps, LLMOps, or AI platform infrastructure operating at scale.
• Orchestration & Infrastructure: Deep technical mastery of Docker, Kubernetes, Helm, and custom Kubernetes Operators, enterprise cloud ecosystems, and Infrastructure as Code (Terraform).
• ML Frameworks & Libraries: Hands-on experience with modern serving engines and frameworks such as PyTorch, Ray, vLLM, Hugging Face DeepSpeed, and Triton Inference Server.
• Programming Languages: Advanced proficiency in Python (C++ is a strong plus) and familiarity with distributed systems architecture.
• Technical Leadership: Track record of driving architectural decisions, mentoring engineers, and collaborating closely with AI researchers and product teams.

Preferred Qualifications

• Experience with machine learning applied to the energy domain, especially dealing with datasets spanning imagery, time series, grid models and text.
• Direct experience deploying edge AI models to constrained industrial hardware using TensorRT, ONNX Runtime, or OpenVINO.
• Track record of contributions to open-source MLOps or LLMOps frameworks (Ray, vLLM, Kubeflow, LangChain, LlamaIndex).
• Familiarity with agent orchestration frameworks (e.g., LangGraph, AutoGen, LlamaIndex, Semantic Kernel) and LLM evaluation tools.

What we offer

A culture that supports growth, ownership, and meaningful impact, along with:

• Competitive salary and equity
• Medical, dental, and vision coverage
• Generous PTO and flexible hybrid work model
• 401(k) with employer contribution
• Professional development
• The ability to work on important real-world problems within an Alphabet-backed environment

The US base salary range for this full-time position is $166,000 - $244,000 + bonus + equity + benefits. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits.