The roleOnce a model is deployed, keeping it fast, reliable, and economical at scale is where production inference is won or lost. You'll own the surface that makes that happen: how deployments autoscale, how traffic is routed, how the system fails over, and how workloads scale across clusters and regions. You'll own these as products end to end - both how they work under the hood and how customers configure and observe them - and you'll help set and define the roadmap that infrastructure and product teams alike can build towards. This space is largely still evolving - think Cloud Infrastructure in mid-2000s. Your job is to make it 10x easier to reliably scale and serve AI models in production and set the market standard.
Impact and outcomes you'll drive- You will own how workloads scale and where they land - autoscaling to demand (up under load, down to zero when idle) and a single placement policy expressing region, compliance regime, and capacity preference, with compliance-bound workloads given right-of-way on sensitive capacity.
- You will make production inference reliable by default - every request reaches a healthy replica, rolling deploys never drop traffic, region-aware routing with multi-region / active-active and fallback as first-class policy, and health-aware recovery from stuck or bad replicas.
- You will build the release engine beneath safe rollouts - the traffic-shifting that powers canary/shadow/A/B, warm-ups, drain, and probes.
- You will push the cost/performance frontier for serving AI at scale - latency, throughput, uptime, and cost-efficiency, plus a measurable decline in MTTR through self-serve incident management.
What we're looking for- 8+ years in product management, including deep experience with infrastructure, distributed systems, or ML serving.
- You reason fluently about scaling, routing, failover, and the cost/performance frontier - and you earn the respect of staff engineers doing it.
- You've owned capabilities end to end, backend through UX, rather than a single slice.
- You drive cross-team roadmaps and the dependencies beneath them, and you're at your best defining a category that doesn't fully exist yet.
Bonus: hands-on with GPU infrastructure, Kubernetes, or serving frameworks like vLLM, TensorRT-LLM, or SGLang.
Not for You if- You don't like getting technical.
- You prefer only strategy, UX, or writing great docs over doing whatever it takes to ship great products for customers.
- You lean toward applied AI over building platform, systems, GPUs, models, and scaling platforms and infrastructure.
- You're not interested in the foundational, sometimes unglamorous work of making AI systems reliable and scalable at the infrastructure level.
BENEFITS- Competitive compensation, including meaningful equity.
- 100% coverage of medical, dental, and vision insurance for employee and dependents
- Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
- Paid parental leave
- Fertility and family-building stipend through Carrot
- Company-facilitated 401(k)
- Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
