THE ROLEAs a Global Capacity Manager focused on TPUs at Baseten, you will lead the "engine room" for our non-NVIDIA accelerator fleet, architecting, securing, and optimizing the Google Cloud TPU (and broader emerging accelerator) capacity that powers our customers' AI workloads. You'll own the end-to-end journey of capacity management for this fleet, from securing large-scale TPU pod allocations to building the automation that ensures reliable uptime across multi-cloud environments.
This role is a great fit for entrepreneurial engineers who want to bridge the gap between high-finance asset management and deep infrastructure engineering, with a specific focus on the TPU ecosystem. You will act as the fleet orchestrator for Google's TPU architecture, ensuring Baseten never experiences a capacity outage while maintaining elite unit economics as we diversify beyond NVIDIA.
To be clear, this is a high-stakes engineering role. You will be hands-on with Kubernetes orchestration while also leading specialized pods focused on the latest generation of TPU hardware, like Google's Trillium (v6e) architecture, and partnering closely with the Model Performance (MP) team to ensure workloads are tuned for TPU-specific execution.
EXAMPLE INITIATIVES- The TPU Frontier: Architecting the infrastructure readiness and deployment strategy for Baseten's TPU clusters, including pod slicing and topology planning
- Global Workload Orchestration: Building "multi-cloud capacity management" systems to move customer workloads seamlessly across TPU regions and pod configurations to optimize cost and latency
- Precision Accelerator Triage: Developing automated operators to identify, cordon, and repair unhealthy TPU pods in under an hour
- The Supply Chain of Intelligence: Partnering with leadership and Google Cloud to secure and reserve dedicated TPU capacity for Baseten's largest enterprise customers
RESPONSIBILITIES- Lead Specialized Pods: Act as the lead for TPU pod fleets managing the full lifecycle of acquisition, allocation, and maintenance for those assets
- Advanced Orchestration: Execute complex workload migrations and "sticky" deployment drains across TPU topologies, ensuring deployment scheduling rules meet strict regional and compliance requirements
- Build for Scalability: Design and implement the "next version" of Baseten's capacity management system to handle significant growth in TPU volume alongside our existing GPU fleet
- Financial Modeling: Leverage your understanding of unit economics to build ROI models comparing TPU, GPU, and other accelerator options, ensuring Baseten scales profitably
- Cross-Team Collaboration: Partner closely with MP, SRE, Infra, and FDE teams to ensure workloads are properly tuned for TPU execution and to verify "last mile" follow-through on infrastructure changes
- Incident Response: Lead capacity-crunch response by rapidly reallocating and re-coordinating TPU workloads during high-pressure outages
REQUIREMENTS- Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field
- 5+ years of professional work experience in a high-growth environment, preferably at a hyperscaler (GCP, AWS, Azure) or a specialized accelerator provider
- Hands-on experience with Google Cloud TPUs - pod slicing, ICI (Inter-Chip Interconnect) topology, JAX/XLA, and TPU-specific scheduling and fault handling
- Deep expertise in Kubernetes, including hands-on experience with taints, cordons, node draining, and custom operators
- Demonstrated experience with Go or Python in a production-level environment
- Strong financial literacy and the ability to model complex trade-offs between capacity reliability and cost
- High tenacity and collaborative mindset
NICE TO HAVE- Experience with additional non-NVIDIA accelerators, such as AWS Trainium/Inferentia (Neuron SDK) or AMD Instinct (ROCm)
- Familiarity with multi-accelerator scheduling and cost/performance tradeoff modeling across GPU, TPU, and other platforms
- Prior experience partnering with model performance or ML systems teams to optimize workloads for a specific accelerator
BENEFITS- Competitive compensation, including meaningful equity
- 100% coverage of medical, dental, and vision insurance for employee and dependents
- Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
- Paid parental leave
- Fertility and family-building stipend through Carrot
- Company-facilitated 401(k)
- Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
