Bachelor's degree in Computer Science, Engineering, or related field, or equivalent experience
4+ years of experience in software engineering with a focus on ML infrastructure or distributed systems
Hands-on expertise in distributed training frameworks (e.g., FSDP, DDP, ZeRO) and ML frameworks (e.g., PyTorch, Transformers)
Strong understanding of GPU/accelerator performance optimization techniques
Experience designing and operating large-scale systems in production, preferably cloud-native
Excellent problem-solving and communication skills with cross-domain collaboration experience
Responsibilities
Design, build, and maintain distributed training infrastructure for large-scale foundation models
Implement scalable pipelines for fine-tuning across GPU/accelerator clusters
Optimize training performance using techniques like FSDP, DDP, and mixed precision training
Contribute to developer-friendly frameworks and tooling for efficient training workflows
Collaborate with cross-functional teams to ensure systems meet real-world requirements
Research and implement emerging techniques in training efficiency and model adaptation
Participate in code reviews and system design discussions to uphold high engineering standards
Benefits
Competitive compensation package
Opportunity to be part of a rapidly growing startup in a cutting-edge engineering field
Inclusive and supportive work culture fostering learning and growth
Exposure to various ML startups, providing unique learning and networking opportunities
Full Job Description
THE ROLE
As a Software Engineer at on the Training Infrastructure team, you'll architect and lead development of our training platform, supporting top tier research engineers and model developers. You'll make key technical decisions for the infrastructure enabling developers to deploy, scale, and monitor their workloads with high performance and reliability. You'll own scheduling, storage, networking, reliability, and observability of technical systems in the training stack
EXAMPLE INITIATIVES
Take a look at what we've built so far:
Overview of the product so far
Training docs overview
Story of the Training product
Research we've done
RESPONSIBILITIES
Design and architect scalable infrastructure systems for our ML training platform (e.g. scheduling, storage, and networking)
Partner closely with developers and research engineers to translate complex training requirements into technical solutions
Design and architect a global training scheduler
Design and architect reinforcement learning systems and continuous learning pipelines
Drive long-term improvements to improve reliability of systems and velocity of development
Partner closely with SRE and Capacity teams to unlock state of the art training infrastructure
Make critical architectural decisions balancing performance with system reliability
Lead technical discussions and mentor junior engineers on infrastructure best practices
Contribute to long-term technical strategy and infrastructure roadmap
REQUIREMENTS
Bachelor's degree or high in Computer Science or related field
Proficiency in Go, with
Deep expertise with Kubernetes in production environments
Advanced understanding of distributed systems concepts and performance tuning
Proven experience designing observability systems
Experience with ML/AI workloads and MLOps platforms
NICE TO HAVE
Experience with distributed storage systems
Python experience a plus
Extensive experience with major cloud providers (AWS, GCP) and neo-cloud providers (Crusoe, DigitalOcean, Nebius)
Experience with workload orchestration platforms like Temporal or Airflow
Familiarity or experience with the open source training stack and frameworks (NCCL, PyTorch, Megatron, NemoRL, VeRL, Axolotl, HF Trainer) and distributed training techniques (FSDP, DeepSpeed).
Experience developing AI products, tooling, or agents
BENEFITS
Competitive compensation, including meaningful equity
(U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
Paid parental leave
Fertility and family-building stipend through Carrot
(U.S. only) Company-facilitated 401(k)
Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
