Bachelor's degree or higher in Computer Science or related field
7+ years of professional software experience; 2+ years in a tech lead role
Experience with high-performance computing and distributed systems
Ability to scale ML training systems and optimize cloud resources
Hands-on experience with job scheduling systems and cloud GPU environments (GCP, AWS)
Deep understanding of distributed computing concepts
Proficiency with ML tools like PyTorch and TensorRT
Responsibilities
Architect and implement large-scale ML training pipelines on GCP or AWS
Enhance existing ML infrastructure for parallel processing
Manage and optimize high-performance computing resources
Develop robust solutions for distributed computing challenges
Design systems for job rescheduling and automated retries
Evaluate storage solutions to improve data throughput
Collaborate with ML researchers to identify and resolve bottlenecks
Benefits
Competitive salary and equity in a seed-stage venture-backed startup
Comprehensive health, dental, and vision insurance
Professional growth through training and mentorship
Daily catered lunches and a fully stocked kitchen
Full Job Description
Position Overview
As a ML Training Infrastructure Engineer, you will architect and build the systems that turn our multi-cloud GPU fleet into a training engine our researchers love. Your charter is singular and broad: own training infrastructure end-to-end so that every GPU is busy, every run is reproducible, and every researcher's next experiment is one command away.
What You'll Do
Scale Distributed Training: Architect and own the infrastructure for large-scale GPU clusters. You'll implement sharding, activation checkpointing, and memory optimization (ZeRO, FSDP) to enable the training of massive multimodal models.
Optimize Researcher Ergonomics: Build a research codebase and job scheduling system (Kubernetes/SLURM) that prioritizes fast iteration, automated retries, and seamless failure recovery.
High-Performance Data Handling: Design high-throughput pipelines to ingest and transform terabytes of multimodal robot data (video, proprioception, 3D signals), ensuring dataloaders never starve the GPUs.
Production Inference: Build low-latency inference pipelines for real-time robot control. You'll apply quantization, distillation, and model compilation (TensorRT, Triton) to move models from the lab to the physical world.
Deep Systems Profiling: Dive into the weeds of GPU utilization, I/O bottlenecks, and memory fragmentation to squeeze every bit of performance out of our expanding compute fleet.
What You'll Bring
7+ Years of Engineering: With a track record of leading technical projects in high-performance computing (HPC) or ML infrastructure.
ML Systems Mastery: Deep experience with PyTorch and distributed training frameworks (DeepSpeed, Accelerate). You understand the nuances of mixed precision and gradient accumulation.
Low-Level Intuition: A fundamental understanding of distributed systems, including race conditions, memory management, and NCCL/inter-node communication.
Ownership Mindset: You don't just "deploy" code; you design, build, and operate systems end-to-end to unblock fast-moving research.
Bonus Points For
Experience with Robotics Data Formats (MCAP, Protobuf) or multimodal models (VLAs).
Deep ML systems experience: custom kernels (Triton), compilers, or runtime optimization.
Experience as a founding or early-stage infrastructure hire.
At Dyna Robotics, we build technology for the real world, which requires a team as diverse as the environments our robots inhabit. We are an equal opportunity employer committed to technical rigor and mutual respect.
Don't let a checklist stop you. Data shows that underrepresented groups often only apply if they meet 100% of the criteria. We value problem-solving and grit over keyword matching. If you're passionate about the intersection of geometry and robotics, we want to hear from you-even if you don't check every box.
