Member of Technical Staff, GPU & Distributed SystemsDepartment: Engineering
Employment Type: Full Time
Location: San Francisco
DescriptionRunning a world model in real time means loading gigabytes of weights into accelerator memory, coordinating execution across GPUs and machines, and keeping sessions responsive as demand changes.
You'll build the distributed cloud systems that make this work. Your scope spans multi-node inference, model loading, weight distribution, and scheduling across Kubernetes clusters and cloud providers. You'll write the software that determines where a model runs, how quickly it starts, and how efficiently it executes.
This is a hands-on systems software role with ownership from design and implementation through benchmarking and production. You'll work across storage, networking, GPU memory, and inference runtimes to reduce cold starts, improve tail latency, and get more useful work from every accelerator.
What you'll work on- Multi-node inference. Build distributed worker initialization, sharded model loading, and coordinated execution across GPUs and machines. Work with model engineers to make tensor, pipeline, and sequence parallelism perform well in production.
- Fast model startup. Engineer parallel checkpoint loading, weight streaming, host-to-device transfers, and GPU snapshot/restore to cut cold starts from minutes to seconds.
- Model distribution. Build transfer, caching, prefetching, and replication systems that move weights and large container images efficiently across clusters, regions, and clouds.
- Scheduling and orchestration. Develop Kubernetes controllers, scheduling extensions, and placement algorithms that account for model residency, accelerator memory, interconnect topology, and session latency across AWS and GPU cloud providers.
- GPU communication. Optimize collective operations and point-to-point transfers using NCCL, NVLink, and RDMA. Reduce synchronization overhead and overlap communication with computation.
- Reliable, efficient serving. Build admission control, backpressure, and coordinated recovery from worker failures. Improve utilization across NVIDIA GPUs, AWS Trainium, and other accelerators.
- You'll build reproducible benchmarks, trace bottlenecks across the stack, and measure improvements in startup time, latency, throughput, and fleet utilization.
What We're Looking For- 3+ years building distributed systems, cloud platforms, or performance-sensitive systems software in production.
- Strong programming skills in Go, Rust, C++, or Python, with a solid understanding of concurrency and resource management.
- Hands-on GPU or ML systems experience, such as model loading, accelerator memory management, distributed execution, or inference serving.
- Strong Linux, networking, and storage fundamentals, with experience diagnosing performance across software and hardware boundaries.
- Experience extending Kubernetes through controllers, operators, or scheduling plugins, or building comparable orchestration systems.
- The ability to reason about partial failures, resource contention, and latency-throughput tradeoffs-and turn that understanding into working software.
- Multi-node inference or training, including tensor, pipeline, sequence, or expert parallelism.
- NCCL, RDMA, InfiniBand/RoCE, GPUDirect RDMA, or GPUDirect Storage.
- CUDA streams, asynchronous execution, GPU profiling, or checkpoint/restore.
- Distributed filesystems, tiered caches, or peer-to-peer model distribution.
- Internals of PyTorch distributed, vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or Ray.
- Topology-aware scheduling or capacity management across heterogeneous accelerators and clouds.
You don't need experience with every technology listed. We value depth in a relevant area, strong systems fundamentals, and the ability to work through unfamiliar problems.
