Research Scientist / Engineer, InferenceSan Francisco | Permanent The Opportunity You'll own latency and throughput across two serving stacks. One handles production traffic for customers. The other powers test-time compute and RL rollouts for the research team, so every gain you make speeds up the research itself.
What You'll Do - Cut latency and raise throughput for model serving at very high concurrency
- Speed up RL rollout generation and scale test-time compute for research
- Tune batching, caching, load balancing and parallelism across the serving stack
- Write and profile GPU kernels in CUDA and Triton
- Apply quantization, distillation, speculative decoding and low-precision numerics
- Shape model and hardware co-design with serving cost in view from day one
- Own benchmarking, testing and reliability for every inference service
What You'll Bring Essential - Production experience with vLLM, TensorRT-LLM or similar at high concurrency
- Hands-on continuous batching and KV cache management
- Quantization for inference in production
- Profiling and performance analysis with CUDA and Triton
Desirable - Speculative decoding or multi-node serving
- Performance tuning on H100 or B200 hardware
- Kubernetes and production infrastructure
We encourage you to apply even if you don't meet every requirement. The right mindset matters as much as the right CV.
Apply Now If you've spent your career making models fast, this is a place where that work shapes the research agenda. Send your CV and we'll share the full brief in a confidential conversation
