Full Job Description
Role Impact
Build the inference systems that power Prime Intellect's RL research, scaling toward 1-3T parameter models on NVL72 clusters. Your work will improve rollout throughput and make frequent model updates practical.
Responsibilities
• Extend prime-rl's vLLM integration to bring new model architectures into RL training, with small-scale validation before larger runs.
• Improve rollout throughput for long-horizon agents, tuning concurrency and KV-cache reuse as context lengths grow.
• Optimize multi-node MoE inference, including expert parallelism and the tradeoffs of separating prefill from decode.
• Reduce trainer-to-inference weight-update overhead through sharding-aware transfer and communication overlap, while keeping rollout workers on the intended policy version.
• Build on FP8 and evaluate lower precision. Use trainer/sampler mismatch checks to catch numerical regressions and measure the effect on RL training.
• Work with model researchers on architecture choices that improve inference efficiency, balancing model quality against execution cost on the target hardware.
Requirements
• Experience building or optimizing GPU inference systems, with an understanding of inference-engine internals.
• Strong Python skills and experience with PyTorch or a comparable ML framework.
• Ability to profile end-to-end workloads and turn measurements into verified performance improvements.
• Experience debugging numerical or integration issues in ML systems.
Nice to have
• Contributions to vLLM or another inference engine.
• CUDA or Triton kernel work, particularly for low-precision inference.
Benefits and Perks
• Cash Compensation Range of $150-350k, including equity incentives, aligning your success with the growth and impact of Prime Intellect.
• Flexible work arrangements, with the option to work remotely or in-person at our offices in San Francisco.
• Visa sponsorship and relocation assistance for international candidates.
• Quarterly team off-sites, hackathons, conferences and learning opportunities.
• Opportunity to work with a talented, hard-working and mission-driven team, united by a shared passion for leveraging technology to accelerate science and AI.
If you're excited about the opportunity to build the foundation for the future of decentralized AI and create a platform that empowers developers and researchers to push the boundaries of what's possible, we'd love to hear from you.
