Research Engineer - Pre-training

Role Impact Take Prime Intellect's next open models from architecture experiments through full pre-training runs, scaling toward 1-3T parameters. You'll work across the model and training stack, making research decisions and fixing the systems problems that determine whether a run succeeds. Responsibilities • Build and extend the pre-training code in prime-rl, taking small experiments into full training runs. • Evaluate model architecture changes, including MoE and attention designs, against model quality and training cost. • Profile training jobs and improve end-to-end throughput, working with systems engineers on hardware-aware execution. • Make long runs resilient to failures, with reliable checkpointing and recovery. • Investigate training dynamics and numerical stability, checking that optimizations preserve convergence. • Run controlled ablations to guide larger experiments, keeping the setup and results reproducible. • Work with data and evaluation researchers on training mixtures, including midtraining and long-context experiments. Requirements • Hands-on experience training language models, including modifying training code and following the effects through actual runs. • Strong Python skills and experience with PyTorch or a comparable framework, with enough systems understanding to debug distributed training. • A practical understanding of model architectures and optimization, including their effect on convergence. • Ability to design an ablation that answers a research question and explain what its results do and don't show. • Good judgment about what to investigate yourself and when to bring in a colleague. Benefits and Perks • Cash Compensation Range of $150-350k, including equity incentives, aligning your success with the growth and impact of Prime Intellect. • Flexible work arrangements, with the option to work remotely or in-person at our offices in San Francisco. • Visa sponsorship and relocation assistance for international candidates. • Quarterly team off-sites, hackathons, conferences and learning opportunities. • Opportunity to work with a talented, hard-working and mission-driven team, united by a shared passion for leveraging technology to accelerate science and AI. If you're excited about the opportunity to build the foundation for the future of decentralized AI and create a platform that empowers developers and researchers to push the boundaries of what's possible, we'd love to hear from you.