Bachelor's degree in computer science, computer engineering, or related field
5+ years of low-level system programming experience in C/C++ or Rust
Experience with large-scale, high-concurrent production serving
Familiarity with GPU inference engines like vLLM, Triton, or TensorRT-LLM
Strong background in system optimizations including batching and caching
Experience with CI/CD infrastructure for inference
Responsibilities
Architect and implement scalable distributed infrastructure for model serving
Optimize latency and throughput of model inference under production workloads
Build and operate high-concurrency serving systems with excellent reliability
Benchmark, fine-tune, and accelerate inference engines, including GPU kernel work
Develop custom tools for tracing and fixing issues across the stack
Create robust CI/CD infrastructure for seamless deployment and updates
Accelerate research on scaling compute and model-hardware co-design
Benefits
Equity in the company
Comprehensive medical, vision, and dental coverage
Access to a 401(k) retirement plan
Short & long-term disability insurance
Life insurance
Various discounts and perks
Full Job Description
SR. SOFTWARE ENGINEER, TRAINING/INFERENCE (C++)
At SpaceXSI we are building the high-performance inference platform that serves Grok to millions of users every day. As a Software Engineer, Training/Inference (C++), you will design and optimize large-scale model serving systems end-to-end. You will own everything from distributed infrastructure - global KV cache, continuous batching, load balancing, auto-scaling - to deep low-level optimizations, including GPU kernels, quantization, speculative decoding, and tail latency. This is a high-impact role where your work directly determines how fast and reliably users interact with Grok at massive scale. RESPONSIBILITIES:
Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
Optimize latency and throughput of model inference under real production workloads
Build and operate high-concurrency serving systems with excellent tail latency and high reliability at the scale required to serve millions of users
Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation)
Develop custom tools to trace, replay, and fix issues across the full stack - from orchestration down to GPU kernels
Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates
Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems
BASIC QUALIFICATIONS:
Bachelor's degree in computer science, computer engineering, or science discipline
5+ years of experience with low-level system programming experience in C/C++ or Rust
PREFERRED SKILLS AND EXPERIENCE:
Experience with large-scale, high-concurrent production serving
Experience with GPU inference engines (e.g., vLLM, SGLang, Triton, TensorRT-LLM)
Strong background in system optimizations: batching, caching, load balancing, parallelism
Experience with testing, benchmarking, and reliability of inference services
Experience designing and implementing CI/CD infrastructure for inference
COMPENSATION AND BENEFITS:
$200,000 - $375,000 USD
Base salary is just one part of our total rewards package at SpaceXSI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.
