About the RoleRadixArk is hiring a Performance Engineer in Palo Alto, CA - someone who can push LLM inference and training systems to the limit across real production workloads.
You'll work on the performance-critical path of SGLang, Miles, and the RadixArk infrastructure stack: latency, throughput, GPU utilization, memory efficiency, scheduling, batching, kernel behavior, distributed execution, and cost-per-token. This is not a generic benchmarking role. You'll be working on the systems that determine whether frontier-scale AI workloads are actually usable, affordable, and reliable in production.
Our customers care about real numbers: P99 latency, TTFT, tokens/sec/GPU, throughput under long-context workloads, cost-per-million tokens, RL rollout efficiency, and training-inference consistency. You'll help us measure, debug, and improve these systems across NVIDIA, AMD, Google TPU, and cloud partner environments.
This role is for someone who loves performance debugging, understands that small systems details can create massive product impact, and wants to work at the frontier of AI infrastructure.
What You'll Do- Analyze and improve performance across SGLang, Miles, and RadixArk production deployments
- Benchmark LLM inference and training workloads across GPUs, TPUs, and cloud environments
- Optimize latency, throughput, memory usage, batching, scheduling, routing, and GPU utilization
- Investigate performance regressions in real customer environments
- Work closely with kernel, runtime, distributed systems, and product engineers
- Build internal tooling for profiling, tracing, benchmarking, and regression detection
- Translate customer workload characteristics into concrete performance tuning strategies
- Help define performance metrics that matter commercially, including cost-per-token and serving efficiency
- Partner with customers and cloud partners on deep technical evaluations
- Contribute performance insights back to open-source SGLang and Miles
What We're Looking For- Strong systems engineering background, especially in performance-critical software
- Experience with GPU systems, distributed systems, inference serving, ML runtimes, or high-performance computing
- Familiarity with profiling tools, performance debugging, tracing, and benchmark methodology
- Comfort working with Python and C++
- Experience with CUDA, Triton, Pallas, ROCm, XLA, or kernel-level optimization is a strong plus
- Understanding of LLM inference concepts such as batching, KV cache, prefill/decode, speculative decoding, MoE, long context, and P99 latency
- Ability to debug messy real-world performance issues across software, hardware, and infrastructure layers
- Strong communication skills - you should be able to explain performance tradeoffs to both engineers and customers
- Prior experience with production AI infrastructure, cloud GPU environments, or open-source ML systems is a plus
CompensationWe offer competitive compensation with meaningful equity, comprehensive health benefits, and flexible work arrangements. Compensation is determined by location, level, and experience.
