Job Description GPU Optimisation Engineer - Real-Time InferenceWant to push GPU performance to its limits - not in theory, but in production systems handling real-time speech and multimodal workloads?
This team is building low-latency AI systems where milliseconds actually matter. The target isn't "faster than baseline." It's sub-50ms time-to-first-token at 100+ concurrent requests on a single H100 - while maintaining model quality.
They're hiring a
GPU Optimisation Engineer who understands GPUs at an architectural level. Someone who knows where performance is really lost: memory hierarchy, kernel launch overhead, occupancy limits, scheduling inefficiencies, KV cache behaviour, attention paths. The work sits close to the metal, inside inference execution - not general infra, not model research.
You'll operate across the kernel and runtime layers, profiling large-scale speech and multimodal models end-to-end and removing bottlenecks wherever they appear.
What you'll work on- Profiling GPU bottlenecks across memory bandwidth, kernel fusion, quantisation, and scheduling
- Writing and tuning custom CUDA / Triton kernels for performance-critical paths
- Improving attention, decoding, and KV cache efficiency in inference runtimes
- Modifying and extending vLLM-style systems to better suit real-time workloads
- Optimising models to fit GPU memory constraints without degrading output quality
- Benchmarking across NVIDIA GPUs (with exposure to AMD and other accelerators over time)
- Partnering directly with research to turn new model ideas into fast, production-ready inference
This is hands-on optimisation work across the stack. No layers of bureaucracy. No "platform ownership" theatre. Just deep performance engineering applied to models that are actively evolving.
What tends to work well- Strong experience with CUDA and/or Triton
- Deep understanding of GPU execution (memory hierarchy, scheduling, occupancy, concurrency)
- Experience optimising inference latency and throughput for large generative models
- Familiarity with attention kernels, decoding paths, or LLM-style runtimes
- Comfort profiling with low-level GPU tooling
The company is revenue-generating, its models are used by global enterprises, and the SF R&D team is expanding following a recent raise. This is growth hiring, not backfill.
Package & location- Base salary: up to ~$300,000 (negotiable based on depth)
- Equity: Meaningful stock
- Location: San Francisco preferred (relocation and visa sponsorship can be provided)
If you care about real-time constraints, GPU architecture, and squeezing every last millisecond out of large models, this is worth a conversation.
All applicants will receive a response.
