Senior Performance Engineer - LLM Inference Optimization
Company Name: CarbonForge
Montreal (remote-friendly)
Full-time
Founding team
The role Are you driven by the toughest challenges in AI? We're seeking a Senior Performance Engineer to join our founding team and redefine LLM inference. In this role, you'll architect high-impact solutions across the entire stack-from optimizing kernels, shaping the serving engine, improving the scheduling/orchestration and advancing open-source inference engines. If you're ready to push the boundaries of AI performance, we want to hear from you.
What you'll do
- Design, build, and optimize high-performance inference systems for LLMs and multimodal models, from single-GPU experiments to multi-GPU, multi-node deployments.
- Develop and tune custom kernels and runtime paths (CUDA / TensorRT / Triton / compiler-based optimizations) to push GPU utilization and memory efficiency toward theoretical limits.
- Improve and extend serving engine (e.g., vLLM, TensorRT-LLM, SGLang, or custom runtimes) including batching, KV-cache management, scheduling, and routing policies for heterogeneous workloads (sync, async, batch, streaming).
- Design and operate distributed inference systems: autoscaling policies, load balancing, capacity management, and failure handling under strict latency and reliability SLOs.
- Instrument, profile, and analyze the entire inference lifecycle (from request to GPU to network and storage) to identify bottlenecks and drive systematic performance improvements.
- Collaborate with hardware partners and internal infra teams to co-design software/hardware strategies that improve cost and energy efficiency (e.g., hardware SKUs, topology-aware placement, mixed precision, quantization).
- Explore and implement advanced inference techniques such as speculative decoding, paged KV-cache, tensor/parallellism, LoRA/multi-LoRA serving, mixture-of-experts, and offloading for memory-constrained environments.
- Work closely with research and product teams to translate abstract model and product requirements into concrete constraints and system designs, balancing latency, throughput, quality, and cost.
- Contribute to internal tooling and, when appropriate, open-source projects around inference engines, benchmarking suites, and observability.
What we're looking for - Deep experience building or operating large-scale inference or training systems in production (LLMs preferred, but other large-scale deep learning systems also relevant).
- Understand GPU architectures and performance characteristics (compute vs memory vs bandwidth, kernel launch overheads...).
- Comfortable with the modern inference ecosystem: vLLM, TensorRT / TensorRT-LLM, Triton, SGLang, or similar.
- Strong understanding in systems and distributed-systems fundamentals (data structures, concurrency, scheduling, backpressure, observability, fault tolerance).
- Enjoy debugging performance issues across layers: from kernel traces and CUDA profiles to queueing behavior in the serving layer and autoscaler dynamics.
- Comfortable in a research-adjacent environment: reading papers, prototyping new algorithms, exploring emerging hardware, and turning prototypes into robust systems.
- Can own problems end-to-end, from initial exploration and measurement to deployed systems and long-term operational health.
Nice-to-have experience - Contributions to inference frameworks or libraries (e.g., vLLM, SGLang, TensorRT-LLM, FasterTransformer, custom CUDA/Triton kernels).
- Experience with K8s-based infrastructure (e.g., operators, custom controllers, autoscalers) and modern observability stacks.
- Work on cost or energy-aware scheduling, capacity planning, or optimization for large fleets of accelerators.
- Experience with post-training pipelines (RL, preference optimization, evals) and their interaction with inference infrastructure.
On credentials No PhD required. We care about what you've shipped and what you can prove on a GPU. A pull request closes the loop faster than a CV.
Why join CarbonForge - Mission that compounds. Power is the binding constraint for AI deployment this decade. You'll build the layer that makes every joule count - and every token more useful.
- World-class science bench. Work alongside two Canada CIFAR AI Chairs and core Mila researchers: Pierre-Luc Bacon (UdeM, scientific co-founder) and Christophe Dubach (McGill, scientific advisor). Full support of Mila, Canada's leading AI research institute, with access to its talent pipeline and research bench.
- Real hardware from day one. H100, H200, AMD MI-series, and design-partner engagements already in motion.
- Resources to match the ambition. Seed-funded by strong operators and institutional investors. We have the resources to hire the best, give them the compute they need, and run the long experiments that matter. Founding-engineer equity, direct ownership, no bureaucracy - your code reaches production in weeks, not quarters.
- Remote-friendly. Montreal preferred for regular in-person work with the founding team.
A note on fit If not every bullet above describes you perfectly, apply anyway. We're hiring for trajectory and proof of work, not a checklist. What matters is whether you can ship the outcomes we've described and push us to be better at the work.
