The Role
We are looking for an Inference Runtime Software Engineer to push forward LM Studio's inference stack on-device and in the cloud. You will integrate new inference engines and runtime capabilities, bring up new open-weight models and modalities, and optimize model execution for a wide range of CPU and GPU targets. You will also contribute improvements to the open-source projects we build on.
Qualifications
- Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructure
- Strong programming ability in Python and C++
- Deep understanding of transformer architectures and the mechanics of model inference
- Experience profiling CPU or GPU workloads and reasoning about compute, memory, synchronization, and data movement
- Experience with PyTorch and inference systems such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM
- Strong debugging instincts across model code, runtime internals, operating systems, and CPU or GPU execution
- Takes personal responsibility for the correctness and performance of their work
Bonus Qualifications
- Past contributions to open-source inference runtime projects such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM
Responsibilities
- Maintain and push forward our inference stack on-device and in the cloud
- Bring up new model architectures and multimodal models
- Improve latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimes
- Build runtime capabilities for model loading, batching, scheduling, caching, and distributed execution
- Benchmark and diagnose correctness and performance problems across the inference stack
- Contribute upstream to open-source projects such as llama.cpp and MLX
Benefits
- Competitive salary and equity grants
- Great medical, vision, dental healthcare plans
- Catered team lunch / expensed dinners in the office
- Flexible PTO
- Flexible WFH
- Sun-drenched office in SoHo in NYC
