5-7 years of relevant experience in machine learning or AI systems.
Expertise in debugging and optimizing inference engines like SGLang, vLLM, or TensorRT.
In-depth knowledge of current GPU architectures and frameworks.
Demonstrated ability to design high-traffic serving systems end-to-end.
Excellent problem-solving skills and effective communication.
Responsibilities
Architect and maintain high-performance LLM serving systems in production environments.
Optimize throughput, latency, and cost for open-source LLMs.
Push the boundaries of inference optimization in large language model systems.
Benefits
Flexible work locations: San Francisco or Remote.
Support for special requirements during the interview process.
Full Job Description
Locations: San Francisco or Remote
About The Role
We are specifically seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served.
What You'll Be Doing
Architect and maintain production high-traffic LLM serving systems.
Optimize throughput, latency, and cost for leading open-source LLMs.
What We're Looking For
Strong hands-on experience in LLM inference, with expertise debugging and optimizing major inference engines such as SGLang, vLLM, or TensorRT.
Deep knowledge of state-of-the-art GPU architectures, and effectively exploit them using PyTorch, Triton, CuTe, CUDA, etc.
Proven track record in designing and maintaining end-to-end high-traffic LLM serving systems.
Strong problem-solving skills and ability to communicate technical ideas clearly.
We'd Love If You Have
Experience with Trusted Execution Environments (TEE).
Active contributor to open-source LLM inference engines.
Please let us know if you require any special requirements for your interview and we'll do our best to accommodate.