Qualifications

• B.S., M.S., or PhD in Computer Science, Electrical Engineering, or related field (or equivalent experience).
• Experience with large-scale ML systems and GPU computing optimization.
• Strong proficiency in Python and C++/CUDA; experience with SGLang, vLLM, PyTorch, or similar frameworks.
• Deep understanding of GPU architecture and parallel computing paradigms.
• Experience deploying LLMs in production with model serving and distributed inference.
• Strong systems-level debugging and profiling skills across various stack layers.
• Familiarity with distributed computing frameworks (Ray) is a plus.

Responsibilities

• Design and optimize LLM inference systems across multi-node clusters to enhance throughput and reduce latency.
• Implement and benchmark inference optimizations for production workloads.
• Profile and analyze inference bottlenecks, considering GPU kernel execution and memory constraints.
• Build evaluation harnesses and frameworks to measure various performance metrics.
• Collaborate with research scientists on integrating model architectures into production systems.
• Investigate and apply techniques from research and open-source to improve inference performance.

Benefits

• Work on cutting-edge LLM optimization with real-world impact.
• Access to substantial GPU resources for experimentation.
• Collaborate with a world-class AI research and engineering team.
• Shape AI system performance for leading semiconductor companies.
• Unlimited PTO and comprehensive benefits (medical, vision, dental, 401k).
• Two engineering-centered offices with amenities like free parking and meals.