Advanced Micro Devices, Inc

Senior GPU Inference Performance Engineer

Advanced Micro Devices, Inc • $130K — $180K *
Santa Clara, CA 95051In-Person
Consumer Technology
Less than 5 years of experience
3 days ago
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 5-7 years of experience in GPU performance engineering or related field
  • Deep proficiency with AMD or NVIDIA profiling tools
  • Understanding of GPU architecture and parallelism
  • Experience with LLM serving frameworks and quantization
  • Solid Python and C/C++ programming skills
  • Ability to communicate complex performance analysis clearly
  • Bachelor's or advanced degree in a technical field preferred

Responsibilities

  • Profile and analyze GPU-accelerated inference workloads end-to-end
  • Optimize performance of AI serving engines like vLLM and SGLang
  • Conduct competitive benchmarks between AMD and NVIDIA
  • Profile distributed inference topologies and optimize networking
  • Analyze overhead from GPU operators and Kubernetes scheduling
  • Build and automate performance benchmarking tools and workflows

Benefits

  • Comprehensive health and wellness programs
  • Retirement savings plan with company match
  • Employee stock purchase plan
  • Flexible work arrangements
  • Professional development and training opportunities
Full Job Description
THE ROLE:

We are looking for a Senior GPU Inference Performance Engineer to own end-to-end performance analysis of GPU-accelerated AI inference workloads. You will profile, diagnose, and explain performance across the full stack, from GPU silicon through the software runtime, and drive competitive positioning against other accelerator vendors. This role sits at the intersection of hardware, systems software, and AI serving frameworks, and requires someone who can go deep on a trace and present findings to product and executive stakeholders.

THE PERSON:

A hands-on performance engineer who is equally comfortable reading a GPU trace and briefing executives. You are curious, evidence-driven, rigorous and you don't stop at "X is faster," you explain why, rooted in hardware and software evidence. You collaborate across hardware, systems software, and AI serving framework teams, communicate clearly in written reports and presentations, and thrive at the intersection of silicon, systems, and AI.

KEY RESPONSIBILITIES:
  • Full-stack GPU profiling: Instrument and analyze inference workloads across AMD Instinct (ROCm, rocProfiler, Omniperf) and NVIDIA (CUDA, Nsight Systems/Compute, DCGM) GPUs. Identify bottlenecks spanning HBM bandwidth, compute utilization, kernel scheduling, memory allocation, and PCIe/Infinity Fabric data movement.
  • AI serving framework performance: Profile and optimize inference engines including vLLM, SGLang, and emerging serving runtimes. Understand KV-cache management, continuous batching, PagedAttention, speculative decoding, and quantization (FP8, MXFP4, INT4) effects on throughput and latency.
  • Competitive performance analysis: Design and execute head-to-head benchmarks (AMD vs. NVIDIA) on standardized LLM workloads. Produce clear, data-backed explanations of why performance differs - attributing gaps to specific hardware features (HBM bandwidth, compute density, interconnect topology), software maturity (kernel libraries, operator fusion, graph compilation), or configuration differences.
  • Multi-server inference networking: Profile and optimize distributed inference topologies including prefill-decode (PD) disaggregation, pipeline parallelism, and tensor parallelism across multi-node clusters. Analyze network-level bottlenecks using RDMA/RoCE traces, NCCL/RCCL collective profiling, and NIC-level counters (Pensando, ConnectX). Quantify the impact of network latency, bandwidth, and congestion on end-to-end inference SLAs.
  • GPU operator and Kubernetes stack: Profile the overhead introduced by GPU operators, device plugins, container runtimes (Docker, containerd), and Kubernetes scheduling on inference latency. Identify and resolve jitter, cold-start, and resource contention issues in production serving environments.
  • Tooling and automation: Build reproducible benchmarking harnesses, profiling scripts, and performance regression dashboards. Automate trace collection and analysis to support continuous performance validation across driver, firmware, and framework updates.


PREFERRED EXPERIENCE:
  • Background in GPU performance engineering, HPC, or systems performance analysis
  • Hands-on proficiency with either AMD (ROCm, rocProfiler, Omniperf/Omnitrace) or NVIDIA (CUDA, Nsight Systems/Compute, NCU) profiling toolchains, with deep understanding of GPU architecture: warp/wavefront execution, memory hierarchy (registers • LDS/shared • L2 • HBM), occupancy, and instruction-level parallelism
  • Experience profiling vLLM, SGLang, or equivalent LLM serving frameworks, including quantization workflows (FP8, MXFP4, INT4, AWQ, GPTQ) and their performance implications
  • Experience with multi-GPU and multi-node inference - tensor parallelism, pipeline parallelism, or PD disaggregation over RDMA/RoCE - including RCCL/NCCL profiling and network tools (perftest, ib_write_bw, tcpdump, Memory Fabric counters)
  • Demonstrated ability to explain performance differences in written reports or presentations - not just "X is faster" but why, rooted in hardware and software evidence
  • Strong Python and C/C++ skills; comfort reading GPU kernel code (HIP/CUDA)
  • Experience with Kubernetes GPU scheduling, MIG, and GPU operator performance, or contributions to open-source inference or profiling projects


ACADEMIC CREDENTIALS:
  • Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field preferred; advanced degree desired


This role is not eligible for visa sponsorship.

#LI-TB1

#LI-Hybrid

Benefits offered are described: AMD benefits at a glance.

About Advanced Micro Devices, Inc

Advanced Micro Devices, Inc. Careers

Join the innovative forefront of technology with a career at Advanced Micro Devices, Inc. (AMD), a leader in semiconductor development. As part of our global team, you will contribute to an organization renowned for its dedication to innovation, leadership, and diversity in the tech industry.

Work You’ll Do

At AMD, we offer job opportunities that push the boundaries of what is possible. Our team is composed of professionals who lead the way in microprocessor and graphics technology, driving industry standards and innovation. With AMD, you will be part of a culture that values growth and professional development, ensuring that every team member has the opportunity to excel.

Transform Your Career

AMD is not just about advancing technology, but also about advancing careers. Whether you are looking for an internship, a full-time position, or leadership roles, AMD provides the platform to propel your career to new heights. Our commitment to professional growth is matched by our dedication to diversity and inclusion, making AMD a place where everyone can thrive.

Innovative Work Environment

Join a team of over 12,000 dedicated professionals at the intersection of technology, industry expertise, and digital innovation. At AMD, you will work on groundbreaking projects that shape the future of computing and graphics. Our collaborative environment encourages networking and the sharing of ideas across teams and disciplines.

Career Development and Benefits

AMD is committed to the development of its employees. We offer robust training programs, including leadership development and diversity training, to ensure our team is equipped for both current challenges and future opportunities. Our benefits package is designed to support the well-being and financial security of our employees and their families.

Explore Job Opportunities

From engineering to marketing, AMD offers a range of career paths that cater to diverse skills and interests. Our hiring process is designed to be transparent and engaging, helping you to understand where you fit within our team and how you can contribute to our collective goals.

Stay Connected

Join Our Team Search open positions that match your skills and interest. We look for passionate, curious, creative, and solution-driven team players. Explore the opportunities to join a company that’s committed to your career growth and to innovation in the technology sector.

Keep Up to Date

Stay ahead with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the people who work here.

Job Alert Emails

Personalize your subscription to receive job alerts, latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding career opportunities that await at Advanced Micro Devices, Inc.

Interview and Resume Tips

Prepare for your future with AMD by accessing resources that help you craft your resume and excel in interviews. Our goal is to help you showcase your best professional self and align your skills with the needs of our dynamic team. At Advanced Micro Devices, Inc., we empower our employees to innovate, lead, and grow. Join us in driving the future of technology while building a rewarding and sustainable career.
Learn more about Advanced Micro Devices, Inc
Size
15,500 employees
Market Cap
$100.9 billion
Industry
Manufacturing & Automotive
Net Income
$2.4 billion
Founded
1969
5 Year Trend
+30.9%
Revenue
$9.7 billion
NASDAQ
AMD
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Advanced Micro Devices, Inc

More Consumer Technology Jobs

Find similar Senior GPU Inference Performance Engineer jobs:

NationwideSanta Clara, CA