Advanced Micro Devices, Inc

AI Systems Engineer - AI Model (Training & Inference)

Advanced Micro Devices, Inc • $100K — $130K *
Markham, ON L3R 0G6In-Person
Consumer Technology
5 - 7 years of experience
Reposted Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 5+ years of experience in AI/ML infrastructure with a focus on training and inference.
  • Hands-on experience shipping large language models on real hardware.
  • Expertise in writing GPU kernels for performance optimization.
  • Deep understanding of distributed systems, Kubernetes, and training infrastructure.
  • Familiarity with AMD Instinct hardware, particularly MI-series architectures.

Responsibilities

  • Enable large-scale model training on AMD Instinct GPU clusters while ensuring correctness and throughput.
  • Build and maintain essential training infrastructure, including orchestration and data loading pipelines.
  • Troubleshoot and resolve training-specific challenges across various GPU generations.
  • Optimize communication patterns for training workloads across distributed systems.
  • Establish robust monitoring and compliance infrastructure for training environments.
  • Design and execute validation and testing frameworks for multiple GPU generations.
  • Drive the enablement and optimization of high-performance inference on AMD GPUs.

Benefits

  • Comprehensive health benefits.
  • Retirement savings plan with company match.
  • Work-life balance programs, including hybrid work options.
  • Employee development programs, including training and conference attendance.
  • Paid time off and holiday pay.
Full Job Description
THE ROLE/PERSON:

The AMD AI Group is looking for a Senior Software Development Engineer to own the end-to-end model execution stack on AMD Instinct GPUs - spanning training infrastructure at scale and high-performance inference serving. This role demands someone who has shipped LLMs on real hardware, written GPU kernels that moved production metrics, and built the systems infrastructure (orchestration, storage, monitoring) that keeps thousands of GPUs productive. You will be instrumental in ensuring AMD GPUs are first-class citizens for frontier model training and inference across current and next-generation Instinct accelerators.

KEY RESPONSIBILITIES:

Training Infrastructure & Enablement
  • Enable and optimize large-scale model training (LLMs, VLMs, MoE architectures) on AMD Instinct GPU clusters, ensuring correctness, reproducibility, and competitive throughput.
  • Build and maintain training infrastructure: job orchestration, distributed checkpointing, data loading pipelines, and storage optimization for multi-thousand GPU clusters on Kubernetes.
  • Debug and resolve training-specific issues including gradient norm explosions, non-deterministic behavior across GPU generations, and compute-communication overlap in distributed training (FSDP, DeepSpeed, Megatron-LM).
  • Optimize RCCL collective communication patterns for training workloads, including all-reduce, all-gather, and reduce-scatter across multi-node topologies.
  • Develop monitoring, alerting, and compliance infrastructure to ensure training cluster health, data security, and SLA adherence at scale.
  • Design and build end-to-end validation and testing infrastructure using proxy workloads, synthetic benchmarks, and configurable workload generators to systematically validate platform readiness across AMD Instinct GPU generations.

Inference Optimization & Serving
  • Write and optimize high-performance GPU kernels (GEMM, attention, quantized matmul, GPTQ/AWQ) in HIP, Triton, and MLIR targeting AMD Instinct architectures, with demonstrated ability to outperform open-source baselines.
  • Drive end-to-end inference enablement on new AMD GPU silicon - be among the first to get frontier models running on each new Instinct generation, creating reproducible guides and reference implementations.
  • Optimize inference serving frameworks (vLLM, SGLang, TorchServe) for AMD GPUs: batching strategies, KV-cache management, speculative decoding, and continuous batching for production throughput/latency targets.
  • Develop novel approaches to inference acceleration, including bio-inspired algorithms, SLM-assisted batching, and custom scheduling strategies that exploit AMD hardware characteristics.
  • Build quantization pipelines (FP8, FP6, FP4, GPTQ, AWQ) for production model deployment, ensuring quality-performance tradeoffs are well-characterized across AMD GPU generations.

Cross-Cutting
  • Collaborate with AMD silicon architecture and pre-silicon teams to provide software feedback and validate software stack integration on next-generation Instinct GPU designs for both training and inference workloads.
  • Build observability and automated analysis tooling: log analysis pipelines, anomaly detection, performance baselining, regression detection, and diagnostic workflows for large-scale GPU clusters.
  • Contribute to the open ROCm ecosystem and AMD's developer experience - SDKs, CI dashboards, documentation, and developer cloud enablement.

REQUIRED EXPERIENCE:

Industry experience shipping production AI/ML infrastructure, with hands-on work spanning both training and inference.

PREFERRED EXPERIENCE:
  • Direct experience enabling frontier models (GPT-4 class) on AMD Instinct hardware end-to-end.
  • Background in building anomaly detection, log analysis, or observability systems for large-scale distributed GPU infrastructure.
  • Familiarity with AMD Instinct MI-series architectures (MI300X, MI350X, MI355X) and RCCL communication library.
  • Contributions to open-source AI frameworks (PyTorch, vLLM, SGLang, DeepSpeed, Megatron-LM).
  • Experience designing validation frameworks, proxy benchmarks, or synthetic workload suites for GPU infrastructure at scale.
  • Experience with pre-silicon software validation or hardware-software co-verification workflows.
  • Publications or patents in HPC, ML systems, or GPU kernel optimization.


PREFERRED ACADEMIC CREDENTIALS:
  • Bachelor's or Master's degree or Ph.D in Computer/Software Engineering, Computer Science, or related technical discipline


This role is not eligible for visa sponsorship.

#LI-G11

#LI-HYBRID

Benefits offered are described: AMD benefits at a glance.

About Advanced Micro Devices, Inc

Advanced Micro Devices, Inc. Careers

Join the innovative forefront of technology with a career at Advanced Micro Devices, Inc. (AMD), a leader in semiconductor development. As part of our global team, you will contribute to an organization renowned for its dedication to innovation, leadership, and diversity in the tech industry.

Work You’ll Do

At AMD, we offer job opportunities that push the boundaries of what is possible. Our team is composed of professionals who lead the way in microprocessor and graphics technology, driving industry standards and innovation. With AMD, you will be part of a culture that values growth and professional development, ensuring that every team member has the opportunity to excel.

Transform Your Career

AMD is not just about advancing technology, but also about advancing careers. Whether you are looking for an internship, a full-time position, or leadership roles, AMD provides the platform to propel your career to new heights. Our commitment to professional growth is matched by our dedication to diversity and inclusion, making AMD a place where everyone can thrive.

Innovative Work Environment

Join a team of over 12,000 dedicated professionals at the intersection of technology, industry expertise, and digital innovation. At AMD, you will work on groundbreaking projects that shape the future of computing and graphics. Our collaborative environment encourages networking and the sharing of ideas across teams and disciplines.

Career Development and Benefits

AMD is committed to the development of its employees. We offer robust training programs, including leadership development and diversity training, to ensure our team is equipped for both current challenges and future opportunities. Our benefits package is designed to support the well-being and financial security of our employees and their families.

Explore Job Opportunities

From engineering to marketing, AMD offers a range of career paths that cater to diverse skills and interests. Our hiring process is designed to be transparent and engaging, helping you to understand where you fit within our team and how you can contribute to our collective goals.

Stay Connected

Join Our Team Search open positions that match your skills and interest. We look for passionate, curious, creative, and solution-driven team players. Explore the opportunities to join a company that’s committed to your career growth and to innovation in the technology sector.

Keep Up to Date

Stay ahead with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the people who work here.

Job Alert Emails

Personalize your subscription to receive job alerts, latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding career opportunities that await at Advanced Micro Devices, Inc.

Interview and Resume Tips

Prepare for your future with AMD by accessing resources that help you craft your resume and excel in interviews. Our goal is to help you showcase your best professional self and align your skills with the needs of our dynamic team. At Advanced Micro Devices, Inc., we empower our employees to innovate, lead, and grow. Join us in driving the future of technology while building a rewarding and sustainable career.
Learn more about Advanced Micro Devices, Inc
Size
15,500 employees
Market Cap
$100.9 billion
Industry
Manufacturing & Automotive
Net Income
$2.4 billion
Founded
1969
5 Year Trend
+30.9%
Revenue
$9.7 billion
NASDAQ
AMD
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Advanced Micro Devices, Inc

More Consumer Technology Jobs

Find similar AI Systems Engineer - AI Model (Training & Inference) jobs:

NationwideMarkham, ON