Microsoft

Principal Software Engineering Manager - AI Frameworks

Microsoft • $142K — $274K *
Mountain View, CA 94040In-Person
Information Technology
5 - 7 years of experience
Reposted Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • Bachelor's Degree in Computer Science or a related field with 6+ years of engineering experience in languages such as C, C++, C#, Java, JavaScript, or Python.
  • Master's Degree preferred, with 10+ years of software engineering experience and 6+ years in management, or equivalent.
  • Strong knowledge of software engineering principles and computer architecture, especially related to GPU and neural networks.
  • Experience leading teams in performance analysis and optimization for LLMs, AI systems, or HPC workloads.
  • Demonstrated ability to manage cross-functional initiatives and drive scalable AI solutions from research to production.
  • Proven leadership in team development, including hiring, coaching, and performance management.
  • Exposure to deep learning frameworks like PyTorch or TensorFlow and GPU acceleration technologies such as CUDA.

Responsibilities

  • Lead a team of engineers across the AI software stack for large-scale training and inference.
  • Set technical direction for benchmarking and optimizing performance on GPUs and Microsoft hardware.
  • Drive performance by overseeing benchmarking, profiling, and optimization of workloads.
  • Establish monitoring mechanisms to track performance health and ensure hardware efficiency.
  • Collaborate with research, product, and hardware teams to create scalable, production-ready AI solutions.
  • Balance immediate project needs with long-term investment strategies aligned with company goals.
  • Cultivate a strong engineering culture through mentoring, performance feedback, and team development.

Benefits

  • Comprehensive health insurance coverage.
  • 401(k) retirement plan with company match.
  • Generous paid time off and holiday schedule.
  • Professional development and continuous learning programs.
  • Employee resource groups and diversity initiatives.
Full Job Description
Overview

As a PrincipalSoftware Engineering Manager - AI Frameworks on the team, you will lead and grow a group of engineers working across multiple layers of the AI software serving stack, including fundamental abstractions, runtimes, libraries, and application programming interfaces (APIs). You will be responsible for setting technical direction, prioritizing investments, and ensuring the team delivers high-impact performance improvements that enable large-scale model training and inference.

In this role, you will guide the team's work on benchmarking OpenAI and other large language models (LLMs) across GPUs and Microsoft hardware, driving performance optimization, monitoring regressions, and accelerating time-to-deployment. You will partner closely with researchers, product teams, and platform owners to translate performance insights into production-ready improvements that reduce hardware footprint and support Microsoft Azure's capex efficiency goals.

Responsibilities
  • Lead and develop a team of engineers working across multiple layers of the AI software stack to enable large-scale training and inference.
  • Set technical vision and execution strategy for model performance benchmarking, optimization, and deployment across GPUs and Microsoft hardware.
  • Drive performance outcomes by prioritizing and overseeing efforts to benchmark, profile, debug, and optimize training and inference workloads.
  • Own performance health by establishing mechanisms to monitor regressions, measure impact, and continuously improve time-to-deploy and hardware efficiency.
  • Partner cross-functionally with research, product, infrastructure, and hardware teams to deliver scalable, production-ready AI performance improvements.
  • Balance short-term delivery and long-term investments, ensuring the team's work aligns with organizational goals, platform roadmaps, and Azure capex objectives.
  • Build a strong engineering culture through coaching, feedback, hiring, and career development, enabling the team to operate with increasing autonomy and impact.


Qualifications

Minimum/Required Qualifications:
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Preferred:
  • Master's Degree in Computer Science or related technical field AND 10+ years of software engineering experience, including 6+ years in engineering management,
    OR Bachelor's Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management, or equivalent experience.
  • Strong technical foundation in software engineering principles, computer architecture, GPU architecture, and hardware acceleration for neural networks, with the ability to guide teams working in these areas.
  • Experience leading teams responsible for end-to-end performance analysis and optimization of LLMs, AI systems, or HPC workloads, including use of GPU profiling and performance analysis tools.
  • Demonstrated ability to lead cross-team initiatives, align stakeholders, and translate research or platform capabilities into scalable, production-ready solutions.
  • Proven people leadership skills, including hiring, coaching, performance management, and career development, with a track record of building high-performing, inclusive teams.
  • Exposure to AI / ML infrastructure, including DNN or LLM training and/or inference systems, and experience with at least one modern deep learning framework (e.g., PyTorch, TensorFlow, ONNX Runtime).
  • Familiarity with GPU software stacks and acceleration technologies such as CUDA, ROCm, Triton, or equivalent, sufficient to guide technical direction and evaluate tradeoffs.Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.


#AIInfra

Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $142,800.00 - $274,800.00 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000.00 - $304,200.00 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

About Microsoft

Microsoft is an American multinational corporation that develops, manufactures, licenses, supports, and sells a range of software products and services. Microsoft’s devices and consumer (D&C) licensing segment licenses the Windows operating system and related software, Microsoft Office for consumers, and the Windows Phone operating system. The company’s computing and gaming hardware segment provides Xbox gaming and entertainment consoles and accessories, second-party and third-party video games, and Xbox Live subscriptions; surface devices and accessories; and Microsoft PC accessories. Its phone hardware segment offers Lumia smartphones and other non-Lumia phones. Its D&C segment provides Windows Store, Xbox Live transactions, and Windows phone store; search advertising; display advertising; Office 365 Home and Office 365 Personal; first-party video games; and other consumer products and services as well as operating retail stores. Microsoft’s commercial licensing segments license server products, including Windows Server, Microsoft SQL Server, Visual Studio, System Center, and related Client Access Licenses (CALs); Windows Embedded; Windows operating system; Microsoft Office for business, including Office, Exchange, SharePoint, Lync, and related CALs; Microsoft Dynamics business solutions; and Skype. Its commercial segment offers enterprise services, including premier support services and Microsoft consulting services; commercial cloud comprising Office 365 Commercial, other Microsoft Office online offerings, Dynamics CRM Online, and Microsoft Azure; and other commercial products and online services. The company markets and distributes its products through original equipment manufacturers, distributors, and resellers, as well as online.

Microsoft Careers

Join Microsoft today and be part of a company that values innovation, leadership, and diversity in its workforce. As a global leader in technology and digital transformation, Microsoft offers unparalleled job opportunities that propel your career to new heights.

Explore Career Opportunities at Microsoft

Whether you're a seasoned professional looking for your next challenge or a recent graduate eager to start your career, Microsoft has a position that suits your skills and ambitions. We are committed to fostering a culture of growth and learning, where every team member is supported in expanding their horizons.

Internship Programs

Kickstart your career with a Microsoft internship. Our internships provide invaluable workplace experience and networking opportunities in a supportive and dynamic environment. You'll work on real projects, learn from industry leaders, and gain the skills necessary for a successful career in technology.

Employment Benefits

Choosing a career at Microsoft means more than just a job. Our employees enjoy a range of benefits designed to empower them both professionally and personally. These include comprehensive health benefits, flexible working conditions, and opportunities for career advancement through professional development and diversity training.

Inclusive Culture and Diversity

At Microsoft, we believe that innovation comes from diversity of thought and inclusion. We are committed to a workplace where everyone feels valued and inspired. Our leadership is dedicated to fostering an environment where diverse perspectives lead to breakthrough innovations and a competitive edge.

Grow with Us

Career growth at Microsoft is about more than climbing the corporate ladder; it's about continuous learning, expanding your skills, and improving your capabilities. With access to various leadership and training programs, you can evolve as a professional and make a significant impact within the company and on the global stage.

Hiring Process

Our hiring process is designed to identify true potential. Starting with a review of your resume, followed by interviews that assess your problem-solving abilities and cultural fit, we ensure that all candidates have a fair chance to demonstrate their strengths and potential to contribute to our team.

Networking and Professional Development

Microsoft is a place where you can build a professional network that spans the globe. Our employees benefit from connections with top-tier professionals and industry leaders, which opens doors to innovative projects and collaborative opportunities that are second to none.

Join Our Team

If you're ready to take on exciting challenges and make a difference in the world of technology, explore the job opportunities at Microsoft. Search for open positions that match your skills and interests, and prepare to embark on a rewarding career path filled with innovation and opportunities for personal and professional growth.

Stay Connected

Keep up to date with the latest at Microsoft Careers by subscribing to our job alert emails. Get tailored content that aligns with your career preferences and discover the exciting and rewarding opportunities that await at Microsoft.

SEARCH MICROSOFT JOBS

At Microsoft, your future is limitless. Join us in our mission to empower every person and every organization on the planet to achieve more. Your journey with Microsoft starts here.
Learn more about Microsoft
Size
181,000 employees
Market Cap
$1,762.4 billion
Industry
Enterprise Technology
Net Income
$51.3 billion
Founded
1975
5 Year Trend
+15.5%
Revenue
$153.2 billion
NASDAQ
MSFT
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Microsoft

More Information Technology Jobs

Find similar Principal Software Engineering Manager - AI Frameworks jobs:

NationwideMountain View, CA