OverviewAs part of the
Experience Innovation Office team within CE&S, you'll help set the operational strategy and drive execution for technology initiatives impacting our front-line customer-facing roles, including Support Engineers, Supervisors, Customer Success Managers, and Professional Services professionals. You'll be part of a group that connects business strategy to delivery execution, optimizing systems for efficiency and impact, and ensuring our organization is ready for the future of AI-powered customer engagement.
We are looking for a
Technical Solution Management Specialist (TSM) - AI Evaluation, Research, and Technical Architecture Function to help establish the
XIO AI Innovation Lab as a focused pre 0-to-1 research, validation, and handoff capability for CE&S AI innovation. This role is responsible for translating emerging AI signals, customer needs, Nebula/KM opportunities, and frontier technology patterns into validated research, reference architectures, evaluation plans, and Test Kitchen-ready Proof of Concept candidates.
The TSM will implement a unified framework for hypothesis-driven research, comprehensive evaluation, rigorous testing, safe experimentation, and reusable publication while working closely with portfolio, engineering, data science, business, governance, and prototyping teams.
This role will be pivotal in transforming how CE&S evaluates early AI opportunities by ensuring candidate solutions are researched, responsibly evaluated, technically grounded, and packaged for downstream prototyping or product engagement only when the evidence supports it. The TSM will work at the intersection of technology, business, research, and governance, driving innovation while safeguarding customer and employee experiences.
The primary work location for this position is Redmond, Washington, with a hybrid work requirement of three days per week in the office. However, this role is also open to candidates located anywhere in the United States and may be performed remotely.ResponsibilitiesStrategic Planning & Vision- Define and communicate a future-state vision for AI evaluation, research quality, and technical architecture across the XIO AI Innovation Lab.
- Develop and maintain technical blueprints for research, validation, evaluation, experimentation, and Test Kitchen-ready handoff capabilities.
- Define and maintain related processes and playbooks for scaling pre 0-to-1 motions using the lab flow: Signal -> Research -> Validation -> Research Paper -> PoC Candidate -> Test Kitchen.
- Influence technical roadmaps for consistent product evaluation frameworks and tooling.
- Track best practices, Microsoft product updates, frontier AI research, and emerging AI trends to inform lab strategy and publication direction.
- Ensure alignment of evaluation frameworks with business priorities and Responsible AI standards.
Technical Roadmapping & Execution- Build and scale multi-dimensional evaluation approaches for diverse lab-stage AI workloads, including chatbots, customer self-service agents, predictive models, and generative tools.
- Develop automated testing frameworks, including synthetic scenario generation, adversarial stress-testing, and batch evaluation.
- Implement lab-appropriate experimentation patterns, including offline evaluation, synthetic scenario testing, feature-flagged trials where appropriate, and monitoring plans for downstream PoC owners.
- Partner with engineering, data science, and prototyping teams to integrate evaluation tools into prototype scaffolds, CI/CD pipelines where relevant, and Test Kitchen handoff plans.
- Develop reference architectures, data needs assessments, secure data pipeline considerations, hosting-model options, and reusable solution patterns for candidate AI solutions.
- Package validated research into PoC candidate artifacts with value proposition, evidence, architecture, evaluation plan, governance considerations, and owner model.
Cross-Functional Program Management- Coordinate across business, technical, and governance teams to ensure successful delivery of evaluation programs.
- Help establish and operate the XIO AI Innovation Lab as a lightweight research and validation capability, acting as a consultant and trainer for project teams.
- Drive organizational adoption of best practices, shared infrastructure, common metrics, research publication standards, and Test Kitchen handoff discipline.
- Support focused leadership alignment, research reviews, Demo Showcase updates, and handoffs to prototyping or product teams without slowing early-stage research autonomy.
Customer Experience & UX Research- Apply UX research principles to understand lived experiences and translate insights into actionable improvements for AI solutions.
- Collaborate with domain experts, support engineers, CSU/account managers/CSAMs, and business owners to define success criteria and evaluation rubrics.
- Lead direct customer and field-team engagements and collect performance metrics to inform continuous improvement and readiness for PG engagement.
Governance & Responsible AI- Integrate Responsible AI evaluations, safety, ethics, and compliance checks into all testing and experimentation processes.
- Manage gate reviews, documentation, risk findings, and RAI oversight appropriate to lab-stage research and Test Kitchen handoff readiness.
- Foster a culture of "fail fast, learn fast" with accountability and continuous improvement.
Other- Embody our culture and values.
QualificationsRequired/Minimum Qualifications - Bachelor's Degree AND 4+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains.
- OR equivalent experience.
- 2+ years of experience managing cross-functional and/or cross-team projects.
Additional or Preferred Qualifications - Bachelor's Degree AND 8+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains
- OR product development
- OR equivalent experience.
- 6+ years of experience managing cross-functional and/or cross-team projects.
- 1+ year(s) of experience reading and/or writing code (e.g., sample documentation, product demos).
- Proven track record of AI evaluation, testing methodologies, and performance metrics.
- Experience with automated testing pipelines, synthetic data generation, and A/B experimentation.
- Familiarity with Responsible AI frameworks, governance, and compliance standards.
- Demonstrated ability to produce research papers, decision-grade technical artifacts, reference architectures, or reusable solution patterns.
- Ability to adapt strategy based on emerging technology trends, frontier AI research, customer signals, and industry research.
- Comfort operating in ambiguous pre 0-to-1 environments and moving work from research to prototype-ready handoff.
Technical Solution Management IC4 - The typical base pay range for this role across the U.S. is USD $97,600 - $188,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $125,000 - $206,400 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
