Full Job Description
Do you want to build the agentic platform that millions of administrators rely on to run their cloud desktops? The AWS WorkSpaces Forefront team is reinventing how customers discover onboard and manage WorkSpaces by embedding generative AI directly into our consoles and SDK. As a Senior Software Development Engineer you will be a key contributor to our agentic harness - the always-on platform layer that hosts our LLM agents: the agent loop and orchestration routing across a mixture of specialized experts personalization and memory tool/function calling over MCP retrieval and - a major area of focus for this role - the ranking and evaluation systems that decide what each agent surfaces and prove it's getting better over time. This is the shared substrate that powers AI experiences across multiple WorkSpaces products and you'll help shape and scale it alongside a team of strong engineers.
We are a fungible mix of front-end back-end and ML/AI engineers driving higher customer conversion through AI-powered onboarding building a suite of AI capabilities into our consoles - LLM agents personalization recommendations and experimentation - and unifying our platform at the architectural level. The harness is where these come together: a mixture-of-experts agent platform on Amazon Bedrock (AgentCore/Strands) MCP servers that give agents persistent working memory and typed access to backend services a RAG-backed knowledge base for conversational Q&A ranking and relevance layers that order recommendations and retrieved context and offline/online evaluation harnesses that measure agent quality and gate every change.
At Amazon we hire the best minds in technology to innovate and build on behalf of our customers. As a Senior SDE you'll lead the design and delivery of major components of the harness - with a particular emphasis on ranking and evaluation - tackle ambiguous greenfield problems where no playbook exists and raise the engineering bar across the team through design review mentorship and operational excellence. Because the harness is a multi-product platform you'll partner closely with other engineers UX and applied science across products - your work shows up in customer conversion admin productivity and the velocity of every team building on the platform.
Key job responsibilities
• Lead the ranking and evaluation systems for the harness - relevance and ranking of recommendations and retrieved context and the offline/online evaluation harnesses (golden datasets LLM-as-judge regression and drift detection human-in-the-loop feedback) that measure agent quality and gate changes before they reach customers. -Design the metrics and experimentation that turn agent quality into decision-grade signal (task completion recommendation acceptance conversion) and close the loop from production feedback back into ranking and model/prompt improvements. -Lead the design and delivery of other major harness components - the agent loop orchestration intent routing session and memory tool/function calling or streaming - and contribute significantly to its multi-quarter technical roadmap. -Help define the platform contracts (MCP tool interfaces identity pass-through agent-to-service seams) that let specialized expert agents and product teams build on the harness safely and independently. -Design and scale parts of a mixture-of-experts agent architecture on Amazon Bedrock (AgentCore/Strands) that is low-cost to train and serve with clear patterns for adding new domains. -Contribute to safety and relevance guardrails and the production quality/safety bar alongside the evaluation systems above. -Build highly scalable low-latency fault-tolerant distributed systems and share in operational excellence for the harness (latency availability cost on-call readiness). -Raise the technical bar across the team: participate in and lead design reviews mentor SDEs and influence direction with UX applied science and partner teams across products.
A day in the life
You might spend the morning extending the evaluation harness with new golden cases and a regression gate for the recommendation agent pair with an applied scientist at midday to tune the ranking layer and read out an A/B result and in the afternoon shape the MCP contract for a new expert agent - then write the one-pager that aligns the teams building on it. You'll operate in an agile environment where development cycles are measured in weeks take on the hardest and most ambiguous parts of the platform and be one of the technical leaders the team turns to when the answer isn't obvious. WorkSpaces Forefront owns the admin experience across the WorkSpaces Consoles (Personal Applications) and the WorkSpaces SDK. We're building an agentic control plane for Console and programmatic experiences - an agentic harness (LLM agents routing ranking personalization memory knowledge bases MCP tools and evaluation) plus experimentation and closed-loop learning - while unifying key infrastructure toward a federated console architecture.
BASIC QUALIFICATIONS
- 5+ years of non-internship professional software development experience
- 5+ years of programming with at least one software programming language experience
- 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Experience as a mentor, tech lead or leading an engineering team
PREFERRED QUALIFICATIONS
- 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 168,100.00 - 227,400.00 USD annually
