Product Manager, AI Agents

The Role

As Product Manager, AI Agents, you'll define what it means for an AI operator to be truly excellent at operational work.

This isn't a role focused on adding AI features to an existing SaaS product.

AI is the product.

Relay has to understand an incoming problem, reconstruct context scattered across multiple systems, interpret company-specific rules, determine the right next action, use tools to execute it, recognize when something has gone wrong, know when human judgment is necessary, and carry the work through to a measurable outcome.

And it has to do this in environments where the end user may be a logistics coordinator, warehouse operator, customer service representative, dispatcher, buyer, or operations manager, not an AI expert.

You'll own the product systems and experiences that make those agents intelligent, trustworthy, controllable, and increasingly autonomous.

This is a senior individual contributor role for someone who wants to help define a new category of software from first principles.

What You'll Own

Define how BackOps agents work

Own product strategy across the agent lifecycle: how agents understand work, reason about it, take action, handle ambiguity, recover from failure, learn from outcomes, and collaborate with humans.

You'll help answer fundamental questions such as:

• What context does an agent need to make the right decision?
• Which parts of a workflow should be deterministic and which should be agentic?
• When should an agent act autonomously versus request approval?
• How should agents interpret SOPs, business rules, historical outcomes, and unstructured information?
• How should an agent behave when reality doesn't match the documented process?
• How do we make agent behavior configurable without forcing customers to become prompt engineers?

Turn messy operations into great products

The workflows BackOps automates were rarely designed cleanly in the first place.

You'll spend time with customers and operators understanding how work really happens, including the workarounds, exceptions, judgment calls, and tribal knowledge that never appear in an SOP.

Then you'll determine what the best AI-native version of that workflow should be.

You'll work across areas like order management, shipment exceptions, claims, billing, document processing, scheduling, vendor coordination, and entirely new categories we haven't built yet.

The objective isn't to reproduce the manual process step for step.

It's to build a better one.

Own the quality bar for production AI

For traditional software, you can usually define what the product will do.

For AI products, you have to define what good looks like and build systems for continuously determining whether you're achieving it.

You'll own the product strategy for agent quality, partnering with engineering to develop:

• Evaluation frameworks and success criteria
• Golden datasets and representative test cases
• Simulation and regression testing
• Human evaluation and annotation
• LLM-based evaluation where appropriate
• Production quality monitoring
• Failure taxonomies and root-cause loops
• Feedback systems that turn real-world outcomes into better agent behavior

You'll define the domain-specific quality bar; our platform team will help build the infrastructure that makes evaluating and operating agents at scale possible.

Design human + AI collaboration

The goal isn't autonomy for autonomy's sake.

Some tasks should happen automatically. Some require approval. Some need a human because judgment or accountability matters.

You'll define those boundaries and build experiences that allow operators to understand what Relay is doing, trust it, correct it when necessary, and progressively hand over more work as confidence grows.

Great AI UX at BackOps might mean making something disappear entirely. It might also mean showing exactly the right context at exactly the right moment for a five-second human decision.

Knowing the difference is core to this job.

Make intelligence compound

Every workflow creates data about how operations actually function: which exceptions occur, where people intervene, which decisions work, where processes break, and what ultimately resolves the issue.

You'll help determine how BackOps turns that data into a product advantage, improving agent quality, workflow design, and customer outcomes over time.

Measure outcomes, not demos

You'll define and own metrics such as:

• End-to-end resolution rate
• No-touch / autonomous resolution rate
• Task accuracy and quality
• Human intervention and escalation rate
• Time to resolution
• Failure and recovery patterns
• Customer adoption and trust
• Operational impact generated for customers

A compelling demo is useful.

A production system that completes mission-critical work every day is the standard.

What We're Looking For

You may be a strong fit if you:

• Have 6+ years of relevant product-building experience, with meaningful ownership at the Senior or Staff level.
• Have shipped AI/ML, LLM, agentic, automation, or other technically sophisticated products into production and owned the outcome, not simply added an AI feature to an existing roadmap.
• Understand the unique challenges of non-deterministic products: evaluation, quality measurement, grounding, tool use, context, failure modes, guardrails, human escalation, and continuous improvement.
• Have exceptional product instincts and can take a complicated workflow and make the resulting user experience feel obvious.
• Are technically fluent enough to work deeply with engineers and AI practitioners without outsourcing the hard product decisions to them.
• Have built zero-to-one products or substantially rethought how an existing category should work.
• Stay extraordinarily close to users and are comfortable observing messy real-world processes rather than relying exclusively on feature requests.
• Can distinguish between an impressive AI capability and a product customers will trust with real work.
• Move quickly, think rigorously, and are energized by a product surface where many of the right answers haven't been discovered yet.

Particularly Compelling

We'd be especially interested if you've:

• Built core products at a breakout AI company or an exceptional, high-growth B2B software company.
• Directly owned agentic systems, LLM applications, AI workflow products, AI-native interfaces, or evaluation systems.
• Built products where reliability, accuracy, explainability, or trust had meaningful real-world consequences.
• Worked in logistics, manufacturing, commerce, construction, healthcare, field services, or another operationally complex industry.
• Built for users who aren't highly technical and have little patience for software that requires them to change how they work.
• Developed products where software interacts with both digital systems and real-world operational processes.

You don't need to arrive knowing supply chain.

You do need to be the kind of product builder who can become dangerous in a complicated domain very quickly.

This Probably Isn't the Role for You If

• Most of your AI experience is adding a chatbot or generative feature to an otherwise traditional SaaS product.
• You think the objective is to maximize autonomy rather than maximize customer outcomes.
• You prefer product problems where behavior can be completely specified in advance.
• You stay primarily at the roadmap/PRD layer and expect engineering or research to make the difficult AI product decisions.
• Your first instinct is to replicate a customer's existing workflow exactly as it exists today.