The RoleDisco is a commerce media company run like an AI company. We brought our bidding and ad-serving stack in house. We built an agentic development pipeline where every pull request, every message, and every review flows through our own harness. Our CEO ships models into production through AI agents. That was the first act.
The second act is the part nobody owns yet: the layer that ties all of it together. The model orchestration, the harnesses, the eval loops, the agent graphs, and the routing logic that decides which model answers which problem. Today that layer is a collection of sub-agents pointed at tasks. We want it rethought, productized, and owned.
We9re hiring an AI/ML Product Manager to own Disco9s AI platform end to end. You9re the person who makes sure we9re model-agnostic and harness-agnostic, so when a vendor jacks up prices or a better model ships tomorrow, we swap in a day instead of re-platforming. You own the eval loops that tell us what9s working, the harnesses that make AI safe to push to production, and the agentic pipeline that lets a team of 25 move like a team of 200.
This is a product role with real technical depth. You won9t just write specs and hand them to engineers. You9ll understand the orchestration layer well enough to redesign it, evaluate models and harnesses against each other, and dive into the data and the code to see where tokens are being burned, where an eval is lying, or why a workflow broke. You9re the closest thing Disco has to an owner of the AI operating system.
What You9ll OwnModel Orchestration & Routing- Own the layer that routes every task to the right model: reasoning to the frontier models, volume to the cheap ones, image and creative work to the models that are actually good at it. This is bespoke routing, not one model for everything.
- Keep Disco model-agnostic and harness-agnostic. When a vendor raises prices 10x or a better model drops, we swap without being beholden to any single LLM contract.
- Track the total cost of every model call. Own the LLM budget like it9s your P&L, because it9s about to become one of the largest OpEx lines in the company.
Harnesses & Eval Loops- Own the harnesses that wrap AI so it can ship to production safely: what gets reviewed, what gets verified, what a non-ML user can and cannot push to traffic.
- Build and own the eval loops that tell us what9s actually working. A model, an agent, a workflow, a cron - if it runs on AI, there9s an eval that says whether it9s good, and you own that eval.
- Design eval graphs that catch regressions before they cost money, and the guardrails that keep an enthusiastic experiment from becoming a production incident.
Agentic Pipeline & Automation- Own the agentic development pipeline: the AWS Bedrock harness, the automated PR flow, the QA gates. Make it faster, cheaper, and more reliable.
- Productize the internal AI tooling. Cron generation, workflow automation, memory, context - these are scripts today. They should be products with owners, versions, and evals.
- Attack the silent cost centers. Crons and background jobs that burn LLM tokens are the first target: turn expensive agent loops into cheap, deterministic code where it belongs.
AI Enablement Across the Team- Make every team agentic. When a non-technical teammate wants to build, evaluate, or automate something with AI, your harnesses and loops are what make that safe.
- Own the knowledge of which models are best at what, and keep it current. The model landscape changes monthly; the team shouldn9t have to track it.
- Set the standard for where AI output is trusted and where it gets verified before it ships.
Who You Are- You9ve shipped real AI products or platforms into production, and you can point to the eval loops, harnesses, or orchestration you built around them.
- You think in systems: routing, evaluation, feedback loops, cost. You don9t just add an AI feature, you build the harness that makes it safe and the eval that proves it works.
- You9re technically credible without being a full-time engineer. You can read code, query data, and reason about model tradeoffs, latency, and cost. You don9t need to author the orchestration, but you need to be able to redesign it.
- You9re model-agnostic by instinct. You care about outcomes, not loyalty to a vendor. When a model is bad at a task, you say so and swap it.
- You have strong product judgment and a bias toward shipping. You can take a fuzzy mandate like 9own the AI platform9 and turn it into a roadmap, then execute it.
- You9re comfortable with a level of ambiguity most PMs never see. There9s no playbook for this role and no platform team behind you. You9ll build the thing you want to use.
- You communicate clearly across a small, fast team. You9ll explain a model routing decision to the CEO and a harness change to an engineer in the same week.
- You can dive into the data and the code to understand what9s actually happening for any given workflow, eval, or cost anomaly. No 9that9s someone else9s problem.9
This is a hybrid role based out of our NYC office, with an expectation of 2-3 days in the office each week. Compensation for this role is $190K-$220K OTE, plus meaningful equity. 
