About the RoleWe're looking for an Applied AI Engineer to build production systems that help AI-powered go-to-market workflows improve over time. You will connect agent behavior, customer and operator feedback, evaluation, experimentation, and business outcomes to make these systems more effective, reliable, and responsive to evolving customer needs.
This is a deeply technical, cross-functional role with end-to-end ownership of the agent improvement loop: understand production behavior, identify failure modes, improve how the system decides or acts, and validate the resulting impact.
You will partner with Engineering, Product, Data Science, Sales, and B2B Marketing to turn real-world signals into safer, more effective agent behavior and measurable improvements in customer engagement, conversion, qualified pipeline, and team productivity.
In this role, you will:- Own the production improvement loop across agent behavior, customer and operator feedback, evaluation, experimentation, and verified business outcomes.
- Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and downstream outcomes can be understood in context.
- Define meaningful quality standards, representative evaluation datasets, regression coverage, and production monitoring for real GTM workflows.
- Investigate why agents underperform across context, knowledge, instructions, tools, routing, guardrails, or workflow design.
- Design and ship targeted behavior improvements, including changes to prompting, context construction, decision logic, tool use, and human-review paths.
- Build backend services, APIs, data models, and feedback pipelines that make agent behavior observable, steerable, and reproducible.
- Run controlled experiments, production replays, or staged rollouts to measure whether changes improve quality and downstream business results.
- Partner with Product, Data Science, Sales, and B2B Marketing to prioritize high-value problems and define customer and business success.
- Ship with appropriate safeguards for privacy, security, reliability, human oversight, and safe operational rollout.
You might be a great fit if you have:- 4+ years of software, backend, applied AI, or product-engineering experience building reliable production systems.
- Experience building AI agents, LLM-powered applications, or other model-driven workflows that operated on real production traffic.
- Experience diagnosing and improving agent behavior using production traces, user feedback, evaluation, experimentation, or careful systems design.
- Practical experience with evaluation design, regression testing, human or model grading, online quality signals, or controlled experiments.
- Strong backend engineering skills across Python, APIs, data pipelines, stateful workflows, and production services.
- Strong product judgment and the ability to connect technical changes to customer experience, conversion, qualified pipeline, or operational efficiency.
- Comfort working across model behavior, context, knowledge, tools, workflow state, and human-in-the-loop decisions.
- The ability to work closely with technical and non-technical partners across Engineering, Product, Data Science, Sales, and B2B Marketing.
- A pragmatic mindset: you can scope ambiguous problems, ship useful improvements, and build toward a durable system.
You Might Thrive If- You want to build AI systems that improve from real usage instead of stopping at a successful prototype.
- You enjoy tracing messy production failures back to the decision, context, tool interaction, or workflow issue that caused them.
- You think evaluation is valuable when it helps teams make better product decisions and improve real outcomes.
- You are comfortable moving between applied AI, backend engineering, experimentation, and product judgment.
- You like partnering with operators, sales teams, and marketers to understand the work your systems need to improve.
- You can move from an ambiguous problem to a focused experiment, measured result, and durable implementation.
- You care about trustworthy deployment, clear human-review paths, and reliable production systems.
Nice to Have- Experience building agent evaluation, observability, experimentation, or AI infrastructure products.
- Experience with production replay, LLM grading, human-labeled datasets, shadow evaluation, or staged rollout.
- Experience improving model or agent behavior through context design, prompting, tools, decision logic, or feedback loops.
- Experience with sales, B2B marketing, revenue, CRM, campaign, or other GTM-facing systems.
- Experience measuring customer engagement, qualified pipeline, conversion, or operational efficiency.
