Technical Customer Success Manager (AI Tuning)

HeyMilo AI

$90K — $120K *
Technical Services
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Proven technical skills with coding capability as needed
  • Significant experience in prompt engineering and LLM behavior
  • Expertise in agent systems including tool management and orchestration
  • Proficient user of AI tools like Cursor, Claude, Codex
  • Strong debugging skills across AI and system layers
  • Exceptional communication and ownership qualities
  • Ability to thrive under pressure and manage tight deadlines
  • Experience with AI agents in enterprise settings

Responsibilities

  • Collaborate with customers to set and monitor success metrics
  • Track and analyze the performance of AI agents post-deployment
  • Optimize prompts, workflows, and agent behavior to enhance outcomes
  • Iterate on agent logic and ensure effective context management
  • Identify and resolve issues within prompts, integrations, and systems
  • Translate user feedback into actionable improvements for the product

Benefits

  • Ownership of AI performance in real-world applications
  • Engage with meaningful challenges that focus on customer outcomes
  • Visibility and collaboration across product, engineering, and customer success teams
  • Dynamic and fast-paced work environment focused on execution
Full Job Description
The Role

We're hiring a Forward Deployed AI Engineer to own the performance and success of AI agents in production.

This role sits at the intersection of engineering, product, and customer success. You'll work directly with customers post-deployment to ensure their AI agents are performing at a high level and continuously improving.

What you'll do
  • Work closely with customers and customer success to define and track success metrics
  • Monitor performance of deployed AI agents and identify gaps
  • Tune prompts, workflows, and agent behavior to improve outcomes
  • Iterate on agent logic, tool usage, and context management
  • Debug issues across prompts, integrations, and system behavior
  • Translate real-world usage into product and system improvements


What we're looking for
  • Strong technical background with ability to code when needed
  • Deep experience with prompt engineering and LLM behavior
  • Strong understanding of agent systems (tool calling, context management, orchestration)
  • Heavy user of AI tools (Cursor, Claude, Codex, etc.)
  • Ability to debug ambiguous issues across AI and system layers
  • Strong ownership and communication skills
  • Comfortable working under pressure and tight timelines
  • Able to own outcomes, not just tasks
  • Experience implementing agentic workflows or AI agents in enterprise environments
  • Programming fundamentals + Python/JS experience are a huge plus


Bonus
  • Experience with conversational AI across voice, SMS, or chatbots
  • Experience with telephony or voice systems
  • Experience evaluating and improving production AI systems
  • Familiarity with recruiting, HR tech, or workflow automation


Why join

  • Direct ownership over how AI performs in the real world
  • Work on high-impact problems tied to customer outcomes
  • High visibility across product, engineering, and customer teams
  • Fast-paced, execution-driven environment

Similar Jobs

More Jobs at HeyMilo AI

More Technical Services Jobs

Find similar Technical Customer Success Manager (AI Tuning) jobs: