LLM Engineer

Clera

$135K — $160K *
Consumer Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 3+ years of hands-on experience with production LLM systems
  • Proficient with modern LLM APIs (e.g., OpenAI, Anthropic, DeepSeek)
  • Experience in building LLM-powered automation for production systems
  • Strong system-design skills for multi-step agent architectures
  • Experience in maintaining evaluation pipelines for LLM outputs
  • Hands-on with LLM observability and monitoring tools
  • Practical experience using the HuggingFace ecosystem

Responsibilities

  • Build LLM-powered agents for decision-making in supply chain processes
  • Design prompts and structured outputs to handle messy real-world data
  • Evaluate LLMs for performance across supply chain workflows
  • Ship LLM features from concept through to production
  • Create frameworks for iterating on prompts and agent behaviors
  • Set up monitoring systems to track system performance and accuracy

Benefits

  • Visa sponsorship available
Full Job Description
About the Role

You'll be building an AI Supply Chain Manager from the ground up at a Y Combinator-backed AI/supply chain startup, owning LLM systems end-to-end-from early prototypes to production. This is a high-autonomy role where your work directly powers real-world supply chain outcomes: purchase orders, transfer orders, and supplier bookings for CPG brands globally.
What You'll Do
  • Build and iterate on LLM-powered agents that decide what to produce, where to make it, and how to move it through factories, warehouses, and channels.
  • Design robust prompts, tool definitions, structured outputs, and multi-step agent flows that handle edge cases in messy real-world supply chain data.
  • Select and evaluate LLMs based on latency, cost, accuracy, and use-case fit across the supply chain workflow.
  • Ship LLM features from concept to production, owning the full pipeline from API integration to tested and evaluated systems.
  • Build and maintain evaluation frameworks to iterate quickly and safely on prompts and agent behaviors.
  • Set up observability, monitoring, and feedback loops to track production accuracy and continuously improve system performance.
What We're Looking For
  • 3+ years of hands-on experience shipping LLM-powered features or systems in production environments (not just demos or prototypes).
  • Proficiency with modern LLM APIs such as OpenAI, Anthropic, DeepSeek, Gemini, or similar, with a strong intuition for model selection tradeoffs.
  • Experience building LLM-powered agents or automations that are core to production systems.
  • Strong system-design skills for tools/functions, structured outputs, and multi-step agent flow architecture.
  • Experience designing and maintaining evaluation pipelines for prompts, workflows, and LLM outputs.
  • Hands-on experience with LLM observability, monitoring, and logging tools to track latency, accuracy, and quality in production.
  • Practical experience with the HuggingFace ecosystem in real projects.
  • Bonus: experience running self-hosted LLMs in production or serious prototypes.
  • Bonus: background in supply chain, logistics, or operations domain systems.
Compensation & Benefits

Visa sponsorship is available.
Location

On-site in New York, NY, United States.

Similar Jobs

More Jobs at Clera

More Consumer Technology Jobs

Find similar LLM Engineer jobs: