LLM Engineer

Clera

$135K — $160K *
Enterprise Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 3+ years of experience with deploying LLM systems in production settings.
  • Familiarity with LLM APIs like OpenAI, Anthropic, or similar.
  • Proven track record of building LLM-driven solutions that integrate into production environments.
  • Expertise in system design for tool/function setup and multi-step workflows.
  • Ability to create and uphold evaluation processes for agent performance and outputs.
  • Experience with observability tools for monitoring system performance metrics.
  • Hands-on work with the HuggingFace ecosystem for production projects.

Responsibilities

  • Develop LLM-powered agents for optimizing production and distribution processes.
  • Craft prompts and design outputs to handle complex real-world data scenarios.
  • Assess and choose appropriate LLMs based on key performance metrics and project needs.
  • Manage the complete pipeline of LLM features from concept to deployment.
  • Establish frameworks for evaluating and iterating on agent performance.
  • Implement monitoring systems to track and improve production accuracy.

Benefits

  • Visa sponsorship available.
Full Job Description
About the Role

You'll be a founding LLM engineer at an early-stage, Y Combinator-backed AI supply chain startup, building an AI Supply Chain Manager from the ground up. This is a high-autonomy role where you'll own LLM systems end-to-end-from prototype to production-turning the physical flow of goods into something as programmable as code.
What You'll Do
  • Build and iterate on LLM-powered agents that decide what to produce, where to make it, and how to move it through factories, warehouses, and distribution channels.
  • Design robust prompts, tool definitions, structured outputs, and multi-step agent flows that handle edge cases in messy real-world data.
  • Select and evaluate LLMs based on latency, cost, accuracy, and use-case fit across supply chain workflows.
  • Ship LLM features from concept to production, owning the full pipeline from API integration to tested, evaluated systems.
  • Build and maintain evaluation frameworks to iterate quickly and safely on prompts and agent behaviors.
  • Set up observability, monitoring, and feedback loops to track production accuracy and continuously improve system performance.
What We're Looking For
  • 3+ years of hands-on experience shipping LLM-powered features or systems in production environments.
  • Practical experience with modern LLM APIs (OpenAI, Anthropic, DeepSeek, Gemini, or similar) and a track record of shipping real features with them.
  • Experience building LLM-powered agents or automations that are core to production systems-not just internal demos.
  • Strong system-design skills covering tool/function design, structured outputs, and multi-step agent flow architecture.
  • Proficiency designing and maintaining evaluation pipelines for prompts, workflows, and LLM outputs.
  • Experience with LLM observability, monitoring, and logging tools to track production accuracy, latency, and quality metrics.
  • Demonstrated ability to evaluate and select LLMs based on latency, cost, capability, and use-case tradeoffs.
  • Hands-on experience with the HuggingFace ecosystem in real production or prototype projects.
  • Bonus: Experience running self-hosted LLMs in production.
  • Bonus: Background in supply chain, logistics, or operations domain systems.
Compensation & Benefits

Visa sponsorship is available.
Location

On-site in New York, NY, United States.

Similar Jobs

More Jobs at Clera

More Enterprise Technology Jobs

Find similar LLM Engineer jobs: