Applied AI Engineer - Agent

The General Intelligence Company of New York

• $150K — $180K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 4+ years backend engineering experience, preferably in Python with an impact-focused mindset
  • Hands-on experience with LLMs: adept in prompt engineering, function-calling, retrieval, and evaluation design
  • Proven ability to build evaluation harnesses for driving performance improvements
  • Strong foundation in distributed systems: concurrency, reliability, and data management
  • A pragmatic approach to experimentation, from hypothesis to rollout
  • Skilled in debugging and instrumentation, with a knack for addressing edge cases

Responsibilities

  • Design and implement end-to-end agent improvements, including safety protocols and memory management
  • Build and maintain robust evaluation pipelines, optimizing both offline and online metrics
  • Productionize LLM techniques, managing function orchestration and response handling
  • Enhance core backend systems for reliability and scalability, integrating with various platforms
  • Collaborate with product and infrastructure teams to define metrics and accelerate deployment of solutions
  • Write clean, effective code and document all design decisions and processes

Benefits

  • Mission-driven work focused on creating autonomous agents for businesses
  • Immediate impact on core agent functionalities users experience
  • Be part of a small, agile, and senior team with quick decision-making
  • Work with cutting-edge technology in AI orchestration and memory systems
Full Job Description
We're hiring an Applied AI Engineer to push the boundaries of our Cofounder agent. You'll own core backend systems and applied LLM work: advancing agent reliability and autonomy, building evaluation pipelines, and shipping techniques that measurably improve agent performance. This is a hands-on role with high ownership across research-to-production: prototyping, instrumenting, evaluating, and deploying improvements that show up directly in user outcomes.

What You'll Do
  • Design and implement agent improvements end-to-end: prompting strategies, tool selection, action planning, memory usage, safety/guardrails, and recovery paths
  • Build robust evaluation pipelines for the agent: offline evals (golden tasks, regression suites, behavior tests), online metrics (latency, success rate, fallout modes, cost efficiency), and experimentation frameworks (A/B, canaries, guardrail thresholds)
  • Productionize applied LLM techniques: function/tool-calling orchestration, self-reflection, retrieval/RAG, multi-agent handoffs, caching/embedding strategies, and hallucination reduction
  • Improve core backend systems: reliable job orchestration, retries/backoff, idempotency, and auditability; scalable memory and context routing; data pipelines across Gmail, Slack, Notion, Linear, Google Workspace, etc.; observability and tracing for agent actions/outcomes
  • Partner with product and infra to define success metrics and ship fast, safe iterations
  • Write clean, well-tested code; document design decisions and runbooks


What You'll Bring
  • 4+ years backend engineering experience, preferably Python (we care about impact over years)
  • Hands-on LLM experience: prompt engineering, function-calling, retrieval, embeddings, evaluation design; you've shipped LLM features to production
  • Track record building evaluation harnesses and using them to drive improvements (regression suites, task success metrics, cost/runtime tradeoffs)
  • Solid distributed systems fundamentals: concurrency, reliability, performance, data modeling, lifecycle management
  • Pragmatic experimentation: hypothesis 12 prototype 12 measured improvement 12 rollout
  • Excellent debugging and instrumentation skills; you enjoy finding and fixing edge cases in the wild


Nice To Have
  • Experience with agent frameworks, tool orchestration, and memory architectures
  • RAG systems in production (chunking, retrieval quality, freshness strategies)
  • Redis, Postgres/Supabase, queues (e.g., Celery/Arq/SQS), and event-driven designs
  • Observability stacks (Datadog, OpenTelemetry), and cost/latency optimization


Compensation
  • Competitive salary and meaningful equity
  • Comprehensive benefits and flexible work setup

Similar Jobs

More Jobs at The General Intelligence Company of New York

More Information Technology Jobs

Find similar Applied AI Engineer - Agent jobs: