5-7 years of experience in software engineering with a focus on infrastructure and data systems
Proficiency in Python and experience with tools like Docker and Terraform
Hands-on experience with LLM APIs and agentic systems
Familiarity with bespoke evaluation and monitoring tools, preferably using SvelteKit or React
Strong understanding of machine learning environments and reinforcement learning (RL)
Responsibilities
Own the development of the infrastructure for long-horizon scientific analysis
Build and maintain data pipelines and storage systems for agent operations
Create reproducible execution environments for reliable evaluations
Design and implement evaluation systems and regression gates
Collaborate with domain experts to create measurable evaluation criteria
Develop tools to enhance agent performance and stay updated on industry advancements
Ensure observability and replayability of agent runs through comprehensive tracing
Benefits
Flexible work environment with opportunities for remote work
Access to cutting-edge technology and tools
Collaborative culture with domain experts
Opportunities for professional growth and continuous learning
Engagement in innovative projects that shape the future of AI
Full Job Description
What you will do:
Own the harness: the scaffolding, tooling, and infrastructure that turn frontier models into agents that do long-horizon scientific analysis
Build the data backbone that gets everything to the right place: pipelines, storage, and systems for runtime context, agent trajectories, eval results, and training data
Build sandboxed execution environments with instant spin-up/tear-down, reproducible and deterministic enough to trust for evals and RL
Design and run the eval systems: offline suites, test cases on production traces, LLM-as-judge pipelines, regression gates
Work closely with domain experts and encode their taste into rubrics, golden sets, and review workflows, turning "I know it when I see it" into something measurable
Build the tools the agent needs to do better work, and stay tuned in to the state of the art for new methods, protocols, and patterns worth adopting
Make every agent run observable and replayable: trace every model call, tool call, and state transition, and build the debugging tooling to make sense of it
Engineer the context: memory, compaction, retrieval, and recovery so long-horizon agent runs stay coherent across hours and crashes
Own the loop itself: retries, budget caps, stop conditions, output verification, permissions, and guardrails
Support ML research with environments, reward instrumentation, and rollout infra for RL on agentic tasks
