Senior Applied AI Scientist, Clinical AI Agents

Phare Health

$140K — $300K *
US-AnywhereRemote in New York, NY
Healthcare
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 8+ years of experience in software engineering, ML engineering, research engineering, or applied AI.
  • Proficient in Python and well-versed in building production systems with APIs and structured data.
  • Experienced in transforming complex real-world workflows into actionable AI projects.
  • Familiar with developing evaluation systems, benchmarks, and experiment tracking for AI.
  • Ability to work in ambiguous, high-stakes environments to develop reliable systems.

Responsibilities

  • Design and improve AI systems for healthcare workflows including documentation and revenue cycle automation.
  • Develop agent behaviors related to context handling, memory, and human-in-the-loop review.
  • Define success criteria and standards for clinical agents based on expert workflows.
  • Create feedback loops to monitor performance and make iterative improvements.
  • Prototype new AI capabilities and transition them into reliable production systems.
  • Collaborate with research and ML engineering teams for model selection and internal infrastructure.

Benefits

  • Top-of-market compensation including salary and equity.
  • Flexible paid time off (PTO).
  • Comprehensive health benefits package.
  • 401(k) matching contributions.
  • Work alongside inspiring and mission-driven colleagues.
Full Job Description
The Role
  • We are looking for an Applied AI Engineer/Scientist to build, evaluate, and continuously improve clinical AI agents and supervised ML Models.
  • You will work at the intersection of software engineering, LLM systems, evaluation, model improvement, and deep healthcare workflow understanding. Your job is to turn frontier model capability into reliable production behavior: agents that read complex medical records, use the right clinical and coding context, call the right tools, produce auditable outputs, and improve from real-world failures.
  • You will be embedded in hard healthcare problems - clinical documentation integrity, medical coding, denial prevention, appeals, revenue cycle workflows, and payer logic - and will own the loop from problem framing to agent design, evaluation, deployment, trace analysis, and ongoing improvement.
  • The ideal candidate is a strong engineer who thinks like an applied scientist: rigorous about measurement, comfortable with ambiguity, excited by messy real-world data, and motivated by closing the gap between impressive demos and dependable production systems.


What You'll Do
  • Design, build, and iterate on agentic AI systems for complex healthcare workflows, including documentation, coding, denial management, appeals, and revenue cycle automation.
  • Develop long-horizon agent behavior across context construction, retrieval, tool use, memory, routing, verification, escalation, and human-in-the-loop review.
  • Define what "good" looks like for clinical agents end-to-end, translating expert workflows into specifications, rubrics, gold standards, test cases, and clinically meaningful success criteria.
  • Build rigorous evaluation and feedback loops using expert review, production logs, model outputs, and benchmarks to measure performance, regressions, edge cases, safety, reliability, provenance quality, and business impact.
  • Prototype new AI capabilities from 0 1, then harden them into reliable, explainable, auditable production systems with clear contracts, monitoring, evidence, rationale, and performance gates.
  • Partner with research and ML engineering teams on model selection, fine-tuning, reward modeling, distillation, synthetic data, post-training, and internal AI infrastructure, including instrumentation, experiment tracking, benchmarking, prompt/version management, and reproducible evaluation.


What Makes This Role Different
  • Most AI roles are either too research-heavy or too product-light. This role sits in the middle.
  • You will not only write prompts or run experiments. You will own whether an agent actually works in production. That means understanding the workflow, designing the system, building the evals, inspecting failures, improving the agent, and proving that the improvement matters.

The right person will be excited by questions like:
  • What context does this agent need to make the right decision?
  • How do we know the output is clinically and operationally correct?
  • Which failures are prompt problems, retrieval problems, model problems, tool problems, or product-spec problems?
  • How do we turn expert feedback into a better benchmark or training set?
  • When should we use prompting, RAG, rules, fine-tuning, reward modeling, or a different architecture?
  • How do we make agent outputs auditable enough for clinical and operational review?
  • How do we build a data flywheel that improves the system every week?


You May Be a Good Fit If You
  • Bring 8+ years of software engineering, ML engineering, research engineering, or applied AI experience.
  • Are highly proficient in Python and comfortable building production systems with APIs, structured data, async workflows, testing, logging, and observability.
  • Have experience turning messy real-world workflows into structured AI problems, including classification, ranking, extraction, decisioning, LLM applications, agents, RAG, tool calling, structured outputs, prompting, or evaluation.
  • Have built or operated evaluation systems, benchmarks, annotation workflows, experiment tracking, or regression tests for AI systems.
  • Thrive in ambiguous, high-stakes domains: working with experts, debugging real-world failures, and turning model potential into reliable, correct, safe systems that work for users.


Role Leveling

We are looking for candidates at various levels, ranging from Level 2 to Staff
  • Senior: Team Lead responsible for managing a portfolio of projects that contribute to major technical initiatives.
  • Staff: Impact at the organizational level. Responsible for leading multiple teams or multiple broad initiatives simultaneously, ensuring that high-level technical goals are met across the entire organization.

Benefits
  • Top-of-market compensation (salary + equity)
  • Flexible PTO
  • Comprehensive health benefits
  • 401(k) matching
  • Inspiring, brilliant, mission-driven teammates

Hiring Flow
  • Intro call - your background & our mission alignment
  • Technical deep-dives - pseudo-coding exercise and systems design (not Leetcode)
  • Final in-person interview at one of our hubs (SF, NYC, Austin, or Chicago; travel arranged)
  • References
  • Offer

Interview Logistics Notice
As part of our hiring process, selected candidates will participate in an in-person interview. Candidates located near one of our talent hubs-San Francisco, New York, Austin, or Chicago-will be scheduled to meet with team members in those locations. For candidates residing outside these areas, we will arrange travel to a hub for the interview. Travel accommodation will be provided as needed. We are committed to providing equal employment opportunities and ensuring a fair and inclusive experience for all applicants.
For this US-based position, the base pay range is $140,000.00-$300,000 per year. Individual pay is determined by role, level, location, job-related skills, experience, and relevant education or training.
This job is eligible to participate in our annual bonus plan.
The healthcare system is always evolving - and it's up to us to use our shared expertise to find new solutions that can keep up. On our growing team you'll find the opportunity to constantly learn, collaborate across groups, and explore new paths for your career.
Our associates are given the chance to contribute, think boldly, and create meaningful work that makes a difference in the communities we serve around the world. We go beyond expectations in everything we do. Not only does that drive customer success and improve patient care, but that same enthusiasm is applied to giving back to the community and taking care of our team - including offering a competitive benefits package.

Similar Jobs

More Jobs at Phare Health

More Healthcare Jobs

Find similar Senior Applied AI Scientist, Clinical AI Agents jobs: