Job Type
Full-time
The role You will build the machine learning models and LLM systems that go inside Certus, starting with utilization management. That means working shoulder to shoulder with Physician Advisors to figure out which decisions are worth supporting with a model, building the thing, proving it works against a clinically adjudicated gold set, and shipping it into a product hospital staff use every day. You will own capabilities end to end, from the FHIR feed to the live endpoint. You will not be handed a spec. You will help write it.
We have a clear point of view: AI augments clinical judgment, it does not replace it. Physician Advisors and UM teams stay in the loop. If a model is not explainable or does not move a real number for a hospital, we do not ship it.
Requirements
Build and ship models - Train, evaluate, and deploy models for case prioritization, level-of-care and status-determination support, and denial-risk scoring. Own each one from problem framing through production monitoring.
Build LLM and agent capabilities - Design retrieval systems over clinical notes and payer policy documents. Build summarization for peer-to-peer prep and drafting support for appeals. Version your prompts, log your outputs, and build the guardrails.
Prove it works - Build labeled gold sets with Physician Advisors. Publish accuracy, precision, recall, and the error profile for every model. Run the evaluation harness in CI so nothing ships without passing the gate.
Own the data path - Build feature pipelines from the Certus Connect FHIR feed and Salesforce case data. Handle messy clinical data, missing fields, and inconsistent coding. Keep every transform traceable.
Keep it explainable and compliant - Ship an explanation with every score. Design for HIPAA and HITRUST from the start, not as a review at the end. Flag PHI risk before anyone asks.
Watch it in production - Monitor drift, track accuracy on a rolling sample, set the retraining triggers, and write the runbook. Catch problems before a client does.
Work across the company - Partner with Physician Advisors on clinical ground truth, IT on infrastructure and integration, and product engineering on how model output lands in Navigator and Radar.
You're a fit if you have - 3 to 6 years building and shipping machine learning models that real users depended on in production.
- Strong Python. Fluent with the standard modeling stack: pandas, scikit-learn, and at least one of XGBoost, LightGBM, or PyTorch.
- Hands-on experience building with LLMs beyond API calls: retrieval, structured output, prompt versioning, and evaluation of generative output.
- SQL good enough to build your own features from a messy relational source without waiting on anyone.
- Experience deploying models as services and monitoring them after launch, including at least one model you had to debug or retrain in production.
- A working approach to evaluation. You build the test set before the model and you can explain your metric choices.
- The habit of writing things down. Documentation, runbooks, and written analysis are part of how you work.
Bonus points - Healthcare experience, especially revenue cycle, utilization management, CDI, coding, or payer operations.
- Worked with clinical data standards: FHIR, HL7, ICD-10, CPT, MS-DRG, or claims data.
- Built under a regulator: HIPAA, HITRUST, SOC 2, or FDA SaMD.
- Salesforce data model exposure, or experience integrating ML output into a Salesforce-based application.
- MLOps depth: CI for models, feature stores, experiment tracking, or containerized model serving.
- Built explainability into a model that a non-technical domain expert had to trust and act on.
- Experience with agent frameworks and multi-step LLM workflows running in production, past the prototype stage.
Logistics - Remote, anywhere in the US. Our HQ is in Tampa, and we will fly you in for onsite time with the team a few times a year.
- Reports to the VP, Applied AI and Decisioning Systems.
- Individual contributor. No direct reports.
- Full-time. Health, dental, vision, and 401(k).
Salary Description
$135,000-$180,000
