Job Description Summary:As a
Staff AI Engineer, you define and drive the architecture of AI and agentic systems across multiple teams and product domains. This is a senior individual-contributor leadership role: you influence high-impact architectural decisions, evolve the practices and standards for building agentic AI, and turn experimental AI capabilities into reliable production systems. You set direction for multi-agent orchestration, production RAG (hybrid search, re-ranking, and query routing), tool and MCP integration, and the evaluation and observability stack that keeps them dependable. You mentor senior engineers and represent AI engineering in cross-functional and strategic initiatives. A background in classical ML is an asset; the primary requirement is a proven track record of shipping production agentic AI.
KEY RESPONSIBILITIES- Define and drive technical direction for AI and agentic systems, and contribute to the AI platform roadmap across teams
- Influence architecture decisions for compute, cloud, and AI infrastructure across teams
- Lead the design of large-scale AI/LLM systems: inference platforms, APIs, and distributed architectures
- Architect production multi-agent systems end-to-end: orchestration, state management, tool integration, and failure handling
- Define and drive best practices and standards for AI/LLM systems across teams (agent design, evaluation, observability, reliability)
- Lead complex production debugging and incident response across teams, and harden the resulting fixes into platform guardrails
- Mentor senior engineers and emerging technical leaders, raising the engineering bar
- Lead technical design reviews and architecture decision records (ADRs) for critical AI infrastructure
- Contribute to capacity planning and cost optimization strategies for AI/LLM infrastructure
GENAI / AGENTIC AI CAPABILITIES- Define and drive vector database and RAG architecture decisions across systems and teams: structured RAG, hybrid search (dense + sparse + keyword), re-ranking, and query routing
- Lead multi-agent platform architecture decisions: runtime selection, orchestration patterns, and enterprise integration strategy
- Set the technical direction for MCP (Model Context Protocol) adoption and agent runtime infrastructure
- Shape agent infrastructure adoption: evaluate and standardize frameworks, tooling, and deployment patterns for agentic AI
- Architect evaluation infrastructure for non-deterministic LLM systems: synthetic golden-set generation, hierarchical weighted scoring (component, composite, and system-level F1), bootstrap confidence intervals, and paired A/B comparison, treating a change as real only when it is both statistically significant and clears a minimum effect size
- Gate deployments on eval results: tiered regression thresholds (hard-gate vs monitor components) wired into CI so a measurable quality regression blocks the release, with observability via tracing across multi-step chains and tool calls and drift detection on LLM inputs and outputs
- Drive LLM cost optimization at scale: model routing, caching, batching, token budget management, and provider cost analysis
REQUIRED SKILLS & QUALIFICATIONS- Master's or Ph.D. degree in Computer Science, Software Engineering, or a related field.
- 10+ years of professional experience in ML/AI or software engineering, including 4+ years in senior or staff-level roles with production system ownership
- Demonstrated engineering leadership, including driving technical strategy and influencing cross-team decisions
- Expertise in platform and distributed-systems architecture at scale: model serving, APIs, data platforms, and AI/LLM infrastructure
- Hands-on experience architecting and operating production agentic AI or LLM systems (multi-agent workflows, production RAG, tool and MCP integration)
- Deep understanding of embedding models, retrieval algorithms, and vector database internals
- Strong production debugging, reliability, and incident-response skills
- Experience building rigorous evaluation for non-deterministic AI systems, including statistical methods (such as bootstrap confidence intervals and minimum effect-size thresholds) to separate genuine quality changes from run-to-run model variance
- Cost-awareness for cloud AI/LLM workloads: capacity planning and cost optimization
- Proven mentorship of mid-level and senior engineers
- Strong communication skills for executive and cross-functional audiences
PREFERRED QUALIFICATIONS (NICE TO HAVE)- Experience in Healthcare, FinTech, or other regulated industries
- Experience building AI/LLM systems or platform components from the ground up
- Defined best practices for AI-assisted development (Claude Code): code quality standards, review, and responsible usage across teams
- Track record of conference talks, published papers, or significant open-source contributions
- Experience with GPU-accelerated inference and model serving optimization
- Familiarity with workflow orchestration and streaming architectures for real-time AI
ModMed Benefits Highlight: At ModMed, we believe it's important to offer a competitive benefits package designed to meet the diverse needs of our growing workforce. Eligible Modernizers can enroll in a wide range of benefits:
United States- Comprehensive medical, dental, and vision benefits, including a company Health Savings Account contribution,
- 401(k): ModMed provides a matching contribution each payday of 50% of your contribution deferred on up to 6% of your compensation. After one year of employment with ModMed, 100% of any matching contribution you receive is yours to keep.
- Generous Paid Time Off and Paid Parental Leave programs,
- Company paid Life and Disability benefits, Flexible Spending Account, and Employee Assistance Programs,
- Company-sponsored Business Resource & Special Interest Groups that provide engaged and supportive communities within ModMed,
- Professional development opportunities, including tuition reimbursement programs and unlimited access to LinkedIn Learning,
- Global presence and in-person collaboration opportunities; dog-friendly HQ (US), Hybrid office-based roles and remote availability for some roles,
- Weekly catered breakfast and lunch, treadmill workstations, Zen, and wellness rooms within our BRIC headquarters.
