Harris Computer is hiring a Staff AI Engineer to join their, expanding team! We’re seeking a seasoned expert to pioneer the quality and reliability of our cutting-edge AI and machine learning solutions for healthcare. As a Staff AI Engineer focused on Quality Assurance, you will be the technical authority responsible for ensuring our AI systems are robust, fair, accurate, and safe.
This is a unique opportunity to define the future of AI quality in a domain where it matters most, mentoring a talented team and establishing the gold standard for testing intelligent systems that impact patient care. This is a full-time, remote position open to candidates in the U.S. or Canada.
What your impact will be:
- Build the core AI system end-to-end, from the service layer and data pipelines to the evaluation infrastructure, and own its architecture.
- Design and build the evaluation infrastructure, including dataset management, automated test harnesses, and replay of AI systems against historical workflows.
- Build and maintain evaluation datasets with clinician-reviewed correct answers, ensuring they evolve as our systems change.
- Create error metrics weighted by severity, so mistakes with clinical or financial impact are prioritized accordingly.
- Build production monitoring for accuracy, overrides, and safety issues, with actionable alerts for the engineering team.
- Write adversarial tests for AI systems, including hidden or malicious instructions in documents and invented content.
- Work with product and clinical leads to define accuracy thresholds and the criteria for expanding or pulling back an AI capability.
- The candidate must possess a curiosity and willingness to actively adopt and leverage emerging AI tools to improve workflows, solve problems, and drive efficiency along with being comfortable using a range of AI-enabled tools (such as copilots, chat-based AI, and automation solutions) as part of everyday work.
What we’re looking for:
- 7 or more years in software or machine learning engineering, with at least 3 years focused on evaluation, testing, or quality for AI systems.
- A track record of designing and building complete systems from scratch, taking them from prototype to production.
- A track record of building and running evaluation systems that teams use in production, not just for one-off analyses.
- At least 2 years of experience evaluating large language model (LLM) systems in a production environment.
- Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
- 1 year + of experience designing evaluation sets for document extraction, classification, or structured output.
- Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude).
- Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, engineers, and executives.
What would make you stand out:
- Experience working within the healthcare or another highly regulated industry.
- Familiarity with AI-specific testing techniques like metamorphic testing, adversarial testing, or data validation with tools like Great Expectations.
- A history of mentoring engineers and influencing technical direction across multiple teams.
What we offer:
- Plenty of opportunities to grow your career
- Comprehensive medical, dental, and vision benefits
- 3 weeks of vacation plus 5 personal days to recharge
- Employee stock ownership, RRSP program, 401k matching
- A chance to give back through community involvement
- Flexible work arrangements to suit your lifestyle
Salary range: 120,000-160,000 depending on experience and qualifications.
#LI-remote