Job DescriptionSenior AI Evaluation Engineer to join our team full-time in Canada.
About the role:As a Senior AI Evaluation Engineer, become a part of a cross-functional development team, engineering experiences of tomorrow.
Responsibilities:- Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests
- Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites
- Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly
- Stand up production monitoring for drift and regression with the pipeline engineer
Requirements:- 4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering
- Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples
- Independent spine: the team shipping a thing does not set its own pass bar
Desirable:- Healthcare or safety-critical evaluation experience; red-teaming background
What's in it for you?- Strong community: Work alongside top professionals in a friendly, open-door environment
- Growth focus: Take on large-scale projects with a global impact and expand your expertise
- Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications
- Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies
- Care: Healthcare, Basic Life Insurance, Short and Long-term disability insurance according to the Company's Benefit Plans
Interested already? We would love to get to know you! Submit your application. We can't wait to see you at Ciklum.
#LI-VH1
