Job DescriptionAt this time, we are only able to consider candidates who reside and work in British Columbia, Alberta, Manitoba, or Ontario, where the company is registered to operate.Trinnex, wholly owned subsidiary of CDM Smith, is seeking a highly skilled and collaborative Senior AI/ML Quality Assurance Engineer to lead the validation, testing, and monitoring of production AI and machine learning solutions. This role is responsible for ensuring our predictive models are accurate, reliable, explainable, and compliant, while partnering with Data Science, MLOps, Consulting, and Governance teams to support responsible AI delivery.
The ideal candidate combines expertise in AI/ML testing, model performance monitoring, and automation with the ability to translate complex technical findings into clear, actionable insights for both technical and non-technical stakeholders. This position will play a critical role in maintaining the quality and trustworthiness of Trinnex's AI-powered products across infrastructure, environmental compliance, and operational monitoring solutions.
Key Responsibilities- Design, develop, and maintain automated testing frameworks to validate machine learning models, data pipelines, and predictive analytics solutions.
- Validate tabular, geospatial, classification, and risk-scoring models to ensure accuracy, reliability, and alignment with business and regulatory requirements.
- Perform model explainability and quality assessments using industry-standard XAI techniques and document results for internal and client-facing stakeholders.
- Partner with MLOps teams to implement monitoring, alerting, and performance tracking for data drift, concept drift, and model degradation in production environments.
- Support root cause analysis, incident investigations, rollback validation, and recovery testing for AI/ML systems.
- Assist in evaluating emerging Generative AI and Retrieval-Augmented Generation (RAG) solutions, including model performance, retrieval accuracy, safety, and guardrail effectiveness.
- Collaborate with Consulting and Governance teams to develop validation reports, model documentation, and compliance artifacts that support transparent and responsible AI practices.
- Conduct quality reviews of AI and data science deliverables prior to client release, identifying risks, data quality concerns, and model limitations.
#LI-LP1
#LI-REMOTE
Skills & Abilities- Experience validating, testing, or monitoring production machine learning models, particularly classification, regression, or predictive analytics solutions.
- Strong proficiency in Python, SQL, and Git, with hands-on experience handling spatial data structures (e.g., GeoPandas, Shapely, or PostGIS).
- Experience with AI/ML testing, validation, and monitoring frameworks, including automated testing, model performance monitoring, and observability solutions. (e.g. pytest, MLflow, or Deepchecks; Evidently AI, Arize, or cloud observability stacks like GCP Production Monitoring/Cloud Monitoring, Grafana, or Datadog)
- Familiarity with Docker, Kubernetes, and cloud environments (GCP), including application monitoring and production support workflows.
- Ability to translate complex data science and model outputs into clear, actionable documentation for technical and business stakeholders.
- Understanding of model performance measurement, drift detection, and production monitoring concepts.
- Strong technical writing skills with experience creating and reviewing model documentation, validation reports, test plans, runbooks, APIs, or other technical artifacts.
- Ability to collaborate across multidisciplinary teams, including Data Science, MLOps, Software Engineering, Product Management, and Consulting organizations.
Qualifications- Bachelor's degree.
- 5 years of related experience.
Equivalent additional directly related experience will be considered in lieu of a college degree. Domestic and/or international travel may be required. The frequency of travel is contingent on specific duties, responsibilities, and the essential functions of the position, which may vary depending on workload and project demands.
Preferred Qualifications- Bachelor's degree in Computer Science, Software Engineering, Data Science, Geographic Information Systems (GIS), Engineering, or a related quantitative field.
- Experience validating machine learning models, datasets, or analytical solutions within infrastructure, utilities, environmental compliance, water, or wastewater domains.
- Familiarity with geospatial data, spatial analytics, or GIS-based applications.
- Experience working with time-series, sensor, IoT, or SCADA data in operational environments.
- Knowledge of AI governance, model risk management, or responsible AI frameworks.
- Familiarity with cloud-based AI/ML platforms and production monitoring environments.
- Experience evaluating or testing Generative AI, large language model (LLM), or Retrieval-Augmented Generation (RAG) solutions.
- Understanding model explainability, fairness, bias assessment, and AI transparency concepts.
Amount of Travel RequiredNo Travel is required
Additional Information This job posting is for an existing vacancy.
Pay Range MinCAD $144,477.00
Pay Range MaxCAD $252,824.00
Additional Pay Range InformationCDM Smith is committed to fair and equitable compensation practices. This pay range is a good faith estimate representative of all experience levels for the geographic location assigned to the position. In addition to geographic location, a candidate's salary is determined by several other factors including, but not limited to, the role, function and associated responsibilities, relevant work experience, skills, required certifications, and education/training.
Additional CompensationAll bonuses at CDM Smith are discretionary and may or may not apply to this position.
Visa Sponsorship AvailableNo-Please note that all applicants must be legally eligible to work in Canada, for the Company, at the time of hire. Furthermore, this is not a position for which the Company is offering immigration application sponsorship or support
