Job DescriptionTechnical Manager - LLMs, RAG and ML is a strategic role within the Product organization, reporting to the Head of Section - AI Solutions. This role is responsible for translating applied AI strategy into customer-ready capabilities, prototypes, reusable technical patterns, evaluation methods, and production-oriented solution designs that support the Transformational Trust Initiative and broader digital platform enablement across Energy Systems.
This role is responsible for translating applied AI strategy into customer-ready capabilities, prototypes, reusable technical patterns, evaluation methods, and production-oriented solution designs that support the Transformational Trust Initiative and broader digital platform enablement across Energy Systems.
This role sits within Digital & Transformation and operates as a core product partner to the AI Solutions function, working across advisory regions, Renewable Certification, Digital & Data Solutions, Global Service Area Leads, engineering, data science, UX, and domain experts.
This position collaborates directly with the Head of Section - AI Solutions to identify high-impact AI opportunities, define product requirements, validate prototypes and proofs-of-concept, and support the delivery of scalable AI-driven workflows across Energy Systems.
This role is based at Toronto, ON office. What You'll DoAI Product Strategy & Roadmap Contribution- Partner with the Head of Section - AI Solutions to translate the applied AI strategy into an executable technical roadmap, delivery plan, and prioritized set of AI product capabilities
- Contribute to roadmap definition, product vision, and technical strategy for LLM-enabled workflows across the Transformational Trust platform.
- Identify high-impact opportunities where AI, language models, retrieval systems, and automation can improve customer value, workflow efficiency, decision quality, and platform adoption.
- Help define technical standards, delivery patterns, and reusable components for AI-powered product development across advisory and digital platform workflows.
- Provide informed recommendations on model selection, retrieval architecture, evaluation approach, prompt strategy, orchestration patterns, and technical feasibility.
LLM, RAG and AI Solution Development - Design, prototype, and guide delivery of AI-enabled capabilities using Large Language Models, generative AI foundations, Retrieval-Augmented Generation, vector search, and structured output techniques.
- Develop proof-of-concepts, reference implementations, and solution designs that engineering partners can implement, scale, and maintain.
- Design retrieval and indexing approaches for complex document, language, and workflow data, including semantic search, and context construction.
- Apply prompt engineering, tool/function calling, structured reasoning, and multi-step workflow patterns to support reliable AI-enabled product experiences.
- Support fine-tuning, LoRA, model adaptation, and evaluation approaches where needed to improve domain performance, task reliability, and user outcomes.
AI Evaluation, Quality and Responsible Use- Establish practical evaluation methods for LLM-enabled workflows, including accuracy, relevance, reliability, retrieval quality, user experience, and task completion metrics.
- Develop benchmark approaches, test datasets, evaluation rubrics, and monitoring methods to support reliable AI performance over time.
- Partner with the Head of Section, data science, engineering, and governance stakeholders to support responsible AI practices, including transparency, risk identification, data considerations, and appropriate human oversight.
- Define quality standards for prototypes, AI components, prompt patterns, retrieval workflows, and production handoffs.
- Track and communicate AI product performance against KPIs tied to customer value, workflow efficiency, model quality, and responsible AI outcomes.
Cross Functional Technical Leadership- Lead cross-functional AI delivery efforts across product, engineering, data science, UX, domain experts, and customer-facing teams.
- Serve as a technical translator between business needs, user workflows, AI methods, and engineering implementation.
- Provide technical guidance to teams working on LLM applications, retrieval systems, chatbot or agentic workflows, data pipelines, and AI-enabled decision support.
- Support rapid prototyping and iteration while maintaining a disciplined approach to quality, scalability, and responsible deployment.
- Coach emerging AI and product talent as the AI Solutions function grows, helping build a culture of experimentation, rigor, inclusion, and continuous improvement.
Responsibilities• Generous paid time off (vacation, sick days, company holidays, personal days)
• Medical, Dental and Vision benefit plans available to all employees
• Special programs - Employee Assistance Program and accident and critical illness options for you and your family
• Workplace strategies for mental help resources available to all employees
• Registered Retirement Savings Plan (RRSP) with company match
• Company provided life insurance, short-term, and long-term disability benefits
• Education Assistance Program
• Flexible work schedule with hybrid opportunities
• Consumer discounts and rewards through our BenefitsHub
• Advancement opportunities
**Benefits vary based on position, tenure, location, and employee election QualificationsWhat is Required- Bachelor's degree required
- 5+ years of relevant experience
- Significant hands-on experience designing, developing, evaluating, or delivering LLM-enabled capabilities in digital products, applied AI solutions, or customer-facing platforms
- Demonstrated experience with Retrieval-Augmented Generation, vector search, semantic retrieval, indexing strategies, or related methods for working with large text or document datasets
- Experience developing or applying LLM evaluation methods, model quality metrics, benchmark approaches, prompt evaluation, chatbot evaluation, or human-centered AI assessment methods
- Strong understanding of prompt engineering, structured outputs, tool/function calling, model orchestration, and practical techniques for improving reliability in LLM-enabled workflows
- Experience contributing to product strategy, technical roadmaps, product discovery, solution design, or prioritization of AI-enabled capabilities
- Demonstrated track record delivering AI, machine learning, NLP, chatbot, language modeling, or data science products from concept through implementation or production handoff
- Ability to work directly with engineering, data science, UX, product, business stakeholders, and customer-facing teams to translate complex needs into scalable technical solutions
- Strong communication skills, including the ability to explain AI concepts, tradeoffs, risks, and product implications to both technical and non-technical audiences
- Experience leading cross-functional initiatives, technical workstreams, or small teams in a product, AI, data science, or technology environment
- Strong written and verbal English communication skills
- We conduct pre-employment background screening
What is Preferred- Master's degree or equivalent advanced experience in data science, machine learning, artificial intelligence, computer science, or a related field
- Experience with FAISS or similar vector indexing, embedding, semantic search, or retrieval technologies
- Experience with fine-tuning, LoRA, model adaptation, synthetic data generation, or custom objective functions for language models
- Experience developing AI-enabled workflows for advisory, evaluation, due-diligence, risk, compliance, technical review, customer support, or knowledge-intensive professional services
- Familiarity with responsible AI frameworks, data governance, privacy considerations, or model risk management
- Experience delivering customer-facing products or internal platforms using AWS or comparable cloud environments
- Background in complex, technical, regulated, or high-trust sectors
*Immigration-related employment benefits, for example visa sponsorship, are not available for this position*#LI-DNI
#LI-DNP
