Responsibilities: The Artificial Intelligence & Model Risk Auditor will play a key role in supporting the internal audit team in independently assessing the effectiveness of governance and risk management frameworks.
A strong background in frameworks for auditing models is a key requirement for this position.
Specific Artificial Intelligence and Model Risk Audit Responsibilities include:
- Apply frameworks for auditing models, including criteria such as robustness, fairness, interpretability, alignment, compliance, and human-in-the-loop assessments.
- Conduct audits across the model lifecycle, from training through deployment and monitoring, to assess compliance with quality, performance, fairness, and risk management standards.
- Identify model vulnerabilities, including bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation strategies.
- Assess model performance metrics, including accuracy, precision, recall, F1 score, calibration, robustness, and fairness metrics; measure hallucination rates in LLMs; quantify bias and fairness; evaluate confidence scores; and perform stability analyses.
Specific Audit Responsibilities include:
- Conduct multiple complex and concurrent audit assignments, including related scoping, planning, evaluation, and reporting.
- Write audit reports containing value-added recommendations.
- Understand and assess the risks applicable to the areas of audit responsibility through continuous monitoring activities.
- Manage time and priorities while conducting audits.
- Review and prepare risk assessments for audit units.
- Develop and maintain relationships with heads and managers of business lines and functions as part of continuous risk monitoring activities.
- Validate the adequacy and appropriateness of remediation or resolution plans addressing regulatory and internal audit recommendations.
- Investigate serious incidents or cases of fraud, when required.
- Deliver clear presentations to senior management and IG management.
- Keep abreast of developments in auditing standards and key regulations governing compliance and operational risk management.
- Adopt and promote strong ethical values.
- Participate in various committees, as assigned, to report on and provide information about current and emerging business issues to senior management.
Minimum Required Qualifications: - Bachelor's or master's degree in Mathematics, Statistics, Computer Science, Financial Engineering, or another related quantitative field.
- At least three years of professional experience in applied data science, machine learning engineering, or AI research, specifically involving LLMs and traditional machine learning models.
- Programming experience in statistical or mathematical languages such as Python or R, as well as hands-on experience with version control platforms such as GitHub.
- Experience with machine learning and NLP frameworks such as TensorFlow, PyTorch, and scikit-learn, and with relevant libraries such as pandas, Polars, and NumPy.
- Strong communication and synthesis skills, critical-thinking capabilities, and a passion for learning.
- Proficiency in English is required.
Preferred Qualifications: - Professional certification, such as the CIA designation.
- Good knowledge of key U.S. and Canadian banking regulations affecting the areas in scope.
- Ability to analyze model design, training methods, and data pipelines.
FINRA Registrations Required: CFTC Swap Dealer Associated Person (if yes, NFA Swaps Proficiency Program is required): SEC Security-Based Swap Dealer Associated Person: While the description above describes our ideal candidate, we encourage applicants to apply even if they do not fully meet the complete list of qualifications noted.
The expected starting salary range for this position is between $150,000 to $200,000 annually. The actual salary may vary based upon several factors including, but not limited to, relevant skills/experience, time in role, base salary of internal peers, prior performance, business line, and geographic/office location.
In addition, our comprehensive benefits package aims to support our employees in various aspects of their lives. From healthcare and wellness programs to retirement plans and childcare services, we prioritize the well-being and growth of our employees. Our benefits include medical, dental and vision coverage, a 401(k) Savings Plan, backup childcare, life, accident and disability insurance, mental health support and paid time off. Additional details about our benefits offerings, inclusive of eligibility for a discretionary bonus, will be provided during the hiring process.
