The Opportunity:The QA Engineer is accountable for end to end quality assurance of AI enabled mortgage applications and workflows, ensuring that user experiences, APIs, integrations, data flows, business rules, controls, and operational processes work reliably before and after release.
This is a hands on role embedded in a First National product squad from the start of delivery. The QA Engineer develops and executes risk based manual and automated testing, supports UAT and release readiness, and partners with the AI Quality and Evaluation Lead on the evaluation of probabilistic AI outputs, while retaining clear ownership of functional, integration, regression, and end to end workflow quality.
How you will contribute:- Define and maintain a risk based test strategy for each product increment, covering functional, integration, end to end, regression, API, data, user interface, accessibility, performance, security control, and operational readiness testing.
- Work with the Product Manager, the Residential Underwriting Business SME, the UX Designer, the AI Engineering Lead, and engineers to turn requirements, policy rules, workflows, exceptions, and acceptance criteria into traceable test scenarios and representative test data.
- Design, execute, and document manual and automated tests across web applications, APIs, services, data pipelines, document processing, permissions, audit logs, notifications, human review queues, calculations, and controlled write back to core systems.
- Build and maintain automated regression suites and CI/CD quality checks using tools such as Playwright, Cypress, Selenium, Postman, Python, .NET test frameworks, SQL, or comparable technologies, and keep tests reliable as products evolve.
- Validate deterministic rules, calculations, data transformations, interface contracts, error handling, recovery, role based access, privacy controls, and release configurations across development, QA, UAT, and production like environments.
- Partner with the AI Quality and Evaluation Lead and AI Engineers to incorporate golden datasets, AI evaluation results, guardrail tests, confidence thresholds, and known failure modes into end to end product testing, without duplicating enterprise evaluation ownership.
- Collaborate with engineers to improve system testability, observability, logging, and diagnostic capabilities.
- Coordinate defect triage, severity and priority assessment, root cause investigation, retesting, release evidence, UAT support, parallel runs, and go live readiness, and communicate quality risks clearly to product and control stakeholders.
- Use production telemetry, incidents, user feedback, monitoring signals, and escaped defects to strengthen regression coverage, improve testability, and contribute reusable test assets and quality practices across product squads.
The experience you need: - Bachelor's degree in computer science, engineering, information systems or a related discipline, or equivalent practical experience.
- 5 plus years of progressive quality assurance, software testing, test automation, quality engineering, or related technology experience.
- Strong hands on experience testing web applications, APIs, integrations, data flows, and end to end business workflows, including functional, regression, negative, and production readiness testing.
- Practical experience with test automation and delivery tooling such as Playwright, Cypress, Selenium, Postman, Python, .NET test frameworks, SQL, Git based workflows, CI/CD, defect tracking, and test management tools.
- Experience validating complex business rules, calculations, permissions, auditability, data quality, exception handling, and operational controls in enterprise or regulated environments.
- Working knowledge of AI enabled system testing, including probabilistic outputs, RAG, document intelligence, agents, golden datasets, evaluation metrics, confidence, guardrails, drift, and human in the loop controls is strongly preferred.
- Experience supporting UAT, parallel runs, release approvals, defect triage, incident reproduction, root cause analysis, and continuous improvement with business and technology stakeholders.
- Experience in financial services, mortgage lending, lending operations, servicing, broker channels, third party partnerships, or regulated technology environments is preferred.
Relationships:External: Engages implementation partners and technology vendors on test scope, defect resolution, release evidence, and knowledge transfer to First National support teams.
Internal: Embedded in a First National product squad, working with the Product Manager, the Residential Underwriting Business SME, the UX Designer, AI Engineers, the AI Data and Integration Engineer, and full stack engineers. Partners with the AI Quality and Evaluation Lead, AI Engineering Lead, Security, Risk, Privacy, Data Governance, and release management teams to deliver controlled AI capabilities with measurable value.
Working Environment and Physical Demands Analysis:- Office environment
- Periods of high volume with tight timelines
- Long periods of stationary position/sitting
- Prolonged periods of repetitive movement (i.e. using a keyboard and mouse)
- Long periods of time in viewing a computer screen
- Multi-tasking may include speaking to customers on a telephone call while looking up information on a computer program.
