OnTrac

Senior/Staff QA Engineer - Agentic AI Platform

OnTrac$100K — $130K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years of professional QA experience, including 2+ years at senior or staff level.
  • Strong understanding of the Application Development Lifecycle (ADLC) from requirements through release.
  • Solid UI/UX understanding to evaluate usability and flag interaction issues.
  • Hands-on experience testing AI Agent use cases and failure-mode behaviors.
  • Proven experience with automated testing frameworks and maintainable test suites.
  • Ability to manage a Playwright cluster for E2E testing against a live agent platform.
  • Experience in defining test cases from ambiguous product requirements.

Responsibilities

  • Collaborate with Delivery Lead/Solutions Architect and customer stakeholders to ensure product quality.
  • Write precise manual test cases and automate test coverage for AI products.
  • Drive the QA lifecycle from planning and execution to defect triage and regression.
  • Evaluate and improve the usability of the product by identifying design issues.
  • Manage automated tests in a containerized environment with Playwright.
  • Integrate automated testing into CI/CD pipelines effectively.
  • Ensure QA processes align with the fast-paced AI development environment.

Benefits

  • Opportunity to work on cutting-edge AI products.
  • Collaborative environment with influential stakeholders.
  • Autonomy in defining and managing the QA process.
  • Dynamic and fast-moving work culture.
  • Exposure to advanced testing frameworks and techniques.
Full Job Description
Overview

We are seeking a highly skilled and experienced Senior/Staff QA Engineer to support a critical customer engagement. This role is ideal for a detail-oriented tester who thrives owning quality end-to-end - someone equally comfortable writing precise manual test cases and building automated coverage for a fast-moving agentic AI product.

The successful candidate will work closely with the Delivery Lead/Solutions Architect and customer engineering stakeholders to ensure product quality from initial test planning through final release.
Required Credentials
  • 5+ years of professional QA experience, with at least 2+ years at a senior or staff level.
  • Demonstrated experience testing production software through a full Application Development Lifecycle (ADLC).
Required Qualifications
  • ADLC Expertise: Strong understanding of the Application Development Lifecycle - from requirements through release and support.
  • UI/UX Acumen: Solid UI/UX understanding, with the ability to evaluate usability and flag design/interaction issues, not just functional bugs.
  • AI Agent Testing: Hands-on experience testing AI Agent use cases - including conversational flows, agent decision-making, tool/function calling, and edge-case or failure-mode behavior.
  • Automation Frameworks: Proven experience with automated testing frameworks and building maintainable, scalable test suites.
  • Infrastructure Management: Ability to stand up and manage a Playwright cluster to run automated end-to-end (E2E) testing against a live agent platform.
  • Autonomy: Ability to independently define clear, well-structured test cases from ambiguous or evolving product requirements.
  • Lifecycle Ownership: Experience driving full E2E QA lifecycle ownership: test planning, execution, defect triage, regression, and sign-off.
Required Technical Stack Experience (Suggested Completion)
  • Automation Frameworks: Deep expertise with Playwright (including cluster management and parallel execution).
  • Languages: Proficiency in JavaScript / TypeScript or Python for writing automation scripts.
  • AI/LLM Tools: Familiarity with testing LLM outputs, prompt evaluation frameworks, or conversational AI testing tools.
  • CI/CD & DevOps: Experience integrating automated test suites into pipelines (e.g., GitHub Actions, GitLab CI, Jenkins) and working with Docker containerization.
  • Defect Tracking & Tools: Proficiency with Git, Jira, and modern test management tools.


About OnTrac

OnTrac is a courier and delivery service that specializes in overnight and time-critical deliveries. The company was founded in 1991 and has since grown to become one of the largest regional package delivery companies in the United States. OnTrac offers a range of services, including same-day delivery, next-day delivery, and two-day delivery, and serves customers in Arizona, California, Nevada, Oregon, Utah, and Washington. The company is committed to providing its customers with fast, reliable, and cost-effective delivery solutions, and has built a reputation for excellence in the industry.
Learn more about OnTrac
Size
3,000 employees
Industry

Similar Jobs

More Jobs at OnTrac

More Information Technology Jobs

Find similar Senior/Staff QA Engineer - Agentic AI Platform jobs: