Job DescriptionThe AI Systems Quality Specialist will design, build, and oversee the quality assurance framework for EVERSANA's AI Agency Platform.
This role ensures that every feature, integration, and AI-driven process operates reliably and compliantly. The QA Automation Lead will establish automated testing frameworks covering functional, regression, and compliance scenarios, manage integration test suites across connected systems (Google Cloud, Veeva, SFMC, Aprimo), and collaborate closely with DevOps, Product, and AI teams to uphold quality benchmarks.
The ideal candidate combines deep QA automation experience with an understanding of regulated environments and AI-enabled software systems.
ESSENTIAL DUTIES AND RESPONSIBILITIES:Our employees are tasked with delivering excellent business results through the efforts of their teams. These results are achieved by:
Automation Framework Design & Execution- Design, implement, and maintain a comprehensive automation framework for functional, regression, and integration testing.
- Build automated test scripts for APIs, UI workflows, data pipelines, and AI agent interactions using tools such as PyTest or Selenium.
- Integrate automated tests into CI/CD pipelines (Jenkins, GitHub Actions, or Cloud Build) to support continuous delivery.
- Ensure test scripts cover both core platform functionality and partner integrations (Veeva Vault, Vertex AI, Snowflake).
- Standardize QA practices for cross-team use and document testing methodologies.
Functional, Regression, and Compliance Testing- Develop and execute test cases for new features, enhancements, and bug fixes.
- Conduct regression testing prior to each release to ensure platform stability.
- Define and perform compliance and validation testing to ensure platform adherence to HIPAA, GDPR, SOC 2, and MLR workflow standards.
- Work with the AI Validation and Compliance teams to evaluate AI output quality (accuracy, hallucination, tone, compliance alignment).
- Ensure traceability between business requirements, test cases, and validation results through detailed test documentation.
Integration & System Reliability Testing- Oversee integration testing for all platform connectors and data flows (Veeva Vault, Aprimo, SharePoint, SFMC, Google Vertex AI).
- Develop synthetic test environments and mock APIs to validate complex, multi-system workflows.
- Partner with Data Engineering to validate ETL pipelines and analytics dashboards for data accuracy.
- Track and report system reliability metrics (latency, failure rate, API response times).
Defect Management & Continuous Improvement- Identify, log, and prioritize defects using Jira, tracking issue resolution through sprints.
- Collaborate with Product and Engineering teams to verify bug fixes and perform root cause analysis for recurring issues.
- Lead post-release validation and QA retrospectives to improve process efficiency.
- Proactively identify automation opportunities to reduce manual testing effort and cycle time.
Documentation & Audit Readiness- Maintain test documentation, validation reports, and evidence logs for internal and external audits.
- Support compliance reviews by ensuring full traceability from requirement 12 test case 12 result 12 release.
- Create and maintain QA dashboards and reporting templates for leadership visibility.
- Demonstrate a commitment to diversity, equity, and inclusion through continuous development, modeling inclusive behaviors, and proactively managing bias.
- All other duties as assigned
QualificationsMINIMUM KNOWLEDGE, SKILLS AND ABILITIES:The requirements listed below are representative of the experience, education, knowledge, skill and/or abilities required.
- 7+ years of experience in QA automation or software testing, with 3+ years in a lead or senior role.
- Proficiency in Python, JavaScript, or .Net for test automation scripting.
- Experience with test automation frameworks such as PyTest, Selenium.
- Strong understanding of API testing (Postman) and data validation (SQL, Snowflake, BigQuery).
- Proven experience testing integrated SaaS systems in cloud environments (preferably Google Cloud).
- Familiarity with CI/CD and continuous testing practices.
- Excellent documentation, reporting, and stakeholder communication skills.
Additional Information 
