ResponsibilitiesPeraton Labs is seeking a QA/Test Engineer to develop and maintain testing frameworks that ensure the accuracy, reliability, and consistency of a generative AI-powered analytical platform's outputs and functionality. This role focuses on the unique challenges of testing generative AI-driven systems, including non-deterministic outputs, analytical accuracy validation, and regression testing for AI behavior changes.
Key Responsibilities
- Design and implement automated testing frameworks for generative AI agent outputs, including accuracy benchmarks and regression detection
- Build validation frameworks for analytical calculations
- Develop end-to-end test suites for multi-worker analytical workflows
- Implement data quality testing for ingestion pipelines and data fusion outputs
- Create test datasets and golden reference outputs for analytical features
- Build performance testing frameworks for platform scalability assessment
- Develop security testing procedures for sandbox enforcement and access controls
- Establish testing standards and quality gates for the CI/CD pipeline
Qualifications
Required Qualifications
- Minimum of 5+ years of experience with a Bachelor's degree, 3+ years with a Master's degree, or a PhD in Computer Science or related field
- Hands-on QA/test engineering experience
- Strong Python proficiency; experience with testing frameworks (pytest, unittest)
- Experience with automated testing for data-intensive applications
- Understanding of statistical methods for evaluating non-deterministic system outputs
- Experience with API testing, integration testing, and end-to-end testing
- Familiarity with CI/CD integration for automated test execution
- Strong analytical skills and attention to detail
- US Citizenship with the ability to obtain/maintain a Secret clearance
Preferred Qualifications
- Experience testing generative AI/ML systems or NLP pipelines
- Experience with performance testing tools (Locust, k6, or similar)
- Knowledge of security testing methodologies
Target Salary Range$86,000 - $138,000. This represents the typical salary range for this position. Salary is determined by various factors, including but not limited to, the scope and responsibilities of the position, the individual’s experience, education, knowledge, skills, and competencies, as well as geographic location and business and contract considerations. Depending on the position, employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay.
