Role description

We are seeking an AI Testing Engineer to validate and assure quality of AIML systems powering realtime contextual advertising on live video streams The role involves testing multimodal AI outputs video text ensuring accuracy of detected moments and automating validation workflows

This role is critical for native moments validation and systemlevel QA in a realtime streaming environmentKey Responsibilities

Validate AIgenerated outputs against labelled datasets and expected outcomes

Perform content validation for native moments detection across video streams

Design and implement automated testing frameworks Pythonbased

Build validation harness for

JSON parsing and comparison

Structured output validation using defined schemas

Conduct multimodal validation video audio transcript outputs

Execute system integration and endtoend testing

Track accuracy metrics defect logging and root cause analysis

Automate regression and validation pipelines across datasets

Collaborate with Applied Scientists to validate LLM and GenAI outputs

Work within AWS environment using services such as

S3 data storage

Lambda processing

CloudWatch monitoring

Required Skills Experience

Strong Python scripting for automation and testing

Experience in AIML model validation and QA workflows

Handson withJSON parsing schema comparisonData validation pipelines

Experience in testingMultimodal AI systems vision NLP audio LLM outputs and generative AI pipelines

Good understanding ofStructured output validation AI accuracy evaluation techniques Familiarity with AWS cloud environment S3 Lambda loggingmonitoring Experience with mediavideo content analysis workflows preferred Ability to handle mixed manual automated testing approaches