We are seeking a highly detail-oriented AI QA Engineer with a mix of manual and automation testing skills to validate the performance of cutting-edge, AI-driven video analysis systems. In this role, you will focus on verifying how advanced AI, Large Language Models (LLMs), and computer vision models detect key moments and generate metadata for sports content. This position sits at the intersection of traditional sports broadcasting and advanced artificial intelligence. The ideal candidate has a deep passion for sports (understanding rules, metrics, and context), strong technical QA scripting skills, and experience working with video assets, timecodes, and metadata. Req: [redacted] Responsibilities Live Game Auditing: Monitor and audit live sports broadcasts (NBA, MLB, NFL, NHL) to ensure the AI/Inference Service correctly tracks, frames, and labels major moments (e.g., touchdowns, home runs, buzzer-beaters) in real-time Precision QA with Media Assets: Work directly with video frames, timecodes, transcriptions, captions, and JSON outputs to ensure pinpoint alignment between AI detections and actual broadcast moments Model Error Triage & Support: Act as the human-in-the-loop expert to catch AI hallucinations, misinterpretations of complex sports rules, or edge-case errors. Collaborate directly with AWS and core engineering teams to detail bugs and validate model resolutions Metadata & Brand Safety Validation: Review AI-generated labels/tags to ensure they align with sports context and meet advertising industry compliance standards (e.g., IAB guidelines), ensuring content is brand-safe for monetization QA Automation: Design and execute automation scripts to compare AI inference outputs against customer-provided ground truth data at scale Iterative Testing & Documentation: Keep meticulous records of inference service performance, track accuracy metrics, log defects, and help streamline iterative testing processes for ongoing model updates Requirements Hybrid QA Experience: Proven experience in both manual and automation testing, ideally in video-focused or AI-driven environments Automation Scripting: Hands-on automation scripting skills (e.g., Python, Selenium, or similar testing frameworks) to validate JSON outputs and data payloads against ground truth datasets SDLC & QA Methodologies: Strong understanding of testing lifecycles, including detailed bug logging, defect triage, regression testing, and quality reporting LLM & CV Familiarity: Solid understanding of testing LLMs (workflows, prompt/response validation) and conceptual familiarity with computer vision and transcription analysis Media & Video Literacy: Experience working with video frame analysis, timecodes, subtitles/captions, and metadata structures Sports Domain Knowledge: A deep understanding of major sports leagues (NFL, NBA, MLB, NHL), including gameplay rules, terminology, and key metrics Communication: Strong verbal and written communication skills to act as a bridge between technical development teams and business stakeholders Analytical Mindset: An iterative, meticulous approach to data validation, checking model outputs for accuracy, bias, and context Nice to have Prior experience with specialized video testing tools, media processing pipelines, or video player frameworks Professional background in sports analytics, sports media, or digital broadcasting technology