Senior QA Engineer

We are seeking a Senior QA Engineer to validate the performance of AI-driven video analysis systems, with a focus on detecting key moments in sports content. The role combines manual and automation testing, working with pre-labeled video assets provided by the customer.

Responsibilities

• Audit live sports events across leagues such as NBA, MLB, NFL, and NHL to confirm that the AI/Inference Service accurately tracks and labels major sports moments, including touchdowns, home runs, and buzzer-beaters, exactly as they occur
• Apply strong knowledge of sports contexts, terminology, and metrics while working with timecodes, video frames, transcriptions, and captions
• Act as the human expert who identifies when the AI produces errors, hallucinations, or misinterprets a sports rule, partnering directly with AWS engineers to document defects and validate their resolutions
• Review and validate metadata tags generated by the AI to ensure they align with the sport being analyzed and meet advertising industry standards (IAB rules), keeping content brand-safe for ad placements
• Maintain detailed QC documentation, keeping clear records of AI and Inference Service performance, tracking accuracy metrics, and helping establish a smooth testing process that connects traditional sports broadcasting with emerging AI technology

Requirements

• At least 3 years of professional relevant experience in QA engineering
• Experience in both manual and automation QA, preferably within video-focused or AI-driven environments
• Knowledge of testing Large Language Models (LLMs) along with an understanding of the associated workflows
• Automation scripting skills using languages and tools such as Python, Selenium, or similar for comparing JSON outputs to ground truth at scale
• Solid understanding of software development and QA cycles, including defect logging, triage, and reporting
• Ability to interpret labeled data and validate model outputs against expected results
• Familiarity with concepts such as computer vision and transcript analysis, without the need to understand internal model workings
• Strong communication skills for stakeholder interaction and reporting
• Detail-oriented and iterative approach to testing activities
• Fluent English communication skills at a B2 level or higher, both written and verbal

Nice to have

• Hands-on experience with video testing tools or frameworks dedicated to media QA
• Prior work in sports analytics or media technology environments