Summary

This role exists to answer one question independently: do the synthetic-audience and automated systems actually perform as claimed? It is a verification-and-validation function drawn from data science, closer to model validation, ML evaluation, and the independent V&V discipline used in regulated and safety-critical software than to market research. The V&V Data Scientist will design and run the evaluation that benchmarks system output against ground-truth data, owns the accuracy metrics and error bounds, and certifies fitness for use, working independently of the engineer who builds the models. The separation of duties is the point: the builder does not grade their own work.

Responsibilities

• Design and run the independent validation methodology: benchmark synthetic-audience and automated output against real panel holdout and other ground-truth data.
• Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify where each system is and is not reliable.
• Certify fitness for use, translate results into clear "validated for this use / not validated for that use" determinations.
• Operate independently of the AI/Synthetic Engineer: verify the models and pipelines AI/Synthetic Engineer builds without having built them.
• Monitor for model and data drift over time; re-validate as models, prompts, and source data change.
• Maintain the audit trail and evidence base that supports the function's credibility with stakeholders and any external auditor.

Education & Experience Requirements

Education

• Bachelor's degree in computer science, mathematics, or engineering or relevant equivalent experience in lieu of degree. Master's degree preferred.

Experience

• Seven (7) or more years in data science, model validation, ML evaluation, or an independent V&V / QA function.
• Demonstrated experience independently validating models or systems against ground-truth data, evaluation design, benchmarking, and error analysis.

Certifications - NA

Knowledge, Skills & Abilities

• Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification.
• Working understanding of LLMs and synthetic-respondent systems and their failure modes - enough to test them rigorously.
• Independence and rigor: the temperament to challenge results and withhold sign-off, including from colleagues' work.
• Clear communication: able to tell a non-technical executive, with evidence, why a result can or cannot be trusted.
• Ability to effectively leverage artificial intelligence (AI) tools and technologies to streamline workflows, enhance productivity, and improve overall work quality.
• Familiarity with survey/behavioral data as a validation target preferred.

Physical Requirements

This position operates in a typical office environment (which includes a home office setting) and requires the ability to perform essential job functions with or without reasonable accommodation. Physical requirements may include:

• Prolonged periods of sitting at a desk and working on a computer.
• Frequent use of hands and fingers for typing, handling documents, and using office equipment.
• Occasional standing, walking, bending, and reaching.
• Ability to lift and carry up to 30 pounds as needed.
• Clear verbal and written communication skills for effective interaction with colleagues and stakeholders.

Work Environment

Hybrid Schedule (3 Days In-Office/2 Days Remote)

This position follows a hybrid work schedule, with Tuesday through Thursday in office and Monday and Friday remote. Employees must be available during standard business hours, with core hours beginning between 8:00-9:00 a.m. and concluding between 5:00-6:00 p.m. local time.

Travel: Occasional 0 - 10%

#LI

The hiring range for this position is $110,000 to $135,000 per year. This range is an estimate, and the actual salary may vary based on the candidate's experience, skills, and qualifications. SHRM offers a competitive and comprehensive total rewards package. The benefits for this position include professional growth and development, health, dental, vision, well-being, health savings, flexible spending, retirement, open leave, and annual discretionary bonus and incentives.