Job Summary
The Experimentation Analytics Lead will support the growth of the rapid experimentation program and e-commerce domain experimentation roadmaps. This role will serve as a practitioner and subject matter expert for website experimentation, partnering with Product, UX, Analytics, and Engineering teams to design high-quality experiments, evaluate results, and ensure product decisions are grounded in statistically sound evidence. The ideal candidate combines strong statistical expertise with practical experience running digital experiments at scale and the ability to translate experimental results into clear business decisions.

Key Responsibilities
• Partner with Product and UX teams to develop hypotheses, success metrics, and experimentation strategies for digital experiences.
• Design A/B and multivariate experiments, including audience allocation, sample sizing, test duration, primary KPIs, and guardrail metrics.
• Conduct statistical power analysis and determine minimum detectable effects for proposed experiments.
• Analyze experiment results and determine statistical and practical significance.
• Apply advanced experimentation techniques, including variance reduction, sequential testing, heterogeneous treatment effect analysis, and other appropriate statistical methodologies.
• Evaluate experiment validity, including sample ratio mismatch, instrumentation issues, novelty effects, traffic imbalance, and other sources of bias.
• Develop measurement approaches for tests with uneven audience allocation or operational constraints.
• Conduct segmentation and follow-up analyses to understand how experiment effects differ across devices, customer groups, product categories, and experiences.
• Establish experimentation standards, governance, and best practices across digital product teams.
• Partner with analytics implementation teams to validate tracking and ensure experiment data quality.
• Translate experimental findings into clear launch, iterate, or stop recommendations for Product and senior stakeholders.
• Guide agencies and vendors supporting experimentation and analytics initiatives.
• Coach and mentor junior team members as needed.

Required Qualifications
• Bachelor's degree or equivalent experience.
• 4-6 years of experience working in the respective domain area.
• Advanced proficiency with domain-specific tools.
• Strong statistical expertise and practical experience with digital experimentation.
• Strong business acumen and knowledge of the retail industry.
• Excellent verbal and written communication skills.
• Ability to deliver non-technical presentations to executive-level business partners.
• Excellent attention to detail and strong organizational skills, with the ability to manage multiple work requests, projects, and initiatives.
• Experience guiding agency and vendor work.

Preferred Qualifications
• 1-2 years of experience as a team or peer lead, including coaching and mentoring junior team members.