The Quality of Experience Experimentation team has a broad and cross-functional mandate to design, implement, and analyze A/B tests related to the quality of the video streaming experience, and to innovate on testing and analysis methodology. Our current work includes:
• Partnering with engineers to optimize the adaptive video streaming algorithms that determine key aspects of the viewing experience such as video quality, delay before playback starts, and rate of playback interruptions.
• Working with the video and audio encoding teams to optimize content delivery to cellular and other low-bandwidth networks, opening up the Netflix product to more people around the world.
• In collaboration with the Consumer Insights team, working to understand how our customers perceive video quality, and how video quality impacts viewing habits.
• Exploring ways to efficiently and effectively localize our content (video and audio assets, subtitles, text descriptions) to new markets and languages around the globe.
- Partner closely with scientists, engineers, and consumer insight researchers to design, run, and analyze A/B and multivariate hypothesis tests aimed at optimizing the streaming video experience.
- Analyze experimental data with statistical rigor. Ensure accurate interpretation by combining business acumen with detailed data knowledge and statistical expertise.
- Contribute to existing, and lead new, internal research and development efforts to improve experimentation methodologies. Current research areas include, but are not limited to, Response Surface Methodology (RSM), survival analysis, and nonparametric test analysis via computationally efficient bootstrapping.
- Develop causal inference methodology to analyze behavior in cases where A/B testing may not be possible.
- Build statistical models that better leverage test results to uncover insights or effect changes in the product.
- Translate analytic insights into concrete, actionable recommendations for business or product improvement, and communicate these findings.
- Drive efforts to enable independent interpretation of results through education, improved tools, and data visualization.
- PhD degree in Statistics, Mathematics, Operations Research, Psychology, Sociology, Econometrics, or related field, or a Master’s degree in statistics or related, and a domain-specific PhD that involved substantive applied statistics work.
- 3+ years relevant post-PhD experience with a proven track record of leveraging analytics and large amounts of data to arrive at actionable solutions to large-scale problems, in either research or business settings.
- Solid applied statistical knowledge and intuition, and solid grounding in statistical theory; experience with experimental design, analysis of hypothesis tests, and statistical modeling.
- Strong algorithmic thinking and independent research ability.
- Passion for learning and innovating new methodologies at the intersection of statistics, applied math, and computer science.
- Exceptional interpersonal and communication skills coupled with strong business acumen. Must be able to translate business objectives into actionable analyses, and analytic results into actionable business and product recommendations.
- Impactful presentation skills, including the use of meaningful data visualizations to convey information and results clearly and concisely.
- High-energy self-starter with a passion for your work, attention to detail, and a positive attitude.