Do you want to shape Amazon's experimentation and impact-measurement capabilities in the era of Generative AI? As we navigate the rapidly changing landscape of e-commerce, your role will be crucial in shaping how Amazon teams, sellers, and vendors optimize catalog listings for a superior customer experience. Join the Catalog Experimentation and Impact Measurement (CEM) team on our mission to create the best product listings that help customers make well-informed, fast buying decisions. We build the feedback loops that let Large Language Models and Amazon teams enrich product data through experimentation at Amazon scale.
## What you'll own
In this role, you will lead the conceptualization, delivery, and worldwide adoption of the **Catalog Experimentation Platform (CEP)**. Through experiments, we bring the voice of the customer to our business stakeholders. You will collaborate with economists to help users design experiments correctly, and guide them in interpreting outcomes generated through rigorous statistical analysis. You will advocate for decisions in the best interest of our customers even when the outcomes are unpleasant for the experimenters.
A central part of your charter is **impact measurement**. Measuring the impact of every initiative that intends to improve product data quality is crucial for prioritization decisions and for fine-tuning each initiative's strategy. CEM operates a two-part measurement approach: one that estimates how improvements in product data quality translate into incremental sales and customer benefit, and another through which partner teams run controlled experiments while the team reviews their design and aggregates results across the organization. You will collaborate with economists to identify and apply the best possible causal-inference methodology for each initiative, and you will extend the scope of the platform to make it the go-to destination at Amazon for measuring the impact of catalog improvement programs.
Finally, to make experimentation and impact measurement work at Amazon scale without compromising reliability, you will collaborate with your engineering counterparts to build systems for automation, self-service mechanisms, and guardrails.
## Who you'll work with
You will partner closely with economists and scientists (for experiment design and causal inference), engineering counterparts (for automation, self-service, and guardrails), and the partner and enrichment teams across Amazon whose catalog programs depend on trustworthy impact measurement.
Key job responsibilities
1. Work backwards from the customer experience to define strategy and build a product roadmap for the Catalog Experimentation Platform (CEP), used to run A/B experiments on catalog listings.
2. Launch new products and features by partnering with CEP engineering and science teams.
3. Drive adoption by partnering with the teams building large language models for the catalog and with business stakeholders across different product categories.
4. Partner with economists and other product managers on the team to develop frameworks and shape how business stakeholders and leadership think about impact-measurement methodologies.
5. Represent CEP in Amazon-wide forums on experimentation and impact measurement.
A day in the life
Learn about the strengths and challenges of experimentation platforms used across the industry. Develop product strategy - positioning, go-to-market, and competitive advantage - in collaboration with your engineering and science counterparts. Track key product metrics and communicate wins, misses, and learnings to your key stakeholders. Conduct office hours to guide experimentation users. Communicate strategy and key updates to org leadership through monthly flashes and business-review forums.
About the team
Amazon's customers rely on the completeness, consistency, and correctness of Amazon's product data to make well-informed purchase decisions, and improving the quality of that product data is a continuous process. Amazon Selection Catalog Systems (ASCS) is building the Catalog Experimentation and Impact Measurement platform (CEM), which enables teams across Amazon to make data-driven decisions about which product-data changes will simplify and improve the customer experience. It empowers its users to remove personal bias and let data guide decision-making by testing their hypotheses against a representative sample of a broader population. Not least, CEM helps large language models draw causal inferences between catalog-quality improvement actions and customer-experience metrics.
BASIC QUALIFICATIONS
- 5+ years of product or program management, product marketing, business development or technology experience
- Bachelor's degree
- Experience with feature delivery and tradeoffs of a product
- Experience owning/driving roadmap strategy and definition
- Experience with end to end product delivery
- Experience contributing to engineering discussions around technology decisions and strategy related to a product
- Experience managing technical products or online services
- Experience in representing and advocating for a variety of critical customers and stakeholders during executive-level prioritization and planning
PREFERRED QUALIFICATIONS
- Bachelor's degree in math/statistics/engineering or other equivalent quantitative discipline
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 151,200.00 - 204,600.00 USD annually
