WHAT YOU'LL DO - Run rigorous evaluations across frontier models (OpenAI, Anthropic, Google/Gemini) and continuously mature the stack, methodology, and process behind them.
- Define what makes a high-quality evaluation dataset and robust, cost-appropriate graders, and translate what end users care about into measurable criteria.
- Drive synthetic data generation and data-generation pipelines to ensure they are high-quality, reliable, and scalable.
- Own online evaluations, turning real production usage into rich insight on how our product is actually used.
- Lead each model launch end to end, managing the process, the stakeholders, and the timeline.
- Own the external narrative for each model launch - partnering with Product Marketing on blog, social, and PR content, and creating video demos yourself.
- Build and manage deep relationships with model providers and AI companies - securing early access to frontier models, delivering technical deep-dives back to providers, and identifying collaboration opportunities on evaluations and benchmarking.
- Identify, pitch, and structure opportunities to collaborate with providers and other AI companies specifically on evaluations and benchmarking.
- Run evaluations to decide whether a model is production-ready, and own the decision of which models power agentic products at Box.
- Assess the compliance posture of production models, partner on model pricing decisions, and define how and when we deprecate models.
WHO YOU AREWe are an AI-first company. This means you approach your work with a growth mindset and find ways to leverage AI to help make faster, smarter decisions that will 10X your impact at Box.
- You have a minimum of 2 years of product management experience.
- You know the AI landscape deeply, follow frontier models closely, and can hold your own in a technical conversation with any model provider.
- You're technical and hands-on: statistically literate and comfortable enough in code (Python, SQL, notebooks) to build your own evals, run your own analyses, and prototype your own data pipelines.
- You've driven cross-functional launches across Product Marketing, Communications, Developer Relations, and Engineering.
- You're an excellent written and verbal communicator who can represent Box externally, presenting complex, technical topics simply to senior stakeholders.
- You operate with extremely high agency, taking ambiguous, undefined problems and driving them to resolution on tight timelines.
- You're fluent across many stakeholders-from Product Marketing and Sales to Security, Legal, and Compliance-adapting your communication to each audience.
- You're a natural disambiguator who makes messy, fast-moving AI problems simple and clear.
Preferred skills
- Direct experience with AI/ML products, model evaluation, or benchmarking.
- Experience managing external partnerships with vendors or providers.
Box lives its values, with community and in-person collaboration being a core part of our culture. Boxers are expected to work from their assigned office a minimum of 3 days per week.Your Recruiter will share more about how we work and company culture during the hiring process.
At Box, we believe unique and diverse experiences benefit our culture, our products, our customers, our company, and our world. We aim to recruit a passionate, high-performing workforce that reflects the world we live in.If you are head-over-heels about this role but unsure if you meet all the requirements, we encourage you to apply!
Box is committed to fair and equitable compensation practices. Actual base salary (or OTE if commissionable role) is dependent upon factors such as: knowledge, skill level, experience, and work location. This role is also eligible for equity and benefits. For more information, check out our benefits and perks.
United States Pay Range
$160,500-$200,500 USD
