eBay

AI Analytics Manager

eBay • $141K — $189K *
Toronto, ON M3C 0E3In-Person
Information Technology
5 - 7 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • Experience in building LLM applications and AI-powered analytics products.
  • Strong Python skills for developing AI application logic and backend services.
  • Understanding of LLM orchestration and context management.
  • Experience in RAG and retrieval systems for structured/unstructured data.
  • Familiarity with prompt engineering and its application in AI systems.
  • Expertise in designing LLM evaluations and automated quality checks.
  • Ability to turn ambiguous business questions into clear analytical workflows.

Responsibilities

  • Develop AI-powered analytics chatbots and conversational interfaces for user interaction.
  • Design agentic workflows for interpreting user intent and generating responses.
  • Build orchestration flows that connect LLMs with SQL engines and analytics APIs.
  • Develop mechanisms for translating analytical questions into reliable workflows.
  • Design retrieval architectures for structured and unstructured analytics knowledge.
  • Create evaluation frameworks to assess the quality and accuracy of AI outputs.

Benefits

  • Comprehensive medical plans and financial benefits.
  • Full range of paid time off including parental leave.
  • RRSP eligibility for retirement savings.
Full Job Description
Analytics is undergoing a fundamental shift-from traditional business intelligence and data science toward AI-native, conversational analytics products.

We are building a new generation of analytics experiences where business and product teams can interact with data through natural language, diagnose performance, uncover root causes, and take action with the support of AI agents.

We are looking for a technical, product-minded Data Scientist / AI practitioner to help build these experiences, with a particular focus on analytics chatbots, AI orchestration, retrieval and grounding, agentic workflows, and evaluation.

If you are motivated to move beyond traditional analysis into building intelligent systems that combine LLMs, enterprise data, analytical context, and business workflows, this is an opportunity to help shape how eBay interacts with data in the AI era.
About the Role

In this role, you will design and develop the intelligence layer behind AI-powered analytics chatbots and agentic analytics products.

You will work across the end-to-end conversational analytics stack-including LLM orchestration, intent understanding, retrieval, context engineering, tool calling, SQL/data access, analytical reasoning, response generation, and evaluation.

A key part of the role will be designing systems that can translate natural-language questions into reliable analytical workflows: identifying the right data and business context, retrieving relevant knowledge, invoking analytical tools, reasoning across results, and returning grounded and explainable answers.

You will partner closely with Analytics, Data Engineering, Product, and Engineering teams, as well as Legal, Privacy, Security, and Responsible AI, to move AI capabilities from experimentation into reliable enterprise products.
What You'll Do
Build Conversational & Agentic Analytics Systems
  • Develop AI-powered analytics chatbots and conversational interfaces that allow users to explore business performance using natural language.
  • Design multi-step agentic workflows that can interpret user intent, retrieve context, query data, perform analytical reasoning, and generate grounded responses.
  • Build orchestration flows connecting LLMs with SQL engines, analytics APIs, semantic layers, knowledge bases, and other analytical tools.
  • Design tool-calling patterns and agent routing strategies that determine which data sources, analytical workflows, or specialized agents should handle a request.
  • Develop structured response patterns supporting analytical outputs such as narratives, tables, charts, follow-up questions, and recommended actions.
  • Translate ambiguous analytical questions into deterministic and AI-assisted workflows that are reliable enough for enterprise decision-making.
Design the Retrieval & Context Layer
  • Build retrieval architectures across structured and unstructured analytics knowledge, including metric definitions, schemas, SQL examples, dashboards, analytical documentation, historical analyses, and business context.
  • Design RAG and context-engineering workflows using techniques such as semantic search, embeddings, metadata filtering, hybrid retrieval, reranking, and dynamic context assembly.
  • Develop strategies for retrieving structured context-including relevant schemas, tables, dimensions, metrics, and example SQL-to support accurate data querying.
  • Optimize knowledge organization, chunking, deduplication, indexing, and retrieval strategies for analytical use cases.
  • Design mechanisms to maintain and continuously update domain-specific analytics knowledge as definitions, schemas, and business logic evolve.
Develop AI Orchestration & Analytical Reasoning
  • Design LLM orchestration pipelines spanning intent classification, planning, retrieval, tool selection, query generation, execution, reasoning, and response synthesis.
  • Develop single-agent and multi-agent architectures, including specialized agents for different analytical domains or tasks.
  • Build workflows for natural-language-to-SQL, query validation, data retrieval, and grounded interpretation of query results.
  • Implement model routing and fallback strategies based on task complexity, latency, accuracy, context-window requirements, and cost.
  • Design conversation-state and context-management approaches for multi-turn analytical interactions.
  • Implement guardrails, validation layers, structured outputs, error handling, and fallback behaviors to improve system reliability.
Build Evaluation & Continuous Improvement Systems
  • Develop evaluation frameworks for accuracy, groundedness, retrieval quality, SQL correctness, analytical reasoning, latency, cost, and user usefulness.
  • Build golden evaluation datasets containing representative user questions, expected queries, analytical outputs, and business context.
  • Create automated regression testing for prompts, retrieval configurations, models, and orchestration changes.
  • Evaluate individual components of the chatbot pipeline-including retrieval, tool selection, SQL generation, reasoning, and final-answer quality.
  • Analyze user interactions, feedback, failure patterns, and agent traces to identify opportunities for product and system improvements.
  • Build proactive and reactive feedback loops that continuously improve analytics knowledge, prompts, retrieval, and agent behavior.
Productionize AI Analytics Capabilities
  • Develop Python-based services and reusable components supporting conversational analytics and agent workflows.
  • Partner with engineering teams to integrate AI capabilities with APIs, data services, authentication systems, caching layers, and enterprise infrastructure.
  • Support production considerations including observability, tracing, latency optimization, scalability, reliability, and cost management.
  • Prototype new LLM, retrieval, and agent architectures and help transition successful approaches into production.
  • Establish reusable patterns and technical standards that enable conversational analytics capabilities to scale across multiple business domains.
Partner Cross-Functionally
  • Work closely with Analytics and domain teams to encode metric definitions, analytical workflows, business logic, and domain knowledge into AI systems.
  • Partner with Product and Engineering teams to translate analytical use cases into scalable product capabilities.
  • Work with Legal, Privacy, Security, and Responsible AI partners to ensure AI analytics products are compliant, secure, explainable, and safe.
  • Balance rapid experimentation with production reliability, analytical accuracy, governance, latency, and cost efficiency.
Required Qualifications
  • Experience building LLM applications, AI-powered analytics products, conversational AI systems, or agentic workflows.
  • Strong Python skills and hands-on experience developing AI application logic, APIs, data workflows, or backend services.
  • Strong understanding of LLM orchestration, including tool/function calling, multi-step workflows, context management, and structured outputs.
  • Experience building RAG and retrieval systems across structured and/or unstructured enterprise data.
  • Experience integrating LLM applications with SQL databases, analytical APIs, data platforms, or query engines.
  • Strong understanding of prompt and context engineering, including system prompts, tool prompts, dynamic context construction, and prompt iteration.
  • Experience designing LLM evaluations, golden datasets, automated quality checks, and regression testing.
  • Strong understanding of analytics, business metrics, data models, and how users investigate and consume data-driven insights.
  • Ability to translate ambiguous business questions into well-defined analytical workflows and technical system designs.
  • Familiarity with privacy, security, safety, and governance requirements for enterprise AI systems.
Desired Qualifications
  • Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Machine Learning, Analytics, Information Systems, or a related technical field, or equivalent practical experience.
  • 7+ years of experience across data science, analytics engineering, machine learning, AI application development, or related technical roles.
  • Hands-on experience with agent frameworks and LLM application libraries such as LangChain, LangGraph, LlamaIndex, or equivalent technologies.
  • Experience with vector databases, embeddings, semantic search, hybrid retrieval, and reranking.
  • Experience with natural-language-to-SQL systems, semantic layers, metadata catalogs, or analytics query engines.
  • Experience building or integrating FastAPI or similar API-based services for AI applications.
  • Familiarity with containerized and cloud-native environments such as Docker and Kubernetes.
  • Experience with LLM observability, tracing, caching, model routing, and inference cost optimization.
  • Experience designing multi-agent systems or domain-specific agent architectures.
  • Experience developing enterprise-scale AI, analytics, or business intelligence products.
  • Experience partnering with Legal, Privacy, Security, or Responsible AI teams to productionize AI capabilities.
What Success Looks Like
  • Analytics chatbots reliably translate natural-language questions into grounded analytical workflows and accurate answers.
  • The orchestration layer effectively coordinates retrieval, data querying, analytical tools, specialized agents, and LLM reasoning.
  • Retrieval systems consistently provide the right business definitions, schemas, metrics, SQL examples, and analytical context for each request.
  • AI-generated responses are trusted because they are grounded in source data, explainable, reproducible, and aligned with business definitions.
  • Evaluation frameworks detect regressions across retrieval, SQL generation, reasoning, and final-answer quality before they reach users.
  • New analytical domains can be onboarded through reusable knowledge, retrieval, workflow, and evaluation patterns rather than bespoke chatbot development.
  • The platform can iterate quickly while maintaining enterprise standards for accuracy, reliability, security, compliance, latency, scalability, and cost.


Additional Details

The base pay range for this position is expected in the range below:
C$141,600 - C$189,000

Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience. The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including RRSP eligibility, various paid time off benefits, such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

This job posting relates to an existing vacancy within eBay.

About eBay

As a leader in automotive ecommerce, eBay Motors brings you millions of potential customers. Our new simplified listing plans give you unmatched levels of customization and control. So whether you're listing twenty cars a month or an entire lot, you know you're getting the value that you deserve.

eBay Careers

Joining eBay’s dynamic team offers more than just job opportunities; it’s a chance to be part of a global marketplace that values innovation, leadership, and diversity. As a leader in online retail, eBay provides a platform where ambitions flourish and career growth is fostered through comprehensive diversity training and professional development. Work You’ll Do At eBay, we empower you to drive your career path forward with a range of employment options, from internships to full-time positions. Our culture thrives on innovation and the pursuit of excellence in the digital commerce landscape. By joining our team, you’ll collaborate with skilled professionals who are adept at blending technology and creativity to achieve high-impact results. Explore eBay’s Diverse Opportunities Whether you’re starting out with an internship or seeking a leadership role, eBay’s global presence offers vast opportunities for professional growth and networking. Our commitment to diversity and inclusive culture allows every team member to feel valued and motivated. Innovate and Lead At eBay, every position contributes to our mission of creating a more connected and seamless e-commerce experience. We encourage our team to lead projects that challenge the status quo and promote innovation. With eBay, you’re not just filling a role; you’re part of a pioneering company that’s continually reshaping the world of online retail. Benefits and Growth eBay is dedicated to the well-being and development of our employees. We offer competitive benefits, leadership training, and opportunities for career advancement. Investing in our team’s growth is paramount, as they are the driving force behind our success. Join Our Team Ready to take the next step in your career? Explore the job opportunities at eBay where your skills and creativity can shine. We are hiring individuals who are passionate, curious, and eager to drive change in the e-commerce industry. Stay Connected Keep up to date with the latest career tips, company news, and industry insights available through our eBay Careers Blog. Personalize your experience by subscribing to job alert emails that match your career interests and skills. Apply Now Start your journey with eBay today. Search open positions that align with your professional aspirations and submit your resume. Our interview process is designed to understand your capabilities and potential to grow with us. Join eBay and be part of a team that’s committed to creating impactful change in the marketplace. Discover the opportunities waiting for you at eBay – a place where your career can thrive in an environment of innovation and leadership.
Learn more about eBay
Size
10,800 employees
Market Cap
$21.7 billion
Industry
Retail & Consumer Goods
Net Income
$5.6 billion
Founded
1995
5 Year Trend
+2.3%
Revenue
$10.7 billion
NASDAQ
EBAY
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at eBay

More Information Technology Jobs

Find similar AI Analytics Manager jobs:

NationwideToronto, ON