Key Responsibilities:Data Analysis & AI Performance- Analyze chatbot performance metrics such as user satisfaction, deflection rates, response accuracy, latency, and token usage.
- Monitor LLM model performance to detect quality issues, edge cases, and opportunities for prompt or response optimization.
- Perform deep-dive analyses on user feedback, conversation patterns, and intent classification to improve the product.
- Design and execute A/B tests to measure the impact of new features, prompt changes, and model configurations.
Operations & Process Optimization- Manage and optimize end-to-end workflows for model deployment, monitoring, and incident response.
- Build automated alerting systems for operational metrics, SLAs, and AI performance thresholds.
- Coordinate cross-functional workflows across data science, engineering, product, and business teams.
- Identify and resolve data quality issues, pipeline bottlenecks, and system inefficiencies through root cause analysis.
Analytics & Reporting- Create and maintain dashboards in SQL, Python, and BI tools (Databricks, Splunk, Dynatrace).
- Develop executive-level reports on chatbot performance, user trends, ROI, and contact center impact.
- Build data documentation, metadata standards, and analytics playbooks for stakeholder self-service.
- Track and report KPIs such as user engagement, task completion, escalation patterns, and cost per interaction.
Business Intelligence & Requirements- Translate business problems into data-driven solutions with measurable success criteria.
- Conduct qualitative and quantitative research to uncover user needs and feature opportunities.
- Write user stories with data-backed acceptance criteria, wireframes, and process flows.
- Lead requirements sessions and synthesize inputs into actionable insights and prioritized roadmaps.
AI-Specific Focus Areas- Monitor guardrail effectiveness, content safety, and compliance with Responsible AI policies.
- Analyze multi-turn conversation flows for issues like topic drift and response inconsistency.
- Evaluate RAG performance including citation accuracy and knowledge base coverage.
- Track model cost drivers such as token consumption and resource usage to optimize efficiency.
Here is what you need:- US Citizen (Public Trust eligible)
- Strong data analysis skills: SQL and Python.
- Working understanding of AI/ML concepts: LLMs, prompt engineering, model evaluation metrics, NLP fundamentals.
- Experience with BI/visualization tools: Databricks, Splunk, Tableau, PowerBI, or similar.
- Knowledge of conversational AI, chatbot analytics, and customer experience metrics.
- Ability to communicate complex technical findings clearly and translate insights into executive-ready recommendations.
Preferred experience:- Hands-on experience with GenAI chatbot analytics for high-volume user environments (1M+ monthly users).
- Direct involvement in LLM performance monitoring, guardrail testing, or RAG evaluation.
- Prior work optimizing AI operational workflows such as model deployment or incident response.
- Experience designing A/B tests or conducting deep-dive analysis on conversational data.
- Familiarity with operational monitoring tools such as Splunk or Dynatrace.
As required by local law, Accenture Federal Services provides reasonable ranges of compensation for hired roles based on labor costs in the states of
California, Colorado, Hawaii, Illinois, Maine, Maryland, Massachusetts, Minnesota, New Jersey, New York, Vermont, Virginia, Washington, and the District of Columbia, and the city of Cleveland. The base pay range for this position in these locations is shown below. Compensation for roles at Accenture Federal Services varies depending on a wide array of factors, including but not limited to office location, role, skill set, and level of experience. Accenture Federal Services offers a wide variety of benefits. You can find more information on benefits here. We accept applications on an on-going basis and there is no fixed deadline to apply.
The pay range for the states of California, Colorado, Hawaii, Illinois, Maine, Maryland, Massachusetts, Minnesota, New Jersey, New York, Vermont, Virginia, Washington, and the District of Columbia, and the city of Cleveland is:
$100,200-$203,400 USD
