Senior Data Scientist

Traba is hiring a Senior Data Scientist to join the founding Agents team and lead measurement and modeling for our agentic platform from 01. You'll make the core calls on how agent quality is defined, measured, and improved; set the bar for statistical and scientific rigor; and build the evaluation, experimentation, and modeling foundations that every agent we ship is measured against. As a Senior Data Scientist at Traba, you'll own how we model and understand agent performance inside real customer workflows-capability, reliability, and unit economics-and partner with engineering, product, and operations leadership on the decisions that shape the platform. **Responsibilities:** - Provide strategic insights and recommendations to senior leadership through in-depth statistical analysis and modeling. - Design, build, and maintain the metrics, models, and reporting that track agent quality, reliability, adoption, and unit economics for stakeholders across the Agents and Operations teams. - Build evaluation and experimentation as a first-class discipline-datasets from production traces, rubrics, automated graders, regression suites, and the experiment design and analysis that prove causation-so every agent improvement we ship is backed by evidence. - Identify key business challenges and opportunities-including agent failure modes, tool-use patterns, and cost and latency-and build statistical and machine-learning models to drive product improvements and growth initiatives. - Architect scalable analytics and modeling infrastructure to ensure data integrity, governance, and accessibility for both human and agent consumers. - Oversee the development and maintenance of Traba's data warehouse to ensure data availability and governance. - Work closely with the Agents team and Operations leadership to understand their data needs and provide actionable, statistically grounded insights that drive continuous process improvement and operational efficiency. - Provide Operations teams with models, self-service analytics, and advanced technologies-including AI-assisted tools-enabling them to independently analyze operational data and optimize their daily activities. - Mentor the scientists and analysts who build alongside you, and set the standards that define what "good" looks like for measurement, modeling, and experimentation at Traba. **Qualifications:** - Experience: 7+ years in data science, machine learning, applied statistics, or quantitative research, with 2+ years of hands-on work modeling or measuring LLM- or agent-based systems in production. - Education: BS/MS/PhD in data science, statistics, machine learning, computer science, mathematics, economics, or a related quantitative field (or equivalent work experience). - Technical Skills: - Strong proficiency in Python and common ML and statistics libraries (e.g., scikit-learn, PyTorch, pandas, statsmodels). - Strong proficiency in SQL. - Experience designing and analyzing experiments (A/B testing) and applying statistical inference or causal methods. - Experience with LLM evaluation and observability tools like Langfuse, Braintrust, or internal harnesses, and with building automated evaluators. - Communication Skills: Excellent data storytelling skills to effectively engage with stakeholders. - Collaboration Skills: Strong ability to work across departments, identifying and prioritizing analytics problems to deliver actionable insights. - Curiosity and Initiative: Intense curiosity to ask "why?" and use data to find answers, combined with a "no task too small" mentality. - Self-Motivation: Ability to work independently and as part of a team in a fast-paced startup environment. **Bonus Skills:** - Experience with notebook tools like Jupyter, Hex, Hyperquery, or equivalent. - Experience with modern data stack tools like dbt or equivalent. - Experience building internal agents or MCP servers for analytics workflows, or prior work at a vertical AI or AI-native data company (e.g., Hex, Omni, dbt). - Experience fine-tuning, distilling, or rigorously evaluating LLMs, or applying causal inference and experimentation at scale. **Benefits:** - Start-up equity - Competitive Salary - 100% Paid health, dental & vision coverage - Dinner Provided via DoorDash, free DashPass & stocked kitchen for NY employees - 68d Commuter benefit - 3cb Gympass Benefit - 44 Additional: One Medical Membership, Gympass, HSA via Optum, Talkspace, HealthAdvocate, Teledoc Health **Salary Range Details** The compensation range for this position is set between $180,000 and $215,000, reflecting our market analysis and other relevant considerations. However, exceptions may be made for candidates with qualifications that significantly differ from those outlined in the job description. We also offer a highly competitive equity package designed to ensure you share meaningfully in the long-term success and upside of the business. The position is based onsite in New York City, five days per week, giving you the opportunity to collaborate closely with a high-performing team in a fast-paced, energetic environment. Being in person supports real-time decision-making, stronger teamwork, and the kind of creative problem-solving that drives meaningful impact.