Bachelor's degree in Computer Science, Machine Learning, or related field.
2+ years of professional ML experience with end-to-end production ownership.
Proficiency in Python and PyTorch for model training and evaluation.
Experience with training or distilling language models in production environments.
Familiarity with ML serving frameworks and optimization techniques.
Responsibilities
Train and fine-tune small language models for real-time enforcement.
Design multi-stage inference pipelines for various workloads.
Build synthetic data pipelines to enhance model training.
Develop evaluation frameworks to catch model regressions quickly.
Collaborate cross-functionally to translate customer needs into model solutions.
Benefits
Comprehensive health coverage including medical, dental, and vision insurance.
Generous paid time off and sick leave policies.
401(k) plan with company match to support retirement savings.
Equity options as part of the compensation package.
Professional development opportunities for career advancement.
Full Job Description
Nature of the Specialized Duties • Training, Fine-Tuning, and Distilling Production Small Language Models and Classifiers (25% of time)
Owning the full training lifecycle for the SLMs and classifiers in SAGE's real-time enforcement path, including base-model selection, supervised fine-tuning, preference optimization (DPO/RLAIF), and distillation from frontier teacher models.
Training anomaly and action-severity models that catch novel agent-side attack patterns at real-time decision latency, such as supply-chain compromises or emergent destructive behaviors not covered by any explicit policy. Severity scores route the highest-impact events to Agent Rewind for precise remediation.
Designing adversarial training pipelines like purpose-built adversarial agents and automated red-teams whose outputs feed directly into the next training run, turning every discovered weakness into a permanent model improvement.
Pushing the pareto frontier of accuracy, latency, and cost for governance-specific tasks through deliberate post-training choices (LoRA, quantization-aware training, distillation recipes, GRPO, etc.) and validating the wins on production traffic patterns.
• Engineering High-Performance Model Serving and Inference Infrastructure (25% of time)
Designing multi-stage inference pipelines that handle both real-time enforcement (inline prompt, response, and tool-call blocking) and high-throughput batch workloads (offline scoring, back-testing, corpus mining) while processing billions of tokens daily across Global 2000 customer agent fleets.
Optimizing live deployments through shared GPU pools, KV-cache-aware routing, continuous batching, FP8/INT8 quantization, and speculative decoding to minimize inference cost while holding sub-second P99 SLOs.
Building serving-layer infrastructure that lets SAGE block agent prompts, responses, and tool calls in real time without becoming a latency bottleneck. This includes model gateway design, request routing, and graceful degradation.
Owning canary, shadow, and A/B traffic patterns so new model variants are validated against live customer traffic before they take enforcement decisions.
• Building Synthetic Data Pipelines and Online + Offline Evaluation Frameworks (20% of time)
Designing automated data curation pipelines that mine live customer environments (with privacy and tenancy guarantees) for high-value per tenant training examples, such as long-tail violations, near-miss policy edges, or novel agent behaviors, and routing them back into the training loop for each customer.
Building automated policy back-testing by replaying historical agent traffic against new model and policy versions to catch regressions and recommend policy improvements before customer-visible deployment.
Building online evaluation systems for live model decisions, including shadow scoring, drift detection, calibration monitoring, and policy-coverage gap analysis, ensuring quality regressions surface in minutes rather than weeks.
Generating synthetic data using frontier teachers (adversarial prompts, policy-edge cases, multi-turn interactions) with evaluation that confirms synthetic data improves downstream quality, not just dataset size.
• Insights Mining, Failure Diagnosis, and Adaptive Model Improvement (15% of time)
Building memory and context harnesses that fuse data sensitivity, identity, and historical agent behavior into real-time enforcement decisions to ensure SAGE reasons from each customer's specific context.
Mining agent insights across millions of sessions to surface security gaps, which are then turned into new policy proposals, refinements to existing policies, and signals about upstream issues across the agent ecosystem (Google ADK, Azure AI Foundry, Vertex AI, and others).
Building feedback loops that turn production decisions, customer-flagged false positives, and missed violations into one-click natural-language policy refinements to drive false-positive rates down without sacrificing recall.
Diagnosing model failures end-to-end and distinguishing data, training-recipe, architecture, and serving-layer root causes so fixes land in the right layer the first time.
• Cross-Functional Collaboration and Translating Customer Reality into Modeling Problems (15% of time)
Providing technical leadership on a pillar of the SAGE model stack (training infrastructure, eval methodology, serving architecture, or insights pipeline), mentoring engineers ramping into ML, and shaping the team's technical roadmap.
Partnering with Product Management, customer-facing teams, and security analysts to translate customer agent-governance requirements into well-scoped modeling problems, and pushing back when ML is the wrong tool.
Communicating model behavior, tradeoffs, and limitations clearly to non-ML stakeholders, such as product managers and enterprise security leaders, so model decisions are made with full context.
Collaborating with Agent Cloud platform, security engineering, and AI research teams to integrate new SLMs into the real-time enforcement path with the right latency, observability, rollback, and tenancy guarantees.
Minimum Requirements for the Position
Education: A Bachelor's degree (or higher) in Computer Science, Machine Learning, Computer Engineering, Statistics, or a closely related technical field is required. Designing production SLM training and serving systems requires a deep theoretical understanding of modern deep learning, optimization, and systems performance. Specialized Technical Knowledge:
2+ years of professional ML experience with demonstrable end-to-end production ownership; you have taken models from training to serving real customer traffic and stayed accountable for them through post-launch iteration.
Proficiency in Python and PyTorch (or equivalent) for production-grade training and evaluation.
Hands-on experience training, fine-tuning, or distilling language models or classifiers in a production setting, including SFT and at least one preference-optimization technique (DPO, RLAIF, or RLHF).
Production experience with serving frameworks (vLLM, SGLang, TensorRT-LLM, or equivalent), including optimization involving continuous batching, KV-cache strategy, and inference-time quantization.
Experience designing closed-loop ML systems, including the eval, telemetry, data-curation, and synthetic-data infrastructure that turns production signals back into training data and the next model release. You have built (not just used) at least one such loop.
Comfort operating at production scale, including debugging models that handle high QPS in safety-critical request paths where errors have customer-visible consequences.
Preferred Qualifications:
Deep background in AI safety and red-teaming, including hands-on experience with adversarial ML, prompt injection defense strategies, and automated evaluation suites for enterprise-grade LLM safety.
Expertise in model evaluation methodology, specifically building "LLM-as-judge" pipelines, calibration monitoring, and adversarial benchmarks that surface the subtle failure modes static metrics often overlook.
Experience with context-fusion and retrieval systems that synthesize disparate signals - such as data sensitivity, user identity, and behavioral history - into high-fidelity model decisions.
Production experience with low-latency inference for streaming or safety-critical request paths where model throughput and P99 SLOs are paramount.
Mastery of label-efficient training and data mining, utilizing weak supervision, active learning, and embedding-based retrieval to surface the production examples that drive the most significant quality improvements.
Hands-on knowledge distillation experience, successfully transferring capabilities from frontier teacher models to specialized, small-scale student models for production serving.
Familiarity with the agentic ecosystem, including tool-use frameworks, model gateway architectures (MCP, LiteLLM, or equivalent), and autonomous agent patterns.
Active open-source contributions to mainstream ML training, serving, or evaluation libraries.
The minimum and maximum base salaries for this role are posted below; additionally, the role is eligible for bonus potential, equity and benefits. The range displayed reflects the minimum and maximum target for new hire salaries for the role based on U.S. location. Within the range, the salary offered will be determined by work location and additional factors, including job-related skills, experience, and relevant education or training.
US Pay Range
$188,500-$282,700 USD
About Rubrik
Rubrik is a cloud data management company based in Palo Alto, California. The company offers a suite of software that provides backup and recovery, cloud migration, disaster recovery, and data governance services. Rubrik's software is designed to work across multiple cloud platforms and on-premises data centers. The company was founded in 2014 by Bipul Sinha, Arvind Jain, Soham Mazumdar, and Arvind Nithrakashyap. Rubrik has raised over $553 million in funding and is valued at over $3 billion.
