Position Overview:As a Senior AI Systems Engineer, you9ll build and run the machine learning and GenAI systems that power Benevity9s products. You9ll turn promising prototypes into reliable, production-grade features that help impact leaders do more good. You9ll own significant systems from design through monitoring, work closely with data scientists and product managers, and help raise the bar for the engineers around you. You9ll work closely with the Staff engineers on architecture and strategy, and lead delivery in your own area.
What you9ll do:- Design, build and operate ML and GenAI features end to end, from data ingestion and feature pipelines to model serving and monitoring.
- Build LLM-powered applications such as search, assistants and agents, including RAG pipelines, tool integrations and multi-step workflows.
- Create evaluation frameworks for GenAI systems that combine offline benchmarks, online metrics and human review.
- Implement guardrails for GenAI applications, including prompt injection defenses, hallucination mitigation and responsible AI practices.
- Put MLOps standards into practice: versioning, automated testing, A/B testing, drift detection and safe rollbacks.
- Translate business problems from product managers and data scientists into clear technical requirements and scoped delivery plans.
- Write design docs, take part in design reviews and give thoughtful code reviews.
- Mentor mid-level engineers on production ML practices and system design.
- Balance latency, cost and quality when choosing models, prompts and infrastructure.
- Find and fix bottlenecks in pipelines, serving and platform performance.
What you9ll bring:- Bachelor9s or Master9s degree in Computer Science, Mathematics or a related field, or equivalent professional experience.
- 5+ years of software engineering experience, including 3+ years building and deploying ML models in production.
- Hands-on experience across the ML lifecycle: training, evaluation, deployment, monitoring and retraining.
- Strong Python skills and experience with ML frameworks such as PyTorch, TensorFlow or scikit-learn.
- Experience with at least one major cloud platform (AWS, Azure or Google Cloud) and with distributed workloads.
- Working knowledge of data pipelines, feature engineering and large-scale data processing.
- Familiarity with ML tooling such as experiment tracking, model registries, feature stores or orchestration frameworks.
- Experience with CI/CD for ML, including automated validation and rollback strategies.
- Experience shipping LLM-based features, including prompt engineering, few-shot techniques and tool or function calling.
- Experience building RAG systems, including embeddings, vector databases and retrieval tuning.
- Exposure to agentic workflows and orchestration frameworks such as LangChain, LlamaIndex or ADK.
- Solid grasp of APIs, microservices and reliability fundamentals.
- Awareness of data governance, privacy, bias and explainability in AI systems.
- Clear communication and a collaborative approach with technical and non-technical partners.
Great-to-haves:- Certification in relevant cloud platforms or technologies.
