Description We're seeking a Software Engineer to bridge the critical gap between cutting-edge ML research and production-ready solutions. You'll transform research prototypes into robust, deployable systems that end users can confidently put into production. This role uniquely combines software engineering, DevOps practices, and ML solution delivery.
- AI Model Benchmarks: Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
- Agentic AI Benchmarks: Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
- Evaluation Dataset Development: Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
- Metrics, Analysis, and Reporting: Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.
Clearance A current Top-Secret/SCI with polygraph security clearance is required. Candidates cannot be sponsored or nominated for a government security clearance under this position.
Experience Seven (7) years' experience as a SWE in programs and contracts of similar scope, type, and complexity is required. Bachelor's degree in Computer Science or related discipline from an accredited college or university is required. Four additional (4) years of SWE experience on projects with similar software processes may be substituted for a bachelor's degree.
Required skills:
- InspectAI
- Python
- MongoDB
- Jupyter notebooks
- Data visualization tools
- Statistical analysis
- Data cleaning and validation
- Cloud or containerized environments
- Git-based version control
- Collaboration tools such as Jira or Confluence
- Experience operating in airgapped or constrained environments
Benefits Erias Ventures provides a complete package of wealth, health, and happiness benefits. The expected salary range for this position, depending on education and years of experience is $210,000 - $230,000.
Wealth Benefits: - Above Market Hourly Pay
- 11% Roth or Traditional 401k with Immediate Vesting and Deposit
- Spot Bonuses for Assisting with Business Development and Company Growth
- Professional Development Bonuses for Certificates and Degrees
Health Benefits: - Company subsidized Medical Coverage
- 100% Company Paid Vision and Dental Coverage
- 100% Company Paid Long Term Disability, Short Term Disability, and Group Life Insurance
- Monthly Wellness Reimbursement
Happiness Benefits: - Paid Time Off with Flexible Work Schedules and Birthday Off
- Amazon Prime Membership andMonthly Internet Reimbursement
- Technology and Productivity Allowance for Equipment and Supplies
- Morale Building and Company Events to Celebrate our Successes and Build our Community
- Onboarding and Annual Swag
- Company Paid Professional Development and Training
Referrals & Inquiries Do you know a cleared professional seeking to advance their career? Interested in earning some extra cash? If so, refer them to us with their name and contact details, and you could be eligible for a referral bonus of up to
$10,000 for each successful hire.
Not seeing the right position right now? Reach out to us, and we'll notify you as new contracts and opportunities become available!
Please send referrals and inquiries to:
[email protected] To learn more about our company visit our webpage or LinkedIn . 
