Everforth ECS is seeking a Senior Evaluation Automation Engineer to work in the National Capital Region covering the Pentagon, Falls Church, and Fairfax. Please Note: This position is contingent upon contract award.
The War Data Platform (WDP) is a key initiative within the U.S. Department of War's (DoW) AI-First strategy introduced in early 2026. The WDP focuses on operational warfighting data and aims to accelerate the deployment of artificial intelligence (AI) on the battlefield. The WDP extends to Unclassified, Secret, and Top Secret environments, and supports collaboration between Combatant Commands, Joint Staff directorates, Senior Executive Service leaders, and operational analysts.
This position leads the automation of evaluation and testing for AI/ML models in secured DoW environments.
• Develops, builds, and maintains automated test and evaluation checks executed within unclassified-first deployment pipelines supporting artificial intelligence and machine learning model serving missions for DoW programs, Joint Staff analysts, Combatant Command elements, and Senior Executive Service leadership.
• Designs and sustains automated unit, regression, and evaluation harness components that validate model artifacts across multiple enclaves and security domains.
• Executes higher-domain parity test and evaluation gate checks and resolves enclave-specific constraints to preserve gate credibility across promoted releases.
• Implements automated scanning, packaging, and deployment workflows using Kubernetes, GitLab Continuous Integration, PyTest, SonarQube, Tenable Nessus, and hardened artifact pipelines to validate reliability, operational suitability, and mission alignment of production-ready models.
• Integrates evaluation harnesses into enterprise DevSecOps pipelines to support mission assurance objectives and operational readiness requirements.
• Builds automated workflows supporting API endpoint configuration, proxy provisioning, and model zoo onboarding activities aligned with enterprise artificial intelligence model serving architectures.
• Produces mission-critical deliverables including automated test suites, evaluation harness documentation, readiness scorecards, operational risk assessments, and deployment decision artifacts.
• Collaborates with Platform One, Cloud One, multi-national engineering teams, and cross-service mission partners to advance automation maturity, strengthen deployment consistency, and reinforce program value commitments across all environments.
• Supports Tier-4 incident response actions to maintain service-level agreements and operational continuity for mission stakeholders.
• Performs other duties as assigned.
• Current Secret security clearance with the ability to obtain and maintain a Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI).
• 10-12 years of experience developing automated evaluation and testing solutions for AI/ML models in secure environments.
• Strong proficiency with Kubernetes, GitLab CI, PyTest, SonarQube, and Tenable Nessus for automated test harnesses .
• Demonstrated ability to design and maintain multi-domain, enclave-aware deployment pipelines and gate checks .
• Experience with cross-domain data replication, model zoo integration, and API gateway configuration workflows.
• Agile development background with application lifecycle management and knowledge of programming languages such as Java, Python, and Ruby .
• CompTIA A+ certification (or equivalent foundational IT support knowledge).
