Modern Technology Solutions

AI/ML Test Engineer

Modern Technology Solutions • $90K — $130K *
Springfield, VA 22153In-Person
Aerospace & Defense
Less than 5 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 4+ years in AI/ML testing or operational evaluation workflows.
  • Familiarity with automated testing and data validation processes.
  • Capability to handle multiple evaluation tasks in dynamic settings.
  • Experience in troubleshooting technical and data workflow issues.
  • Strong analytical and communication skills.

Responsibilities

  • Execute AI/ML evaluation activities supporting operational testing.
  • Perform integration testing and operational validation for AI/ML workflows.
  • Support automated evaluation pipelines for model performance assessment.
  • Develop and maintain data annotation workflows for evaluations.
  • Validate datasets and support model testing across various scenarios.
  • Conduct integration testing within representative operational environments.
  • Adapt evaluation workflows to meet changing mission requirements.
  • Collaborate with cross-functional teams for T&E execution.

Benefits

  • Opportunity to work in a high-security environment with cutting-edge technologies.
  • Access to collaborative work with industry experts and mission stakeholders.
  • Engagement in mission-focused activities impacting national security.
  • Potential for professional development in AI/ML advancements.
Full Job Description
AI/ML Test Engineer in Springfield, VA. Qualified candidates must be U.S. citizens with a TS/SCI clearance and CI Polygraph.

Responsibilities:
  • Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
  • Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
  • Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
  • Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
  • Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
  • Conduct integration testing of AI/ML models within operationally representative environments and workflows.
  • Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
  • Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
  • Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
  • Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
  • Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.


Qualifications Required:
  • 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
  • Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
  • Ability to support multiple evaluation activities across dynamic operational environments.
  • Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
  • Strong analytical, organizational, and communication skills.


Education Requirements:
  • Bachelor's degree in IT or STEM related field, or additional relevant experience in lieu of a degree.


Desired Qualifications:
  • Experience supporting DoD, NGA, or Intelligence Community programs.
  • Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
  • Experience working in Agile, DevSecOps, or cloud-based operational environments.
  • Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
  • Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.


Clearance Requirements:
  • Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.


#LI-RC2

About Modern Technology Solutions

Modern Technology Solutions, Inc. (MTSI) is a leading-edge engineering and technology company that provides critical solutions for complex military and commercial programs. MTSI has been providing engineering and technical services to the defense industry since 1993. The company has a strong focus on innovation and is committed to delivering the best solutions to its clients. MTSI has a highly skilled workforce of engineers, scientists, and technical experts who are dedicated to providing the highest quality services to its clients. The company has a proven track record of success and has received numerous awards and recognition for its work.
Learn more about Modern Technology Solutions
Size
1,000 employees
Industry
Information Technology
Founded
1993
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Modern Technology Solutions

More Aerospace & Defense Jobs

Find similar AI/ML Test Engineer jobs:

NationwideSpringfield, VA