Proficient in tuning neural networks/LLMs on custom datasets
Experience building AI/ML models in Python or R
Familiarity with deep learning frameworks like PyTorch, TensorFlow, or Keras
Skilled in GPU-accelerated computing
Strong SQL skills including CTEs and nested subqueries
Experience with GitHub and Jenkins for CI/CD
Responsibilities
Test and validate AI systems, focusing on LLM-based systems
Design and execute experiments to benchmark LLMs
Tune neural networks and LLMs for specific use cases
Build AI/ML models and ensembles using Python or R
Utilize deep learning frameworks for model development
Leverage GPUs for efficient training and evaluation
Write complex SQL queries against analytical datasets
Communicate findings and methodologies to diverse stakeholders
Benefits
9% retirement contribution
100% of family medical premiums covered
4 weeks of paid time off (PTO)
Opportunity for visible impact in a small business environment
Collaborative culture where your voice is valued
Full Job Description
Active TS/SCI required. U.S. citizenship required.
Location: On-site in McLean, VA (Full-Time).
Prove which LLMs actually work for the mission. You'll design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks so the team ships AI systems they can trust.
What you'll do
Test, evaluate and validate AI systems with a focus on LLM-based systems (including RAG testing)
Design and run experiments that benchmark LLMs against defined measures
Tune neural networks / LLMs on custom datasets and apply results to concrete use cases
Build AI/ML models and ensembles in Python or R (library selection, preprocessing, model assessment)
Use deep learning frameworks such as PyTorch, TensorFlow or Keras
Leverage GPUs for accelerated training and evaluation
Write solid SQL (CTEs, set operations, aggregates, nested subqueries) against analytical datasets
Work in GitHub and Jenkins-style CI for reproducible experiments
Communicate methodological choices and model results to technical and non-technical stakeholders
What you bring (required)
Active TS/SCI
Hands-on RAG testing experience (must-have)
Tuning neural nets / LLMs on custom datasets for real use cases
Building AI/ML models and ensembles in Python or R
PyTorch, TensorFlow or Keras (key)
GPU-accelerated computing (key)
Testing, evaluating or validating LLM-based AI systems (key)
Experiment design to benchmark LLMs (key)
SQL including CTEs, set ops, aggregates and nested subqueries
GitHub and Jenkins (or equivalent) version control / CI
Nice to have
Hugging Face Transformers library and hub
Building benchmarks that measure LLM performance
Cloud computing development and architecture
Flask or similar front-end/web frameworks
Text classification and topic modeling (scikit-learn or deep learning)
Semantic search applications
Tableau dashboards
Verification/validation test benches, Explainable AI (XAI), ONNX
A 9% retirement contribution, 100% of family medical premiums paid, and 4 weeks of PTO. We're a small business that's been delivering since 2019, so your work is visible and your voice counts.
