Scope of the Role: Innodata partners with leading foundation model labs, hyperscalers, and enterprise AI teams to build the data, evaluation, and post-training systems that make modern LLMs trustworthy and production-ready.
As a Technical Solutions Architect for Evals & Fine-Tuning, you are the technical face of Innodata to our most demanding customers. You sit at the intersection of client AI/ML teams, our research scientists and ML engineers, our subject-matter expert workforce, and our platform teams. You translate ambiguous customer goals - "improve factuality on long-context legal QA," "build a safety eval suite for our next model release," "design a DPO pipeline for our coding assistant" - into concrete, scoped, deliverable engagements.
This is a senior individual-contributor role for someone who has done the work: built fine-tuning pipelines, designed eval harnesses, argued with stakeholders about benchmark validity, and earned credibility with sophisticated ML buyers.
What You'll Own:- Lead technical discovery with prospective and existing customers - foundation model labs, frontier AI teams, and large enterprises - to understand model objectives, gaps, and constraints.
- Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context).
- Architect engagements that combine Innodata's platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains.
- Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
- Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly.
- Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent.
- Feed customer signal back into Innodata's R&D and product roadmap - what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction.
- Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation).
- Represent Innodata externally - at customer reviews, conferences, and in technical content.
You'll Thrive in This Role If You Have:- 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training.
- Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
- Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design.
- Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents).
- Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement.
- A consultative mindset: you ask sharp questions, you push back when a customer's stated request won't actually solve their problem, and you are comfortable owning a recommendation.
- Bachelor's or advanced degree in computer science, machine learning, computational linguistics, or related field - or equivalent demonstrated experience.
The expected salary range for this position is $190,000 - $225,000 USD per year, based on experience, skills, and qualifications.
