Virtusa Corporation

Senior Consultant

Virtusa Corporation$120K — $145K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 8-11 years of IT experience with strong expertise in SDET roles.
  • Minimum 3 years evaluating AI, LLM, and Agentic systems.
  • Expertise in Java, Python, and Node.JS scripting.
  • Proven experience with automated testing tools like Selenium or Playwright.
  • Advanced knowledge of AI/ML concepts, including Transformer architectures and Vector DBs.

Responsibilities

  • Define evaluation models for LLM and Agentic workflow safety.
  • Build and implement automated evaluation frameworks for AI systems.
  • Deploy agentic frameworks in cloud platforms like AWS, GCP, and Azure.
  • Validate functional and non-functional quality metrics.
  • Mentor and develop QA/SDETs into specialists in AI testing.
  • Support presales and enhance market authority through thought leadership.

Benefits

  • Opportunities for professional development and mentoring.
  • Access to cutting-edge technologies and methodologies.
  • Chance to lead and be at the forefront of AI testing.
  • Collaborative environment with agile onshore/offshore teams.
Full Job Description
8-11 years in IT (Strong SDET) experience, including a minimum of 3 years dedicated to evaluating AI, LLM, and Agentic systems, preferably in a product-led organization or AI research lab. Define evaluation models for LLM and Agentic workflow safety and reliability, combining traditional QA foundations with next-gen AI testing paradigms to lead niche skills groups. Build automated evaluation frameworks for LLMs, RAG pipelines, and Agentic workflows. Implement "LLM-as-a-judge" benchmarks, HITL, and tools like LangSmith, TruLens, DeepEval, Phoenix, and Ragas into CI/CD. Deploy agentic frameworks (Perception/Reasoning/Action) in AWS, GCP, and Azure using containerization, service governance, and cloud orchestration. Validate functional output quality (hallucinations, relevance, tool-calling precision) and non-functional vectors (red-teaming, prompt injection, jailbreaking, toxicity, latency, cost). Mentor QA/SDETs into niche AI testing specialists, support presales, and drive market authority through articles and open-source benchmarks. Requires expert-level Java, Python, and Node.JS scripting alongside advanced hands-on automation with Selenium or Playwright, RestAssured, mobile testing, and strong manual testing foundations. AI/ML expertise must cover Transformer architectures, Vector DBs (Pinecone, Milvus), fine-tuning, and frameworks like LangChain, LlamaIndex, or AutoGen. Proven experience with BLEU, ROUGE, G-Eval, BERTScore, Context Precision, Faithfulness, MLflow, Weights & Biases, and cloud infrastructure-as-code. Requires a product mindset for testing non-deterministic systems, consultative exec-level communication, and management of onshore/offshore agile teams.

About Virtusa Corporation

Virtusa Corporation is a global provider of digital business strategy, digital engineering, and information technology (IT) services and solutions. The company helps its clients transform their businesses through innovative technology solutions that enable them to improve customer engagement, increase operational efficiency, and drive revenue growth. Virtusa Corporation was founded in 1996 and is headquartered in Southborough, Massachusetts. The company has over 20,000 employees and operates in over 30 countries around the world. Virtusa Corporation is committed to sustainability and has received numerous awards for its environmental and social responsibility initiatives.
Learn more about Virtusa Corporation
Size
22,883 employees
Market Cap
$1.5 billion
Industry
Net Income
$44.6 million
Founded
1996
5 Year Trend
+22.3%
Revenue
$1.2 billion
NASDAQ

Similar Jobs

More Jobs at Virtusa Corporation

More Information Technology Jobs

Find similar Senior Consultant jobs: