Virtusa Corporation

Senior Consultant

Virtusa Corporation$120K — $145K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 8-11 years of IT experience, with strong proficiency as a Software Development Engineer in Test (SDET)
  • Minimum of 3 years of experience in evaluating AI, Large Language Models (LLMs), and Agentic systems
  • Expert-level programming skills in Java, Python, and Node.JS
  • Hands-on experience in automated testing frameworks, particularly with Selenium or Playwright
  • Deep understanding of AI/ML concepts, especially Transformer architectures and Vector databases
  • Familiarity with evaluation metrics such as BLEU, ROUGE, and BERTScore
  • Experience leading and mentoring teams in niche AI testing disciplines

Responsibilities

  • Define and develop evaluation models for LLMs and Agentic workflows focusing on safety and reliability
  • Build and implement automated evaluation frameworks for LLMs and related systems
  • Integrate tools like LangSmith and TruLens into CI/CD processes
  • Deploy agentic frameworks on cloud platforms (AWS, GCP, Azure) using modern orchestration practices
  • Validate both functional and non-functional output quality for AI systems
  • Mentor and develop QA/SDETs into specialists in AI testing
  • Support presales activities and contribute to market authority through publications and open-source contributions

Benefits

  • Opportunities for mentorship and professional growth
  • Access to cutting-edge tools and technologies in AI testing
  • Collaborative work environment with agile onshore/offshore teams
  • Chance to contribute to impactful open-source projects
  • Engagement in thought leadership activities through article publication
Full Job Description
8-11 years in IT (Strong SDET) experience, including a minimum of 3 years dedicated to evaluating AI, LLM, and Agentic systems, preferably in a product-led organization or AI research lab. Define evaluation models for LLM and Agentic workflow safety and reliability, combining traditional QA foundations with next-gen AI testing paradigms to lead niche skills groups. Build automated evaluation frameworks for LLMs, RAG pipelines, and Agentic workflows. Implement "LLM-as-a-judge" benchmarks, HITL, and tools like LangSmith, TruLens, DeepEval, Phoenix, and Ragas into CI/CD. Deploy agentic frameworks (Perception/Reasoning/Action) in AWS, GCP, and Azure using containerization, service governance, and cloud orchestration. Validate functional output quality (hallucinations, relevance, tool-calling precision) and non-functional vectors (red-teaming, prompt injection, jailbreaking, toxicity, latency, cost). Mentor QA/SDETs into niche AI testing specialists, support presales, and drive market authority through articles and open-source benchmarks. Requires expert-level Java, Python, and Node.JS scripting alongside advanced hands-on automation with Selenium or Playwright, RestAssured, mobile testing, and strong manual testing foundations. AI/ML expertise must cover Transformer architectures, Vector DBs (Pinecone, Milvus), fine-tuning, and frameworks like LangChain, LlamaIndex, or AutoGen. Proven experience with BLEU, ROUGE, G-Eval, BERTScore, Context Precision, Faithfulness, MLflow, Weights & Biases, and cloud infrastructure-as-code. Requires a product mindset for testing non-deterministic systems, consultative exec-level communication, and management of onshore/offshore agile teams.

About Virtusa Corporation

Virtusa Corporation is a global provider of digital business strategy, digital engineering, and information technology (IT) services and solutions. The company helps its clients transform their businesses through innovative technology solutions that enable them to improve customer engagement, increase operational efficiency, and drive revenue growth. Virtusa Corporation was founded in 1996 and is headquartered in Southborough, Massachusetts. The company has over 20,000 employees and operates in over 30 countries around the world. Virtusa Corporation is committed to sustainability and has received numerous awards for its environmental and social responsibility initiatives.
Learn more about Virtusa Corporation
Size
22,883 employees
Market Cap
$1.5 billion
Industry
Net Income
$44.6 million
Founded
1996
5 Year Trend
+22.3%
Revenue
$1.2 billion
NASDAQ

Similar Jobs

More Jobs at Virtusa Corporation

  • Virtusa Corporation
    Senior Director - Niche Sales
    $160K — $200K *
    New York, NY 10025 (New York County)
    Healthcare
    In-Person
  • Virtusa Corporation
    Architect (ATC)
    $135K — $160K *
    Jersey City, NJ 07305 (Hudson County)
    Information Technology
    In-Person
  • Virtusa Corporation
    QA Automation engineer
    $95K — $115K *
    New York, NY 10025 (New York County)
    Technical Services
    In-Person
  • Virtusa Corporation
    Architect
    $120K — $145K *
    Jersey City, NJ 07305 (Hudson County)
    Enterprise Technology
    In-Person
  • Virtusa Corporation
    Lead Software Engineer
    $110K — $130K *
    Pittsburgh, PA 15237 (Allegheny County)
    Information Technology
    In-Person

More Information Technology Jobs

Find similar Senior Consultant jobs: