Bank of America Corporation

AI Engineer - Guardrails and Observability

Bank of America Corporation$130K — $155K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's or Master's degree in Computer Science, Data Science, or related field, or equivalent experience.
  • 10+ years of software engineering experience building and scaling production applications.
  • Strong Python development experience including common ML and data libraries.
  • Hands-on experience with OpenShift or Kubernetes in production environments.
  • Experience deploying and supporting LLM or RAG-based applications.
  • Understanding of AI risks, agent behavior, and governance frameworks.
  • Experience integrating third-party platforms and supporting vendor applications.

Responsibilities

  • Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
  • Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications.
  • Monitor application performance, safety, latency, throughput, and guardrail metrics.
  • Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
  • Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
  • Build observability dashboards and alerting solutions.
  • Troubleshoot production issues and optimize performance, scalability, and resilience.

Benefits

  • Opportunity to work with emerging AI technologies and frameworks.
  • Professional development through mentorship and leading cross-functional initiatives.
  • Engagement with high-impact projects in a fast-paced, innovative environment.
Full Job Description

Job Description:

This role centers on engineering and operating third-party applications end to end within an on-premises OpenShift platform. The position supports GenAI operations, including guardrails and observability for LLM and RAG-based applications across the enterprise. The engineer will deploy and support vendor platforms such as Galileo and LangSmith, monitor AI application quality and safety, maintain model evaluation workflows, and provide operational support in a fast-paced environment.

The domain is GenAI operations: the applications you run provide guardrails and observability for LLM- and RAG-powered systems across the enterprise. You will work with these vendor tools to monitor application quality, safety, and performance—tracking metrics such as attribution, grounding, prompt-injection detection, tone, bias, and PII handling—and surface them through dashboards and alerting. Where the platform uses model-based evaluation (LLMs-as-Judges and specialized/small language models), you will help operationalize, calibrate, and maintain those workflows.

Because the AI landscape changes quickly, the ideal engineer learns new tools and environments fast, brings a pragmatic build-and-operate mindset, and can lead cross-functional work to move initiatives forward.

Responsibilities:

  • Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
  • Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications.
  • Monitor application performance, safety, latency, throughput, and guardrail metrics.
  • Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
  • Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
  • Build observability dashboards and alerting solutions.
  • Troubleshoot production issues and optimize performance, scalability, and resilience.
  • Contribute to emerging AI evaluation and guardrail strategies for agentic workflows.
  • Lead and mentor engineers while driving cross-functional initiatives and engineering best practices.

Required Qualifications:

  • Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
  • 10+ years of software engineering experience building and scaling production applications.
  • Strong Python development experience, including common ML and data libraries.
  • Hands-on experience with OpenShift or Kubernetes in production environments.
  • Experience deploying and supporting LLM or RAG-based applications.
  • Understanding of AI risks, agent behavior, and governance frameworks.
  • Experience integrating third-party platforms and supporting vendor applications.
  • Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
  • Knowledge of inference performance optimization and scalability.
  • Experience implementing AI guardrails, observability, and model evaluation processes.
  • Experience working in Agile environments.
  • Proven ability to lead technical initiatives and influence stakeholders.
  • Strong analytical, problem-solving, and communication skills.

Desired Qualifications:

  • Experience delivering technology solutions within large, regulated enterprise environments, preferably financial services.
  • Experience with annotation pipelines, feedback loops, fine-tuning, and alignment techniques.
  • Familiarity with prompt lifecycle management and versioning.
  • Experience with Galileo, LangSmith, Splunk, or similar observability platforms.
  • Familiarity with LlamaIndex, Haystack, or other GenAI frameworks.
  • Experience with evaluation and guardrails for agentic AI workflows.
  • Self-directed, collaborative working style across engineering teams.

Skills:

  • Application Development
  • Automation
  • Influence
  • Solution Design
  • Technical Strategy Development
  • Architecture
  • Business Acumen
  • DevOps Practices
  • Result Orientation
  • Solution Delivery Process
  • Analytical Thinking
  • Collaboration
  • Data Management
  • Risk Management
  • Test Engineering

Shift:

1st shift (United States of America)

Hours Per Week: 

40

About Bank of America Corporation

Bank of America Merrill Lynch is the corporate and investment banking division of Bank of America. It provides services in mergers and acquisitions, equity and debt capital markets, lending, trading, risk management, research, and liquidity and payments management. It was formed through the combination of the corporate and investment banking activities of Bank of America and Merrill Lynch following the acquisition of the latter by the former in January 2009. Bank of America completed the acquisition of Merrill Lynch & Co on 1 January 2009. Bank of America began rebranding all of its corporate and investment banking activities under the Bank of America Merrill Lynch name in September 2009. In April 2010, Bank of America Merrill Lynch appointed Christian Meissner as head of investment banking for Europe, Middle East and Africa. In April 2011, Bank of America Merrill Lynch integrated its corporate and investment banking operations into a single division. In October 2013, Bank of America Merrill Lynch was recognised as the Most Innovative Investment Bank of the Year in The Banker's Investment Banking Awards.

Bank of America Corporation Careers

Join the dynamic team at Bank of America Corporation, a premier global financial institution where innovation, leadership, and growth go hand in hand. As one of the largest banks in the world, we offer unparalleled job opportunities and a culture that values diversity, inclusion, and professional growth. Work You’ll Do At Bank of America Corporation, you’ll be part of a team that’s dedicated to making a real difference. Whether you’re helping families buy their first home, advising businesses on expansion, or developing cutting-edge financial technologies, your work will have an impact. Our commitment to leadership in the financial industry has never been stronger, and we need passionate, skilled professionals to lead our journey. Explore a World of Opportunities From entry-level positions to leadership roles, Bank of America Corporation offers a variety of career paths in areas such as investment banking, technology, marketing, and risk management. Our job opportunities span the globe, providing the chance to work alongside the best in the industry and develop skills that will propel your career forward. Internship Programs Kickstart your career with Bank of America Corporation’s internship programs. These opportunities provide hands-on experience and a chance to engage in meaningful work that complements your academic studies. Interns gain invaluable networking opportunities, receive mentorship from seasoned professionals, and learn about the culture and operations of a global financial leader. Benefits and Growth Bank of America Corporation is committed to the well-being and continuous professional development of our team members. We offer a competitive benefits package that supports the health, financial stability, and work-life balance of our employees. Our training programs and development initiatives ensure that every team member has the opportunity to grow and advance within the company. Inclusive Culture We believe our strength lies in our diversity. Bank of America Corporation fosters an inclusive environment where all employees can thrive. Through diversity training and a commitment to equal opportunities, we cultivate leadership and innovation that reflect the wide-ranging communities we serve. Join Our Team Are you ready to advance your career at a company that’s at the forefront of the financial industry? Explore the positions available at Bank of America Corporation and find where your skills and interests align with our needs. We are continuously hiring and looking for individuals who are curious, creative, and eager to drive change. Stay Connected Keep up to date with the latest from Bank of America Corporation Careers by subscribing to our job alert emails. Tailor your subscription to receive updates that match your career interests and get insider tips that can help you during your application and interview process. Bank of America Corporation is not just a company—it’s a place where you can shape your future and the future of finance. Join us and be part of a team that’s redefining what a bank can be.
Learn more about Bank of America Corporation
Size
208,000 employees
Market Cap
$260.3 billion
Industry
Net Income
$17.8 billion
Founded
1998
5 Year Trend
-1.4%
NASDAQ

Similar Jobs

More Jobs at Bank of America Corporation

More Information Technology Jobs

Find similar AI Engineer - Guardrails and Observability jobs: