Career Area:
Technology, Digital and Data
Job Description:
As an AI Engineer, you will design, develop, and optimize advanced AI interactions to power production-grade software applications. You will move beyond basic prompt drafting to build robust, scalable, and reliable AI systems utilizing state-of-the-art Large Language Models (LLMs). Your role is critical in orchestrating autonomous AI agents, implementing Retrieval-Augmented Generation (RAG) pipelines, and establishing rigorous testing and observability frameworks. You will bridge the gap between complex business logic and machine intelligence, ensuring high-quality, safe, and deterministic AI behaviors.
What You Will Do:
Prompt Architecture & Tuning
- Design Advanced Prompts: Architect, version, and refine highly effective system, contextual, and dynamic prompts using advanced techniques (e.g., Chain-of-Thought, Tree-of-Thoughts, ReAct, Few-Shot learning).
- Mitigate Risks: Minimize model hallucinations, jailbreaking, and prompt injections through programmatic guardrails and robust adversarial prompting strategies.
- Model Agnostic Tuning: Adapt, benchmark, and fine-tune prompt structures across diverse open-source and proprietary frontier models.
Agentic Workflows & RAG Systems
- Build AI Agents: Design and orchestrate multi-agent systems and multi-step reasoning workflows using frameworks like LangGraph, LangChain.
- Optimize RAG Pipelines: Integrate and fine-tune Retrieval-Augmented Generation architectures, managing semantic search, chunking strategies, embedding models, and vector database queries.
- Tool Execution: Implement precise function-calling and tool-use prompts enabling LLMs to interact reliably with external APIs, databases, and legacy codebases.
Testing, Evaluation & Observability
- Build automated testing suites using RAGAS and Evals frameworks to assess RAG accuracy and enable continuous validation and benchmarking of Retrieval‑Augmented Generation outputs across datasets and Models .
- Agent Testing: Build automated regression testing suites and CI/CD pipelines to validate prompt stability, agent trajectory correctness, and tool-calling accuracy.
- Implement LLM Observability: Set up production monitoring and observability systems using platforms like Azure Application insights, Microsoft Foundry.
- Monitor Metrics: Track, analyze, and optimize production runtime metrics including semantic drift, token usage, cost efficiencies, and end-to-end latency.
Collaboration & Continuous Innovation
- Cross-Functional Delivery: Work alongside Product Managers, Data Scientists, and Software Engineers to translate commercial requirements into deterministic AI capabilities.
- Technology Scouting: Keep pace with rapid weekly advancements in LLM capabilities, evaluation frameworks, and orchestration libraries to sustain a competitive edge.
What You Have:
- Core Prompt Engineering: Expert-level mastery of structured prompt formats (XML, JSON wrappers), system-level steering, and context-window optimization.
- Production Programming: High proficiency in Python or TypeScript for constructing backend AI logic, processing asynchronous data streams, and handling API integrations.
- Orchestration Frameworks: Hands-on experience building apps using LangChain, LangGraph, native SDKs.
- Vector Infrastructure: Solid experience working with vector databases (e.g., Azure AI Search, AWS Open Search) and hybrid search retrieval methods.
- Data & ML Fundamentals: Strong capabilities in data manipulation (Pandas, NumPy), data cleaning, token handling, and foundational understanding of NLP transformers.
Top Candidates Will Have
- AI Observability Stacks: Prior implementation of Application Insights, OpenLLMetry, or custom evaluation/guardrail layers.
- Machine Learning Systems: Knowledge of deep learning architectures, fine-tuning frameworks (QLoRA, PEFT), and platforms like Hugging Face, PyTorch, or TensorFlow.
- Cloud Infrastructure: Experience deploying containerized AI microservices using Docker, Kubernetes, AWS, or GCP.
- Education: Bachelor’s or master’s degree in computer science, Data Science, Artificial Intelligence, Computational Linguistics, Mathematics, or a highly technical discipline.
- Experience: 2+ years of dedicated professional experience developing production software integrated with GenAI/LLM pipelines.
- Track Record: Proven portfolio demonstrating successfully deployed RAG systems or operational AI agents in commercial environments.
Additional Info:
• The required location for this role is East Peoria-IL, Irving-TX
Summary Pay Range:
$97,530.00 - $158,480.00
Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar.
Benefits:
Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits.
Medical, dental, and vision benefits*
Paid time off plan (Vacation, Holidays, Volunteer, etc.)*
401(k) savings plans*
Health Savings Account (HSA)*
Flexible Spending Accounts (FSAs)*
Health Lifestyle Programs*
Employee Assistance Program*
Voluntary Benefits and Employee Discounts*
Career Development*
Incentive bonus*
Disability benefits
Life Insurance
Parental leave
Adoption benefits
Tuition Reimbursement
* These benefits also apply to part-time employees
This position requires working onsite five days a week.
Visa Sponsorship is not available for this position.
Posting Dates:
October 27, 2026 - November 8, 2026
