Principal Architect - Reinforcement Learning (Physical AI)

Career Area:

Job Description:

Job Summary: 

The Learning Systems Architect owns how the machine learns — the end-to-end system that turns simulation, digital twins, and fleet data into high-performance, field-proven policies and models. 

The Principal Architect – Reinforcement Learning (RL) Systems is responsible for the architecture of the learning system that produces Caterpillar’s Physical AI models: the reinforcement learning, imitation learning, and foundation-model post-training pipelines that train perception, prediction, planning, and control policies for excavators, dozers, loaders, and haul trucks. Where the Simulation Engine architect owns the runtime the policy trains in, the Digital Twin architect owns the fidelity of what it trains against, and the AI Factory architect owns the compute and data platform it trains on, this role owns the learning problem itself: how tasks are formulated as observations, actions, rewards, and curricula; which policy and model architectures are used; how large-scale closed-loop training in simulation is structured; how sim-to-real transfer is achieved and measured; and how learned policies are evaluated, made safe, and deployed onto machines. 

The architect defines the reference architecture for learning-based autonomy at Caterpillar: Gym-style task and environment specifications on top of Isaac Lab / Isaac Sim, Omniverse, and hybrid neural simulation (NVIDIA Cosmos-class world models); massively parallel GPU-based RL (PPO, SAC, GRPO and successors) and imitation learning (behavior cloning, distillation, offline RL) at fleet-data scale; vision-language-action and foundation-model post-training with RL; domain randomization, curriculum, and system-identification strategies for sim-to-real transfer; reward and metric design tied to productivity, safety, machine health, and operator-quality outcomes; and the policy evaluation, regression, and safety-case methodology that gates deployment. The role sets standards for reproducibility, experiment rigor, and benchmark hygiene, and partners with the Experimentation & Operations architect to run training and evaluation at scale. 

This role that turns research into repeatable engineering, and is measured by field task performance of deployed policies — not by simulation benchmarks alone. 

What You Will Do: 

• Developing detailed architecture deliverables to solve business problems. 

• Designing an application's technical infrastructure, such as specific databases, programming languages, utilities, and testing approaches. 

• Leading the evaluation and deployment of new technologies to add or enhance existing digital technical capabilities. 

• Participating in addressing business requirements for applications and collaborating with cross-functional teams to deliver digital solutions that meets business results. 

What You Will Have: 

• Requirements Analysis: Knowledge of tools, methods, and techniques of requirement analysis; ability to elicit, analyze and record required business functionality and non-functionality requirements to ensure the success of a system or software development project. 

• Analytical Thinking: Knowledge of techniques and tools that promote effective analysis; ability to determine the root cause of organizational problems and create alternative solutions that resolve these problems. 

• Platform Architecture: Knowledge of technologies and methods to design processing mechanisms and roadmaps to execute business application systems; ability to design these roadmaps and deploy supportive interfaces for end-users to access related systems, in accordance with standards and processes. 

• Target Architecture: Knowledge of target architecture; ability to develop the IT blueprint and roadmap while aligning the architecture and processes with business strategies and objectives. 

• Effective Communications: Understanding of effective communication concepts, tools and techniques; ability to effectively transmit, receive, and accurately interpret ideas, information, and needs through the application of appropriate communication behaviors. 

Top Candidates Will Have: 

• Reinforcement Learning for Physical Systems: Typically 8+ years (or PhD plus 5+) in robot learning with RL taken from algorithm to deployment on real hardware: PPO, SAC, GRPO, model-based / world-model RL, reward shaping, curriculum, and compounding-error handling in closed-loop training. 

• Imitation Learning and Foundation-Model Post-Training – Behavior cloning, offline RL (IQL, GAIL), distillation, diffusion policies; vision-language-action or robot foundation models with RL-based post-training (RLHF / DPO / GRPO). 

• Large-Scale Closed-Loop Training in Simulation – Architected RL frameworks on GPU-parallel simulators (NVIDIA Isaac Lab / Isaac Sim, MuJoCo, Brax) with Ray, Kubernetes, or Slurm; used neural / hybrid simulation (NVIDIA Cosmos, neural reconstruction) for training or augmentation. 

• Sim-to-Real Transfer – Closed sim-to-real gaps via domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction, measured by field task success; hydraulic or heavy-machinery dynamics highly valuable. 

• Policy Evaluation and Safety – Metrics for learned policies (safety, progress, productivity, comfort), statistically rigorous policy comparison, continuous evaluation and regression, adversarial / long-tail scenario testing, runtime safeguards; safety-case thinking (ISO 26262 / SOTIF / ISO 13849). 

• Controls and Robotics Foundations – MPC, impedance control, trajectory optimization; judgment on learned versus classical approaches; perception-to-action pipelines under real-time constraints. 

• Data Flywheel Literacy – Shapes demonstration and fleet data collection, auto-labeling, and synthetic data strategy for imitation and offline RL. 

• Software Engineering – Strong Python and PyTorch (JAX a plus), working C++ for real-time integration, CUDA familiarity, experiment tracking, Bazel or monorepo tooling, CI/CD. 

• Architecture Leadership – Turns research into repeatable engineering; sets reproducibility and benchmark standards; mentors ML engineers and researchers; led a shipped learning-based capability. 

• Domain Exposure – Off-highway, construction, or mining autonomy (Bedrock Robotics, Built Robotics, Applied Intuition), or robot learning at NVIDIA (Isaac / GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus. 

• Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or related field preferred; research or publication record a plus. 

Additional Details:

• This position requires the candidate to work full-time at the Irving, TX office (Dallas)
• Domestic relocation assistance is available for this position.
• Visa sponsorship is available with this position.

Summary Pay Range:

Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar. 

Benefits:

Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits.

• Medical, dental, and vision benefits*

• Paid time off plan (Vacation, Holidays, Volunteer, etc.)*

• 401(k) savings plans*

• Health Savings Account (HSA)*

• Flexible Spending Accounts (FSAs)*

• Health Lifestyle Programs*

• Employee Assistance Program*

• Voluntary Benefits and Employee Discounts*

• Career Development*

• Incentive bonus*

• Disability benefits

• Life Insurance

• Parental leave

• Adoption benefits

• Tuition Reimbursement