Caterpillar, Inc

Lead Data Engineer – Physical AI Platform, Data Engineering

Caterpillar, Inc$128K — $208K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor’s degree in Computer Science, Computer Engineering, or related field
  • 8+ years of experience in data engineering or related disciplines with increasing responsibility
  • Extensive experience on complex Caterpillar data platforms, particularly Helios Data Platform
  • Strong experience developing and deploying Python solutions to production
  • Expertise in AWS data services such as Kinesis, S3, and DynamoDB at scale
  • Proficient in SQL, particularly in data quality and validation practices
  • Skilled in software deployment using CI/CD tools like Azure DevOps and Jenkins

Responsibilities

  • Collaborate with Principal Software Engineers and Data Architects to define solution architecture
  • Lead design and optimization of scalable data pipelines and microservices using Python
  • Drive development of cloud-native data ingestion and streaming solutions with AWS services
  • Own design and operational excellence of data integration frameworks supporting CI Autonomy
  • Partner with stakeholders to translate complex requirements into scalable data architectures
  • Establish automated testing, data quality controls, and validation frameworks
  • Lead monitoring and performance tuning of production data platforms using observability tools

Benefits

  • Medical, dental, and vision benefits
  • Paid time off plan (including vacation and holidays)
  • 401(k) savings plans
  • Health Savings Account (HSA)
  • Flexible Spending Accounts (FSAs)
  • Health Lifestyle Programs
  • Employee Assistance Program
  • Career Development opportunities
  • Incentive bonus
  • Tuition Reimbursement
Full Job Description

Career Area:

Technology, Digital and Data

Job Description:

Job Summary

As a Lead Data Engineer, you will design, build, and maintain scalable data pipelines, microservices, and cloud-based data platforms that deliver reliable, high-quality data for business and engineering teams. Working in an agile environment, you will help drive data architecture, performance, reliability, and continuous improvement across modern data solutions.

What You Will Do:

  • Actively collaborate with Principal Software Engineers and Data Architects to define solution architecture
  • Lead the solution design and optimization of scalable data pipelines and microservices in Python, enabling both real-time and batch data processing across enterprise platforms
  • Drive the development of cloud-native data ingestion and streaming solutions leveraging AWS services including Kinesis, S3, DynamoDB, EventBridge, and related technologies
  • Own the design, implementation, and operational excellence of data integration frameworks and source data pipelines supporting CI Autonomy initiatives
  • Partner with business, product, and engineering stakeholders to translate complex requirements into scalable data architectures, workflows, mappings, and system designs
  • Establish and enforce automated testing, data quality controls, and validation frameworks to ensure integrity, reliability, and compliance across distributed data ecosystems
  • Lead operational monitoring, performance tuning, and root-cause analysis of production data platforms using observability tools such as CloudWatch to maintain high availability and service reliability

What You Will Have:

  • Decision Making and Critical Thinking: Ability to lead the analysis and resolution of complex issues within distributed data platforms, designing scalable, and resilient solutions
  • Effective Communications: Ability to communicate across teams by sharing feedback constructively, listening to others, and creating documentation that makes data systems and processes easy to understand and support
  • Software Development: Experience in leading the design and development of backend systems and data pipelines using Python, Java, and modern frameworks, providing technical directions and ensuring the delivery of reliable, scalable solutions
  • Software Development Life Cycle: Experience leading the delivery of data engineering solutions in an Agile environment by guiding work through the full development lifecycle, translating requirements into technical solutions, and ensuring projects are delivered with quality, reliability, and business value
  • Software Integration Engineering: Capability to lead the design and integration of APIs, data pipelines, streaming platforms, and databases to enable reliable data exchange across enterprise systems and partner platforms
  • Software Product Design/Architecture: Expertise leading the design of scalable, event-driven data systems and architectures, guiding technical decisions and ensuring solutions are reliable, maintainable, and aligned with business needs.
  • Software Product Technical Knowledge: Ability to apply strong knowledge of AWS services and data engineering tools to define requirements, support testing and deployment activities, troubleshoot issues, and ensure data solutions are configured, implemented, and operated effectively across environments
  • Software Product Testing: Ability to define and implement testing strategies, including functional, performance, and data quality testing, to ensure reliable, scalable, and high-performing data solutions across the development lifecycle.

Top Candidates Will Have:

  • Bachelor’s degree in Computer Science, Computer Engineering, or related field
  • 8+ years of experience in data engineering or related disciplines with increasing responsibility
  • Extensive experience on modern, large scale, complex Caterpillar data platforms such as Helios Data Platform
  • Strong foundation developing and deploying Python solutions to a production environment
  • Experience leading teams to build high-throughput, scalable data pipelines
  • Strong hands-on experience with AWS data services (Kinesis, S3, DynamoDB, EventBridge, etc.) at scale
  • Strong in SQL, including data quality and validation practices
  • Experience in deploying software using CI/CD tools such as Azure DevOps, Jira, Jenkins, etc.
  • Experience developing microservices that support real-time data ingestion
  • Experience developing software applications using relational and noSQL databases
  • Ability to ensure data integrity across distributed and streaming systems
  • Experience with monitoring, testing, and automation in large-scale data environments

 

Summary Pay Range:

$128,470.00 - $208,770.00

Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar. 

Benefits:

Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits.

  • Medical, dental, and vision benefits*

  • Paid time off plan (Vacation, Holidays, Volunteer, etc.)*

  • 401(k) savings plans*

  • Health Savings Account (HSA)*

  • Flexible Spending Accounts (FSAs)*

  • Health Lifestyle Programs*

  • Employee Assistance Program*

  • Voluntary Benefits and Employee Discounts*

  • Career Development*

  • Incentive bonus*

  • Disability benefits

  • Life Insurance

  • Parental leave

  • Adoption benefits

  • Tuition Reimbursement

       

* These benefits also apply to part-time employees

 

 

This position requires working onsite five days a week. 

 

Relocation is available for this position.

 

 

Visa sponsorship is available for eligible applicants.

 

 

Posting Dates:

About Caterpillar, Inc

Progress Rail Services Corporation, a Caterpillar company, is one of the largest integrated and diversified suppliers of railroad and transit system products and services worldwide. Since its acquisition by Caterpillar Inc. in 2006, Progress Rail has used the Caterpillar international footprint to gain quick and widespread notice and approval across the globe. Product technology, reliability and durability, along with legendary aftermarket parts and service support, have been their keys to success. Caterpillar is the world's leading manufacturer of construction and mining equipment, diesel and natural gas engines and industrial gas turbines. The technology and world reach of Caterpillar, combined with the service standard and historical reliability of Progress Rail, ensures their products and services make a growing impact on the world rail industry. With their acquisition of Electro-Motive Diesel (EMD), Progress Rail is the world's largest builder of diesel-electric locomotives for all commercial railroad applications including freight, intercity passenger, commuter, switching, industrial and mining. EMD is also a global provider of diesel power engines for marine propulsion, offshore and land based oil well drilling rigs and stationary power generation. Electro-Motive is the only diesel engine manufacturer of its size to have produced more than 70,000 engines. This success stems from product reliability, serviceability, load acceptance capability, low operating costs and ease of maintenance with standardized components.

Caterpillar, Inc. Careers

Join the robust team at Caterpillar, Inc., a global leader in construction and mining equipment, where innovation drives everything we do. As part of our team, you'll engage in work that improves the world and furthers your career with every project you touch.

Work You’ll Do

At Caterpillar, Inc., you will have the opportunity to contribute to a variety of groundbreaking projects that extend across the globe. Our commitment to leadership in the industry is unwavering, and we achieve it through a culture that fosters diversity and leadership development.

Explore Job Opportunities and Growth

Caterpillar offers a wide range of job opportunities and career paths, providing a platform for professional growth and development. Whether you're interested in engineering, sales, marketing, or management, Caterpillar has a position that will utilize your unique skills and talents. We are continuously hiring innovative minds to propel our mission forward and maintain our status as industry leaders.

Internship Programs

Kickstart your career with a Caterpillar internship. Our internships provide invaluable workplace experience and networking opportunities that often lead to full-time employment at our company. As an intern, you will work alongside experienced professionals and gain insights that will benefit you as you advance in your career.

Benefits and Culture

Choosing a career at Caterpillar means more than just having a job. It means you're part of a team that values diversity and is committed to creating an inclusive environment. We offer competitive benefits to ensure the health and well-being of our employees and their families. These benefits include health insurance, retirement plans, and paid time off, alongside unique perks like professional development and diversity training programs.

Innovation and Leadership

Drive innovation and industry leadership as part of the Caterpillar team. We empower our employees to lead from their first day on the job, encouraging continuous improvement and career advancement. At Caterpillar, your ideas will shape the future of our business and the industries we serve.

Join Our Team

Search open positions that match your skills and interests. We are looking for passionate, curious, and solution-driven team players. Ready to start your journey at Caterpillar, Inc.? Explore our career portal for the latest job opportunities.

Stay Connected

Keep up to date with company news, career tips, and industry insights—all from the people who work here. Subscribe to our careers blog and personalize your subscription to receive updates that align with your career interests.

Prepare for Your Interview

Make a great first impression. Prepare for your interview at Caterpillar by reviewing common questions and practicing your responses. Tailor your resume to highlight the experiences and skills that make you the ideal candidate for the position you are applying for. At Caterpillar, Inc., every role offers the chance to make an impact. Come be a part of our team and drive the future of our global industries. Your career at Caterpillar is not just about what you'll accomplish, but how you'll grow and the innovation you'll lead. Join us and build a career that matters.
Learn more about Caterpillar, Inc
Size
107,700 employees
Market Cap
$125.3 billion
Industry
Net Income
$2.9 billion
Founded
1925
5 Year Trend
+5.8%
Revenue
$41.7 billion
NASDAQ

Similar Jobs

More Jobs at Caterpillar, Inc

More Information Technology Jobs

Find similar Lead Data Engineer – Physical AI Platform, Data Engineering jobs: