About the roleAs a Senior Data Engineer, you will own the data platform that turns raw fleet and simulation logs into training-ready, searchable, and versioned datasets. You will work closely with perception, prediction, planning, simulation, and safety teams to make data fast to find, easy to trust, and cheap to process at scale.
This role sits at the center of Gatik's data flywheel: logs from our trucks drive scenario mining, model training, closed-loop testing, and the safety evidence we need to expand driverless operations.
What you'll do- Design, build, and operate production-grade pipelines that ingest, decode, transform, and store multi-sensor logs from Gatik's autonomous truck fleet and simulation environments.
- Build internal tools and services for data labeling, curation, indexing, and cataloging, so engineers can find the right drives, events, and scenarios in minutes.
- Partner with ML researchers, autonomy engineers, and data scientists to design schemas and APIs that power model training, evaluation, and debugging.
- Develop and maintain feature stores, metadata systems, and dataset versioning for both structured data (events, metrics, vehicle signals) and unstructured data (camera, lidar, radar).
- Integrate synthetic and simulated data with real-world logs to support hybrid training, scenario replay, and simulation workflows.
- Build scenario-mining and event-tagging pipelines that surface rare and safety-relevant situations on our routes, such as unprotected turns, dock approaches, and adverse weather.
- Optimize pipelines for cost, latency, and traceability, with full lineage from raw log to trained model and evaluation result.
- Partner with simulation, cloud platform, and safety teams to automate closed-loop testing, regression analysis, and fleet performance analytics.
- Set data quality and observability standards, and mentor other engineers through design reviews and code reviews.
What you'll bring- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field, or equivalent practical experience.
- 8+ years building data-intensive software systems, ideally in autonomous driving, robotics, or large-scale ML environments.
- Strong proficiency in Python and SQL, and working familiarity with C++.
- Hands-on experience designing ETL/ELT pipelines with modern frameworks such as Apache Spark, Flyte, Union, Ray, or Airflow.
- Strong knowledge of cloud-native architectures on AWS (S3, EC2, EKS, Glue, Athena) or Google Cloud equivalents.
- Familiarity with sensor data (camera, lidar, radar, GPS/IMU) and serialization formats such as Protobuf, ROS/ROS 2 bags, and MCAP.
- Deep understanding of data quality, observability, and lineage in high-volume systems.
- A track record of building reliable, performant infrastructure that supports both ad-hoc exploration and repeatable production workflows.
Nice to have- Experience in AD/ADAS, autonomous trucking, or robotics, especially with perception, prediction, or planning datasets.
- Familiarity with ML orchestration platforms such as Kubeflow, SageMaker, or Vertex AI.
- Experience with temporal and spatial data, including geospatial indexing (H3, S2, PostGIS), HD maps, and multi-sensor time-series alignment.
- Exposure to synthetic data generation, simulation logging, or scenario replay pipelines.
- Experience with lakehouse and table formats such as Delta Lake, Apache Iceberg, or Parquet at petabyte scale.
- Familiarity with safety-critical development practices and data traceability for safety cases or regulatory requirements.
- Strong software engineering fundamentals: CI/CD, testing, code review, and service deployment best practices.
- Experience working with cross-functional, distributed teams across research and production organizations.
Salary Range: $180,000 - $260,000
