Data Engineer - GCP

Euclid Innovations

• $90K — $130K *
Charlotte, NC 28269In-Person
Information Technology
Less than 5 years of experience
1 month ago
Job Overview by Ladders

Qualifications

  • 5-7 years of hands-on experience with Apache Spark/PySpark for large-scale data processing.
  • Proficiency in Python, particularly for building ETL pipelines.
  • Demonstrated experience designing and developing data engineering workflows.
  • Solid understanding of ETL processes, data ingestion, transformation, and movement.
  • Experience managing large datasets in both batch and streaming contexts.
  • Familiarity with GCP technologies including BigQuery, Dataflow, and Dataproc.
  • Background in data migration and integration projects.

Responsibilities

  • Design and build scalable ETL/data pipelines using Spark and Python.
  • Develop processes to ingest, transform, and manage large datasets.
  • Implement data routing logic to direct data to both cloud and on-prem platforms.
  • Ensure data quality, validation, and reconciliation across different systems.
  • Collaborate with data science and platform teams to support predictive modeling.
  • Optimize performance and scalability for high-volume data processing.
Full Job Description
Key Responsibilities
  • Design and build scalable ETL/data pipelines using Spark and Python
  • Develop data workflows to ingest, transform, and move large datasets
  • Implement data routing logic to direct data to:
    • GCP (BigQuery, Dataflow, Dataproc)
    • On-prem platforms (DPC)
  • Ensure data quality, validation, and reconciliation across systems
  • Collaborate with data science and platform teams to support predictive model pipelines
  • Optimize performance and scalability for high-volume data processing

Required Skills
  • Strong hands-on experience with Apache Spark / PySpark for large-scale data processing
  • Proficiency in Python for data engineering (ETL pipelines)
  • Experience designing and developing data pipelines / data engineering workflows
  • Solid background in ETL, data ingestion, transformation, and data movement
  • Experience working with big data technologies and handling large datasets (batch/streaming)
  • Experience with cloud platforms - GCP (Google Cloud Platform)
    • BigQuery, Dataflow, Dataproc, GCS (Google Cloud Storage)
  • Experience with data migration / data integration projects
  • Understanding of data pipeline architecture and distributed systems
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Euclid Innovations

More Information Technology Jobs

Find similar Data Engineer - GCP jobs:

NationwideCharlotte, NC