GCP Data Engineer

TechWish

• $120K — $145K *
New York, NY 10025In-Person
Information Technology
5 - 7 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • Strong SQL and Python skills
  • 6-10 years of experience required
  • Basic-to-intermediate experience with GCP services (BigQuery, GCS, Composer, Cloud Functions)
  • Solid understanding of ETL/ELT data flows and pipeline design
  • Experience with APIs, file-based ingestion, and orchestration tools
  • Hands-on exposure to migration tools like Storage Transfer Service, gsutil, and bq CLI
  • Working knowledge of Spark (Dataproc) is a plus

Responsibilities

  • Write optimized SQL queries for data extraction, cleaning, transformation, and analysis
  • Automate data workflows and reporting processes to enhance efficiency
  • Document methodologies and findings for clarity and knowledge sharing
  • Execute data migration tasks under senior engineers' guidance
  • Build ingestion scripts and end-to-end data pipelines
  • Support migration of data from Teradata to BigQuery
  • Implement monitoring and logging for data pipelines to ensure reliability
  • Collaborate with cross-functional teams to gather requirements

Benefits

  • Flexible work environment
  • Opportunities for professional development
  • Access to advanced cloud technologies
  • Collaboration with a diverse team
  • Potential for involvement in innovative projects
Full Job Description
Job Description
Key Responsibilities:
  • Write efficient, optimized SQL queries to extract, clean, transform, and analyze large and complex datasets from multiple sources
  • Automate repetitive data workflows and reporting processes to improve efficiency and reduce manual effort
  • Document methodologies, code, and findings clearly for reproducibility and knowledge sharing
  • Execute migration tasks under the guidance of senior engineers/architects
  • Build ingestion scripts, transformation logic, and end-to-end data pipelines
  • Support migration of data from Teradata to BigQuery
  • Implement pipeline monitoring, logging, and alerting to ensure reliability
  • Support testing, validation, and reconciliation of migrated datasets to ensure data integrity
  • Maintain and update technical documentation and operational runbooks
  • Collaborate with cross-functional teams including data analysts, architects, and business stakeholders to understand requirements
Required Skills & Experience:
  • Strong SQL and Python skills
  • 6 -10 years of exp needed
  • Basic-to-intermediate experience with GCP services (BigQuery, GCS, Composer, Cloud Functions)
  • Solid understanding of ETL/ELT data flows and pipeline design
  • Experience working with APIs, file-based ingestion, and orchestration tools
  • Ability to test, validate, and debug data pipelines effectively
  • Hands-on exposure to migration tools such as Storage Transfer Service, gsutil, and bq CLI
  • Working knowledge of Spark (Dataproc) is a plus
  • Understanding of Teradata-to-GCP mapping concepts (e.g., Teradata 12 GCS, ADF 12 Composer)
  • Familiarity with CI/CD concepts and practices
  • Exposure to version control systems such as Git for code management and collaboration
  • Understanding of data security, access controls, and governance best practices in cloud environments
  • Familiarity with data quality frameworks and reconciliation techniques
  • Good communication and problem-solving skills, with the ability to work independently and in a team setting
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at TechWish

More Information Technology Jobs

Find similar GCP Data Engineer jobs:

NationwideNew York, NY