We are looking for a skilled Data Engineer with proven experience in building and maintaining scalable data pipelines and cloud-based data platforms on Google Cloud Platform (GCP). The ideal candidate should have hands-on expertise in Python, SQL, PySpark, BigQuery, Airflow, and modern ETL/ELT frameworks.
The role involves working closely with analytics, business, and engineering teams to enable reliable and scalable data solutions.
Responsibilities- Design, develop, and maintain scalable data pipelines using Python and PySpark
- Build and optimize ETL/ELT workflows using Apache Airflow
- Develop data transformation models and workflows using dbt
- Work with BigQuery for data warehousing, querying, and optimization
- Write efficient and optimized SQL queries for large-scale data processing
- Ensure data quality, integrity, and reliability across pipelines
- Collaborate with Data Analysts, Data Scientists, and business stakeholders for data requirements
- Monitor and troubleshoot production data workflows and resolve issues proactively
- Implement best practices for logging, monitoring, testing, and CI/CD
- Optimize cloud resource usage and cost efficiency on GCP
- Participate in code reviews and engineering best practices
Required Skills- Strong hands-on experience with Python and SQL
- Experience with PySpark and distributed data processing
- Good knowledge of BigQuery and GCP services
- Experience with Apache Airflow for orchestration
- Hands-on experience with dbt for data transformation and modeling
- Understanding of ETL/ELT frameworks and data warehousing concepts
Familiarity with GCP services such as:- BigQuery
- Cloud Storage
- Dataproc
- Composer
- Pub/Sub
- Experience with Git and CI/CD practices
- Strong analytical and problem-solving skills
Good to Have- Exposure to streaming pipelines using Kafka or Pub/Sub
- Understanding of data governance and security concepts
- Experience with Terraform or Infrastructure as Code
- Knowledge of containerization (Docker/Kubernetes)
Preferred Qualification- Bachelor's degree in Computer Science, Engineering, or related field
- GCP certification is a plus
Required Skills
Python BigQuery SQL data engineering gcp
