Qualifications

• 2-4 years of experience in Data Engineering or related roles required
• Proficient in Python and SQL for data manipulation and analysis
• Hands-on experience with PySpark for distributed data processing
• Familiar with BigQuery and GCP services for data warehousing
• Experience with Apache Airflow for orchestration and workflow management
• Practical knowledge of dbt for data transformation and modeling
• Familiar with Git and CI/CD practices for version control and deployment

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark
• Build and optimize ETL/ELT workflows utilizing Apache Airflow
• Develop data transformation models and workflows with dbt
• Work with BigQuery for data warehousing, querying, and performance optimization
• Write efficient SQL queries for processing large-scale data
• Ensure data quality, integrity, and reliability across all pipelines
• Collaborate with analysts, scientists, and stakeholders on data requirements
• Monitor and troubleshoot production data workflows proactively
• Implement logging, monitoring, testing, and CI/CD best practices
• Optimize cloud resource usage and cost efficiency on GCP

Benefits

• Collaborative work environment across teams including Data Analysts and Scientists
• Opportunity to work with cutting-edge technologies like PySpark and GCP
• Emphasis on best practices in data engineering and CI/CD
• Exposure to diverse projects in data transformation and modeling
• Encouragement for participation in code reviews and continuous learning opportunities