Role description
Job Description Developer Senior Developer Data Engineer Databricks on GCP
Service Type Development Services Build
Industry MultiNational FMCG
Environment Hyperscalerheavy GCPfirst Data Platform
Primary Skills Must Have
Databricks on GCP Core
Strong handson experience with Databricks GCP
Workspace clusters jobs and notebooks
Delta Lake ACID transactions time travel schema evolution
Medallion architecture Bronze Silver Gold layers
Experience with Databricks SQL SQL Warehouses for analytics workloads
PySpark Spark Core
Advanced development using PySpark Spark SQL
Performance tuning
Partitioning caching broadcast joins
Handling largescale distributed data processing
Debugging and optimizing Spark jobs in production
GCP Data Ecosystem Core
Strong integration with
Google Cloud Storage GCS for data lake storage
BigQuery for downstream analytics
Dataproc Dataflow nice to have for hybrid pipelines
Understanding of GCP networking IAM and service integration
Data Engineering ETL Core
Designing and building endtoend ETLELT pipelines
Handling batch and near realtime data processing
Data ingestion from enterprise systems SAP APIs files databases
Key Responsibilities
Data Pipeline Development
Design and develop scalable data pipelines using Databricks on GCP
Ingest transform and curate large datasets into analyticsready formats
Implement business logic using PySpark aligned with FMCG data domains
Lakehouse Engineering
Build and maintain Delta Lakebased architectures
Ensure data quality consistency and performance across layers
Enable downstream consumption via BigQuery and analytics tools
Performance Cost Optimization
Optimize Spark jobs for performance and cost efficiency
Reduce data scan volumes and improve query execution times
Collaborate with FinOps teams for costaware data processing
DataOps CICD
Implement CICD pipelines for Databricks using
GitHub GitLab Bitbucket
Deployment automation for notebooks and jobs
Version control and promote code across environments DevUATProd
Security Governance
Implement data governance using
Databricks access controls Unity Catalog if enabled
GCP IAM roles and permissions
Ensure compliance with enterprise data policies and regulations
Collaboration Delivery
Work closely with Data Analysts BI teams and business stakeholders
Translate business requirements into scalable data solutions
Participate in agile ceremonies and global delivery model
