Full Job Description
Core Data Engineering & Architecture: 5+ years of experience designing, building, and optimizing robust, scalable data pipelines (ETL/ELT).Programming Languages: 4+ years of strong, production-grade coding experience in Python and advanced proficiency writing complex, optimized SQL queries.
Big Data & Lakehouse Platforms: 3+ years of experience working with distributed computing frameworks such as Apache Spark (PySpark) and modern lakehouse platforms such as Databricks.
Data Warehousing & Dimensional Modeling: Advanced proficiency in cloud data platforms-specifically Snowflake (or Google BigQuery / AWS Redshift)-including schema design, partitioning strategies, and building optimized dimensional models.
Problem-Solving & Data Quality: Proven track record of implementing rigorous data validation, data cleansing, automated testing, and error-handling frameworks across pipeline stages.
Preferred Skills & Attributes
Cloud Infrastructure: 2+ years of hands-on experience with cloud platforms (AWS, GCP, or Azure), particularly with managed data services and serverless computing.
Orchestration Tools: Experience managing and monitoring complex workflows using tools like Apache Airflow, Prefect, or dbt.
CI/CD & DevOps: Experience deploying pipelines via CI/CD frameworks (GitHub Actions, GitLab CI, or Jenkins) and infrastructure-as-code (Terraform).
Streaming Architecture: Familiarity with real-time data streaming technologies such as Apache Kafka, AWS Kinesis, or Spark Structured Streaming is a strong plus.
Primary Responsibilities
Pipeline Development: Design, build, and maintain automated, scalable ETL/ELT pipelines ingesting high-volume data using Python, PySpark, and Databricks.
Data Modeling & Architecture: Implement Medallion Architecture standards and construct clean relational and dimensional data models in Snowflake to support self-service analytics and BI workloads.
Performance Tuning: Monitor, troubleshoot, and optimize data workflows, Spark jobs, and Snowflake queries for speed, cost-efficiency, and reliability.
Collaboration & Integration: Partner with analysts, data scientists & business teams to translate complex needs into structured, high-quality datasets..
Preferred Experience & Domain
Enterprise tech background; skilled in data value chains, cross-functional Agile delivery.
