5-7 years of experience in data engineering or related field
Strong expertise in Snowflake architecture and SQL development
Hands-on experience with AWS services like S3, IAM, and CloudWatch
Proficient in Python for ETL/ELT frameworks and automation
Experience with Control-M for job scheduling and monitoring
Familiarity with IBM DataStage for complex transformations
Knowledge of CI/CD practices and version control using Git
Responsibilities
Design, develop, and maintain batch and near real-time data pipelines using Snowflake and AWS.
Build and optimize Snowflake data models including dimensional and data vault structures.
Develop ETL/ELT workflows with Python and IBM DataStage, focusing on modernization.
Implement job scheduling and monitoring with Control-M, ensuring operational efficiency.
Ensure data quality and governance standards are met across all data pipelines.
Perform performance tuning and cost optimization in Snowflake and AWS environments.
Collaborate with cross-functional teams to enable data products and curated datasets.
Benefits
Flexible work environment with remote options
Opportunities for professional development and training
Collaborative team culture with Agile methodologies
Access to cutting-edge technology and tools
Health and wellness programs
Full Job Description
Job Summary
We are looking for a Senior Data Engineer to design, build, and optimize scalable data pipelines and data platforms supporting analytics, reporting, and AIML use cases. The ideal candidate has strong hand son experience with Snowflake on AWS, Python based ETLELT development, and enterprise scheduling orchestration tools like Control M, along with legacy enterprise ETL experience in IBM DataStage. You will collaborate across engineering, analytics, and business teams in an Agile delivery model.
Responsibilities
Design, develop, and maintain batch and near real-time data pipelines using Snowflake, AWS, and Python.
Build and optimize Snowflake data models (dimensional, data vault, curated marts).
Develop and maintain ETL/ELT workflows using Python and IBM DataStage; migrate and modernize workloads.
Implement job scheduling, monitoring, and operational support with Control-M (alerting, retries, SLAs, dependencies).
Ensure data quality, governance, lineage, and documentation standards across pipelines.
Perform performance tuning and cost optimization in Snowflake and AWS (query optimization, clustering, warehouse sizing).
Collaborate with Data Science, AI, BI, Product, and Platform teams to enable data products and curated datasets.
Participate in Agile ceremonies, contribute to estimation, sprint planning, and execution.
Troubleshoot production issues, perform root cause analysis, and drive preventative improvements.
Required Technical Skills
Snowflake: Strong expertise in architecture, SQL development, performance tuning, security roles, and data loading/unloading.
AWS: Hands-on experience with S3, IAM, CloudWatch, Glue, Lambda, EC2, Step Functions, EMR, or Kinesis.
Python: Strong programming for ETL/ELT frameworks, API ingestion, automation, unit testing, and logging.
Control-M: Enterprise job scheduling, dependencies, calendars, SLAs, monitoring, and incident handling.
IBM DataStage: Building and maintaining jobs, complex transformations, and production support.
SQL: Advanced skills for transformations, optimization, and validation across large datasets.
CI/CD & Git: Experience with version control and pipeline automation.
Operational Excellence: Monitoring, alerting, and production support in business-critical environments.
Good to Have
Exposure to AI/ML pipelines or feature datasets.
Familiarity with data governance tools (catalogue, lineage, data quality frameworks).
Experience with streaming/event-driven architectures.
