Senior Manager Data Engineering

Compunnel

$120K — $150K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Experience with AWS Glue for data integration and transformation tasks.
  • Proficient in PySpark for developing scalable data processing applications.
  • Knowledge of Amazon Redshift for managing data warehousing solutions.
  • Strong SQL skills for querying and manipulating data effectively.
  • Hands-on experience in data pipeline development and cloud data engineering practices.
  • Ability to perform data analysis and root cause investigation for quality assurance.
  • Demonstrated troubleshooting skills and providing operational support in cloud environments.

Responsibilities

  • Manage and maintain Amazon Redshift consumer cluster infrastructure for optimal performance.
  • Design and support data pipelines using PySpark and AWS Glue for data processing.
  • Monitor and optimize ETL/ELT workflows to ensure timely data delivery.
  • Conduct thorough data analysis to resolve quality and operational issues.
  • Collaborate with cross-functional teams for data integration and modernization projects.
  • Implement best practices in automation and performance tuning for data processes.
  • Support production releases and continuously improve AWS-based data platforms.

Benefits

  • Opportunity to work with cutting-edge cloud technologies.
  • Collaborative work environment with diverse teams.
  • Focus on innovation and continuous improvement.
  • Access to professional development resources and training.
  • Impactful role in enhancing data reporting capabilities.
Full Job Description
plain
JOB SUMMARY
The contractor will be responsible for supporting and maintaining the data pipeline platform that powers performance reporting data staging capabilities.

Key Responsibilities
• Manage and maintain Amazon Redshift consumer cluster infrastructure, ensuring platform stability, performance, scalability, and operational reliability.
• Design, develop, enhance, and support data pipelines built using PySpark and AWS Glue to ingest, transform, and deliver data across the platform.
• Monitor, troubleshoot, and optimize ETL/ELT workflows, data processing jobs, and cloud infrastructure to meet service level expectations.
• Perform data analysis, root cause investigation, and data validation activities to identify and resolve data quality, performance, and operational issues.
• Collaborate with product, engineering, and business teams to support data integration, reporting, and modernization initiatives.
• Implement engineering best practices related to automation, monitoring, performance tuning, and cloud-native data processing.
• Support production releases, incident resolution, and continuous improvement of the AWS-based data platform.

Required Qualifications
• AWS Glue
• PySpark
• Amazon Redshift
• SQL
• Data pipeline development
• Cloud data engineering
• Data analysis
• Troubleshooting
• Operational support

Similar Jobs

More Jobs at Compunnel

More Information Technology Jobs

Find similar Senior Manager Data Engineering jobs: