Job Summary
The Lead ETL Developer is responsible for leading the design, development, maintenance, and optimization of scalable ETL/ELT pipelines and data solutions within an Enterprise Data Warehouse (EDW) environment. The role involves leveraging AWS, Snowflake, Python/PySpark, and IBM DataStage to deliver reliable, high-quality data solutions. The technical lead will collaborate with engineering, architecture, QA, and business teams, drive software development lifecycle (SDLC) best practices, mentor team members, support production operations, and contribute to data platform modernization and cloud-based transformation initiatives.
Key Responsibilities
• Lead the design, development, and maintenance of ETL/ELT pipelines within the EDW ecosystem.
• Ensure alignment with enterprise data architecture, standards, and best practices.
• Translate requirements and data mapping documents into scalable technical implementations.
• Build and optimize data pipelines using AWS Glue, Snowflake, Python/PySpark, and IBM DataStage.
• Implement reusable, metadata-driven, and performance-optimized data integration patterns.
• Enforce coding standards, version control, and CI/CD practices.
• Define and implement unit testing, regression testing, and data validation strategies.
• Partner with QA teams to support system and integration testing.
• Troubleshoot and resolve data and pipeline issues across development, QA, and production environments.
• Collaborate with Product Owners, Business Systems Analysts (BSAs), architects, and upstream and downstream teams to deliver end-to-end solutions.
• Communicate progress, risks, and blockers clearly to stakeholders and leadership.
• Support production operations, including incident triage and root cause analysis.
• Identify opportunities to improve data pipeline performance, reliability, and scalability.
• Contribute to modernization initiatives, including cloud migration, Snowflake optimization, and automation.
• Stay current with industry trends and emerging data engineering technologies.
• Mentor and coach junior and mid-level developers on technical skills and best practices.
• Provide technical guidance and conduct code reviews to ensure quality and consistency.
• Lead development efforts for key initiatives and serve as a project-level technical lead.
Required Qualifications
• Bachelor's degree or an additional 4 years of related work experience in lieu of the degree.
• 5+ years of ETL/Data Engineering experience in enterprise data warehouse environments.
• 5+ years of experience with SQL and database development.
• 5+ years of hands-on experience with Python/PySpark and ETL tools.
• 3+ years of experience with cloud-based data platforms.
Preferred Qualifications
• Strong understanding of data warehousing concepts, data modeling, and SDLC methodologies.
• Proven experience leading development teams or serving as a technical lead on large-scale data initiatives.
• Experience with AWS services, including Glue, S3, and EC2, and Snowflake architecture optimization.
• Knowledge of data pipeline observability, monitoring, and automated testing frameworks.
• Experience defining future-state data architectures or modernization roadmaps.
• Familiarity with CI/CD pipelines, DevOps practices, and version control systems.
• Understanding of data governance, security, and regulatory considerations in financial services.
• Prior experience in the banking or financial services industry.
Certifications
• AWS Solutions Architect certification.
• AWS DevOps Engineer certification.
Primary Skills: AWS, Snowflake, Python/PySpark, ETL/ELT, IBM DataStage, SQL, AWS Glue, Enterprise Data Warehousing.
