5+ years of experience in ETL development and data operations
Strong proficiency in Microsoft SQL Server, T-SQL, and SSIS
Experience with cloud technologies, particularly Azure Data Factory and Databricks
Familiarity with real-time data processing and event-driven architectures
Proven leadership skills in guiding technical teams and mentoring engineers
Knowledge of data governance and quality frameworks
Experience with modern data architectures, including Medallion Architecture
Responsibilities
Monitor and troubleshoot daily ETL loads and pipeline issues
Refactor outdated ETL processes and ensure SLA adherence
Collaborate with AMS team to enhance operational runbooks and incident response workflows
Lead the development of scalable data ingestion pipelines using modern technologies
Design and implement real-time data pipelines for operational insights
Establish data quality and governance frameworks for enterprise data
Prepare datasets for AI/ML applications, if applicable
Benefits
Flexible work hours and remote work options
Professional development opportunities and training
Health, dental, and vision insurance
Retirement savings plan with company match
Generous paid time off and holiday schedule
Full Job Description
ETL Team Lead
Major Accountabilities:
Production Support, Operations & Reliability
The ETL Team Lead owns end-to-end operational support for Canal's existing data stack:
Monitor daily ETL loads across SQL jobs, DHIC (GW DataHub and InfoCenter) and legacy SSIS packages.
Work with AMS team where necessary to troubleshoot pipeline failures, performance issues, schema mismatches, permissions issues, and cloud resource failures.
Work with AMS team where necessary to perform root-cause analysis and to implement permanent fixes.
Ensure SLA adherence and on-time delivery of critical reporting datasets for scheduled ETL jobs .
Provide direction for both AMS and ETL Developers for Legacy & Current ETL maintenance.
Refactor or retire outdated or redundant ETL processes.
Maintain and improve existing pipelines that utilize the following technologies:
Microsoft SQL Server Database programming
T-SQL Scripting
SQL Server Integration Services
Microsoft Powershell
Guidewire DataHub and InfoCenter
Oracle Database programming
Oracle PL-SQL Scripting
SAP BODS (SAP BusinessObjects Data Services)
PostgreSQL Scripting
Operational Excellence
Work with AMS team to assist with the creation and/or enhancement of operational runbooks, SOPs, monitoring dashboards, and incident response workflows.
Partner with other IT operational segments, business SMEs, and AMS team to minimize downtime and to ensure that business SLAs are met.
Improve upon existing, as well as implement new proactive for daily processing.
Business Continuity
Work with AMS team to ensure development support coverage for critical data pipelines (rotation-based).
Support month-end and quarter-end financial reporting cycles.
Coordinate production releases and validate deployments.
The ETL Tech Lead will become the steady-state technical owner of the entire data operations layer during the Canal modernization journey.
Technical Leadership & Collaboration
Serve as technical lead guiding onshore/offshore developers.
Review code, enforce best practices, and mentor junior engineers.
Partner with Scrum Masters, Project Mangers, Enterprise Architecture, QA Automation, Change Management, and AMS support teams.
Data Ingestion, ETL/ELT Development & Optimization
Develop reusable ingestion patterns for Guidewire DataHub and InfoCenter, HubSpot, Telematics, and other facets of the business.
Work with Canal Architects to modernize existing ETL workloads using Delta Lake, Medallion Architecture, and Fabric Lakehouse.
Build scalable data ingestion pipelines using potential upcoming technology (Azure Data Factory, MS Fabric , Databricks, Synapse Pipelines, etc.).
Work to bring in Internal and External integrations data into the platform.
Real-Time, Streaming & Event-Driven Engineering
Proven experience designing and implementing real-time data pipelines using Event Hub, Fabric Real-Time Analytics, Databricks Structured Streaming, and KQL-based event processing.
Develop and enable real-time operational insights and automation capabilities-including Telematics alerting, FNOL automation, and fraud/VNOS/VNOP detection-through event-driven architectures and streaming analytics.
Modern Azure Data Stack Leadership
Lead the strategy, design, and engineering of Canal's modern Azure data ecosystem using next-generation tools and Medallion Architecture, including:
Implementing Medallion Architecture (Bronze/Silver/Gold) across Fabric Lakehouse, Warehouse, Eventhouse, and KQL Database.
Leveraging Delta tables with schema enforcement, ACID compliance, and versioning.
Data Modeling, Curation & Governance
Develop curated, analytics-ready datasets to support Power BI, operational reporting, and advanced analytics use cases.
Assist Canal architect with implementation of Data Governance tools.
Establish robust data quality, validation, alerting, and observability frameworks to maintain accuracy, consistency, and trust in enterprise data.
AI/ML Data Enablement (Optional)
Prepare ML-ready datasets for pricing, risk, fraud detection, underwriting, claims leakage, and predictive insights.
