OverviewJob DescriptionElwyn seeks a
Data Engineer to play a key role in designing, building, and maintaining the organization's cloud-based data platform on Microsoft Azure. This role is responsible for developing reliable, scalable data pipelines, supporting large-scale data processing, and ensuring analytics teams have access to high-quality, well-structured data.
Working closely with the Data Warehouse Senior Manager, Azure Data Architect, analytics teams, and platform partners, the Senior Data Engineer translates architectural standards into robust, production-ready data solutions. This role also supports the organization's transition toward modern lakehouse architectures, including the migration and modernization of existing workloads from Azure Synapse into Microsoft Fabric and OneLake.
The ideal candidate is hands-on, technically strong, and comfortable working across data ingestion, transformation, orchestration, and operational support while reinforcing strong engineering standards across the platform.
ESSENTIAL DUTIES AND RESPONSIBILITIES- Data Engineering & Pipeline Development
- Design, develop, and maintain scalable data pipelines that ingest, transform, and prepare data for analytics and reporting across the Azure data platform.
- Build and support batch and near-real-time data workflows using Microsoft Fabric, Azure Synapse, Azure Data Factory, and related orchestration tools.
- Develop, optimize, and maintain Spark-based processing jobs that handle large and complex datasets.
- Platform Modernization & Lakehouse Enablement
- Support the migration and modernization of existing Azure Synapse pipelines and Spark workloads into the Microsoft Fabric ecosystem and OneLake architecture.
- Implement data transformation logic using modern lakehouse patterns, including Delta Lake storage formats.
- Contribute to the evolution of engineering standards that support Fabric-based lakehouse platforms.
- Data Integration & Analytics Support
- Extract, stage, and integrate data from SQL Server and other operational systems into Fabric- or Synapse-based analytics environments.
- Partner with analytics and reporting teams to ensure data structures are intuitive, performant, and easily consumable for tools such as Power BI.
- Support enterprise reporting and analytics use cases by delivering reliable, well-modeled datasets.
- Reliability, Performance & Operations
- Ensure data pipelines are stable, well-monitored, and optimized for performance and cost efficiency.
- Troubleshoot and resolve pipeline failures, data quality issues, and performance bottlenecks in production environments.
- Assist with managing cloud resources that support the data platform, including awareness of Azure subscriptions, resource groups, and usage patterns.
- Governance, Documentation & Collaboration
- Support metadata management, data discovery, and lineage tracking through integration with Microsoft Purview.
- Follow architectural standards and data engineering best practices established by the Azure Data Architect.
- Document data pipelines, transformation logic, and engineering processes to ensure maintainability and knowledge sharing.
- Collaborate effectively with data engineers, architects, analytics teams, and engineering leadership.
QUALIFICATIONS- Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related technical discipline
- Azure-related certifications or advanced training are beneficial but not required
- 5+ years of experience building and maintaining enterprise data pipelines or data engineering solutions
- Hands-on experience with Microsoft Azure data services, including Azure Synapse Analytics and Azure Data Factory
- Experience working with Spark or distributed data processing frameworks
- Demonstrated experience integrating data from relational systems (e.g., SQL Server) into cloud-based analytics platforms
- Experience supporting large-scale analytics environments used by reporting or data science teams
- Familiarity with lakehouse architectures and modern cloud data platforms such as Microsoft Fabric or Delta Lake
- Experience working in collaborative data teams alongside architects, analysts, and engineering managers
