We are seeking a Senior Data Engineer with hands-on Microsoft Fabric experience to design, build, migrate, and support enterprise data platforms. This role develops scalable ETL and ELT pipelines, implements Medallion Architecture in OneLake, Lakehouse, and Fabric Warehouse, and delivers reliable data for analytics and Power BI. The engineer partners with architects, application teams, analysts, BI developers, infrastructure teams, and business stakeholders to implement secure, governed, and high-performing data solutions.
Core Skills
• Microsoft Fabric architecture and engineering: Microsoft Fabric, Data Factory, Data Pipelines, Dataflows Gen2, OneLake, Lakehouse, Fabric Warehouse, Notebooks, and Medallion Architecture.
• Data pipeline development and programming: ETL/ELT, SQL, T-SQL, Python, PySpark, Spark, batch processing, incremental processing, change data capture (CDC), upsert/merge strategies, and metadata-driven frameworks.
• Data migration, warehousing, and modeling: legacy platform migration, source-to-target mapping, data profiling, cleansing, reconciliation, validation, dimensional modeling, star schemas, fact tables, dimension tables, and slowly changing dimensions.
Key Responsibilities
• Design, develop, test, deploy, and maintain scalable end-to-end data pipelines for batch, incremental, and near-real-time processing.
• Build data ingestion and transformation solutions using Microsoft Fabric Data Factory, Data Pipelines, Dataflows Gen2, Notebooks, SQL, Python, PySpark, Spark, and T-SQL.
• Develop reusable, metadata-driven ETL/ELT frameworks that support relational databases, APIs, files, SaaS applications, cloud platforms, and structured or semi-structured data.
• Implement CDC, incremental loads, upsert/merge patterns, historical processing, error handling, monitoring, and data-quality controls.
• Lead legacy-to-modern data migrations, including data profiling, source-to-target mapping, cleansing, transformation, reconciliation, validation, and cutover support.
• Design and implement Medallion Architecture using OneLake, Fabric Lakehouse, and Fabric Warehouse.
• Create enterprise data warehouse models using dimensional modeling, star schemas, fact tables, dimension tables, and slowly changing dimensions.
• Troubleshoot pipeline, data, integration, and performance issues and optimize Microsoft Fabric workloads and capacity utilization.
• Document data flows, mappings, standards, lineage, and operating procedures.
• Collaborate with technical and business stakeholders to deliver secure, governed, reliable, and scalable data solutions.
Required Qualifications
• 5+ years of professional experience in data engineering, data integration, data warehousing, business intelligence, or a related field.
• Hands-on experience implementing Microsoft Fabric in a production or enterprise environment.
• Experience with Microsoft Fabric Data Factory, Data Pipelines, Dataflows Gen2, OneLake, Lakehouse, Fabric Warehouse, and Notebooks.
• Experience designing and implementing Medallion Architecture and enterprise ETL/ELT solutions.
• Strong SQL and T-SQL development skills.
• Strong Python and/or PySpark experience for data engineering and transformation.
• Experience with Spark-based batch and incremental data processing.
• Experience with legacy-to-modern data migration, data profiling, source-to-target mapping, transformation, reconciliation, and validation.
• Strong knowledge of data warehousing and dimensional modeling, including star schemas, fact tables, dimension tables, and slowly changing dimensions.
• Experience with data quality, monitoring, error handling, troubleshooting, and performance optimization.
• Strong analytical, problem-solving, written communication, verbal communication, and documentation skills.
Preferred Qualifications
• Microsoft Certified: Fabric Data Engineer Associate or another relevant Microsoft certification.
• Experience with Power BI, semantic models, Direct Lake, DAX, and enterprise reporting.
• Experience with Delta Lake, Snowflake, SQL Server, Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, or Azure Databricks.
• Experience integrating APIs and REST services and working with JSON, XML, or other semi-structured data.
• Experience with streaming or near-real-time data integration.
• Experience with enterprise data governance, metadata management, data lineage, cataloging, and automated data-quality testing.
• Experience supporting large-scale modernization or data migration programs.
• Experience working in Agile or Scrum environments.
• Experience mentoring data engineers and establishing engineering standards.
