Job Title: Azure Data Engineer (Azure Data Factory & Azure Data Lake)Position Overview Build and maintain scalable, cloud-based data integration and storage solutions using Microsoft Azure services. This role focuses on developing reliable data pipelines, managing enterprise-scale data lakes, implementing ETL/ELT workflows, and supporting cloud-based analytics and data warehousing solutions.
The ideal candidate brings expertise in Azure Data Factory, Azure Data Lake Storage, data transformation technologies, and cloud data engineering practices.
What You'll Do / Key Responsibilities - Design, develop, and maintain data pipelines using Azure Data Factory.
- Build and manage enterprise-scale data lakes using Azure Data Lake Storage.
- Develop ETL/ELT workflows to ingest data from multiple sources, including databases, APIs, flat files, and cloud applications.
- Implement data transformation processes using Mapping Data Flows, SQL, and Spark technologies.
- Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
- Configure incremental data loads, Change Data Capture (CDC), and data validation processes.
- Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
- Design data models and support data warehousing solutions.
- Implement security, governance, and access controls for cloud data platforms.
- Create technical documentation and maintain data architecture standards.
- Support the migration of on-premises data solutions to Azure cloud environments.
Required Qualifications - Expertise in Azure Data Factory for data pipeline development and maintenance.
- Experience with Azure Data Lake Storage and enterprise-scale data lakes.
- Experience developing ETL/ELT workflows for data ingestion from multiple sources.
- Knowledge of Mapping Data Flows, SQL, and Spark technologies for data transformation.
- Experience with incremental data loads, Change Data Capture (CDC), and data validation.
- Ability to troubleshoot and optimize data pipelines for performance and reliability.
- Knowledge of data modeling, data warehousing, and cloud data platform security and governance.
- Experience supporting on-premises data migration to Azure cloud environments.
#AzureDataEngineer #AzureDataFactory #AzureDataLake #ADLS #ADF #ETL #ELT #SQL #ApacheSpark #DataEngineering #CloudData #DataWarehousing #ChangeDataCapture #MicrosoftAzure #CloudMigration #LI-NS1
