Role Overview:This role involves designing, developing, and maintaining scalable data pipelines utilizing Azure Data Factory (ADF), Databricks, and Snowflake to support enterprise data and analytics initiatives. The engineer will implement and manage real-time and batch data ingestion solutions using Qlik Replicate, ensuring efficient data movement across platforms. Key responsibilities include developing data transformation and ETL/ELT processes with PySpark, SQL, and Databricks, collaborating with stakeholders on data requirements and model design, and optimizing data workflows while adhering to best practices in data governance, security, and cloud architecture.
Key Responsibilities:- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF), Databricks, and Snowflake to support enterprise data and analytics initiatives.
- Implement and manage real-time and batch data ingestion solutions using Qlik Replicate, ensuring reliable and efficient data movement across platforms.
- Develop data transformation and ETL/ELT processes using PySpark, SQL, and Databricks, optimizing performance and data quality.
- Collaborate with business and technical stakeholders to understand data requirements, design data models, and deliver robust data solutions.
- Monitor, troubleshoot, and optimize data workflows while adhering to best practices in data governance, security, and cloud-based architecture.
Required Skills:- Strong hands-on experience in Qlik Replicate development.
- Good experience in Azure Cloud services.
- Good knowledge in Snowflake.
- Strong expertise to create ETL pipelines using Azure Data Factory and Azure Databricks.
- Expertise in Debugging capability of Qlik Replicate tasks.
- Strong in performance optimization.
- Sound knowledge in Azure Databricks.
- Good knowledge in SAP.
- Should have Supply Chain Domain Knowledge.
- Should have Health care domain knowledge.
Qualifications:Preferred Skills:- Digital: Google Data Engineering.
