Previous experience as an Azure Data Engineer or a similar role
Experience in building and optimizing big data pipelines and architectures
Strong analytical skills for working with unstructured datasets
Proficiency in designing and implementing high-quality code
Ability to meet tight deadlines
Experience in testing data from source to presentation layer
Capable of supporting and troubleshooting data pipelines
Excellent communication skills for team collaboration
Responsibilities
Ensure data is cleansed, mapped, transformed, and optimized for storage and use
Develop and maintain innovative solutions using Microsoft Azure
Design solutions leveraging Azure services and related tools
Automate tasks and deploy production-level code with proper testing and versioning
Load transformed data into various storage and reporting structures
Build data pipelines to integrate multiple datasets
Extract, troubleshoot, and maintain the data warehouse
Full Job Description
Responsibilities
Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements
Develop and maintain innovative Azure solutions
Solution design using Microsoft Azure services and other tools
The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications
Build data pipelines to collectively bring together data
Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse
Tech Stack
Python
SQL and NoSQL databases
Scala
Spark-SQL
Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture, ARM Templates
Qualifications
Previous experience as an Azure Data Engineer or similar role
Experience building and optimising 'big data' data pipelines, architectures, and data sets
Strong analytic skills related to working with unstructured datasets
The ability to design and implement well written code
Ability to work to tight deadlines
Ability to test the data from source to the presentation layer
Ability to support/troubleshoot data pipelines
Confident and concise communication skills, with the ability to drive alignment, collaboration, and efficiency within teams