Job DescriptionThe role combines two functions. As a Data Engineer, it automates the data pipeline for analytic products and advanced analyses. As a Data Science Specialist, it helps identify and prioritize use cases and integrate AI techniques (machine learning, NLP) into PHO's work, increasing capacity for modelling, forecasting, and scenario analysis. The role supports both foundational capability/infrastructure work and HealthMap data ingestion and aligns with and advances the Public Health Data Utility (PHDU) on PHO's Azure platform.
Key Responsibilities
- Participate in product teams to analyze system requirements, and architect, design, code, and implement cloud-based data and analytics products that conform to standards.
- Design, create, and maintain cloud-based data lake and Lakehouse structures, automated data pipelines, analytics models, and visualizations (dashboards and reports).
- Liaise with IT colleagues to implement products, conduct reviews, resolve operational problems, and support business partners in the effective use of cloud-based data and analytics products.
- Analyze complex technical issues, identify alternatives, and recommend solutions.
- Prepare and conduct knowledge transfer.
- Automate the data pipeline for analytic products and advanced analyses, including ingestion, ETL, and production for the foundational capability and for HealthMap (DLSPH model data).
- Help identify and prioritize analytic use cases and integrate AI techniques (machine learning, NLP) into PHO's work.
- Increase PHO's capacity for modelling, forecasting, and scenario analysis.
- Build on and advance the PHDU and PHO's Azure data and analytics platform; support data acquisition through data-sharing and governance processes where required.
- Document methods and transfer knowledge to PHO staff so the pipeline and capability can be sustained beyond the engagement.
Key Deliverables
- Automated data pipelines and cloud data lake / Lakehouse structures supporting the capability and HealthMap.
- Analytics models and visualizations (dashboards and reports) conforming to standards.
- Prioritized AI / ML use cases and supporting models (forecasting, scenario analysis).
- Knowledge-transfer documentation and sessions for PHO technical staff.
RequirementsRequired Skills
- Experience with multiple cloud-based data and analytics platforms and coding / programming / scripting tools to create, maintain, support, and operate cloud-based data and analytics products.
- Experience designing, creating, and maintaining cloud-based data lake and Lakehouse structures, automated data pipelines, analytics models, medallion architecture, and visualizations (dashboards and reporting) in real-world implementations.
- Deep experience with modern technology stacks: Azure Storage, Azure Data Lake, Azure Databricks Lakehouse, and Azure Synapse. Power BI, Python, SQL, Azure Databricks, and Azure Data Factory.
- Experience assessing client information-technology needs and objectives.
- Experience problem-solving to resolve complex, multi-component failures.
- Experience preparing knowledge-transfer documentation and conducting knowledge transfer.
- A team player with a track record for meeting deadlines.
Desirable Skills
- Written and oral communication skills to participate in team meetings, write and edit systems documentation, prepare and present written reports on findings and alternative solutions, and develop guidelines / best practices.
- Interpersonal skills to explain and discuss the advantages and disadvantages of various approaches.
- Experience conducting knowledge-transfer sessions and building documentation for technical staff on architecting, designing, and implementing end-to-end data and analytics products.
Expected Skills
- Be an advanced professional able to apply concepts, practices, and procedures in practice.
- Work with minimal direction and lead and train others in technical components and concepts.
- Plan, lead, and deliver complex deliverables that provide options for decisions within the organization.
- Bring a high level of expertise in the required skill set, specialized in the technical area, and provide specific advisory support as required.
Must Haves:
- Experience with multiple cloud-based data and analytics platforms and coding / programming / scripting tools to create, maintain, support, and operate cloud-based data and analytics products.
- Experience designing, creating, and maintaining cloud-based data lake and Lakehouse structures, automated data pipelines, analytics models, medallion architecture, and visualizations (dashboards and reporting) in real-world implementations.
- Deep experience with modern technology stacks: Azure Storage, Azure Data Lake, Azure Databricks Lakehouse, and Azure Synapse. Power BI, Python, SQL, Azure Databricks, and Azure Data Factory.
- Experience assessing client information-technology needs and objectives.
- Experience problem-solving to resolve complex, multi-component failures.
- NOTE: We are looking for the same candidate to cover both data engineering/data pipeline work and data science specialist work.
