Role SummaryThe Data Engineer designs, implements, and maintains the enterprise data architecture using Microsoft Fabric. This role focuses on creating scalable data pipelines, implementing the medallion architecture within lakehouses, and ensuring data quality and accessibility. The position functions as a technical implementer within the data engineering team, collaborating with data consumers and source system owners to create a robust data foundation. All activities will be performed in support of the strategy, vision and values of JE Dunn.
- Autonomy & Decision-Making: Makes decisions on non-routine matters, provides recommendations to supervisor, and refers all exceptions to supervisor.
- Career Path: Senior Data Engineer
Key Role Responsibilities - Core- Design and implement a data mesh architecture using Microsoft Fabric's OneLake Lakehouses, including data governance and security measures
- Establish data transformation processes that ensure data quality and consistency
- Develop and maintain both synchronous (real-time) and batch data integration processes, implementing unidirectional and bidirectional data flows between systems as needed
- Develop and maintain API and delta connections for system interoperability
- Implement data validation processes to ensure successful data exchange between systems
- Monitor system performance, compute resources, and storage utilization
- Troubleshoot and resolve data pipeline failures and performance bottlenecks
- Maintain comprehensive documentation of data architecture, pipelines, and processes
- Foster cross-departmental collaboration to identify data synergies and opportunities
- Monitor and maintain deployed machine learning models
- Document ML model data dependencies, pipelines, and infrastructure requirements
- Support ML model development by ensuring reliable data pipelines for model training and inference
Key Role Responsibilities - Additional CoreN/A
Knowledge, Skills & Abilities- Ability to perform work accurately and completely, and in a timely manner
- Communication skills, verbal and written
- Time management skills
- Ability to prioritize multiple projects
- Ability to drive change through innovation & process improvement
- Ability to communicate to executive audiences and deliver results through influence
- Proficiency in MS Office - Intermediate
- Proficiency in SQL - Intermediate
- Proficiency in Python - Intermediate
- Proficiency in Spark - Intermediate
- Experience implementing data mesh architecture and medallion models
- Experience with various data integration patterns and technologies
- Knowledge of data governance and security best practices
- Experience developing and consuming REST APIs for data integration (Preferred)
- Familiarity with Power BI and Microsoft Fabric (Preferred)
- Experience with Microsoft Purview or similar data catalog and governance tools (Preferred)
- Understanding of machine learning concepts and MLOps (Preferred)
Education- Bachelor's degree in computer science, data engineering, data science or related field (Required)
- In lieu of the above requirements, equivalent relevant experience will be considered.
Experience- 2+ years of experience in data analytics or data engineering (Required)
Working Environment- Must be able to lift up to 10 pounds
- May require periods of travel
- Normal office environment
- Frequent activity: Sitting, Viewing Computer Screen
- Occasional activity: Standing, Walking, Bending, Reaching above Shoulder
Benefits InformationThe benefits package aligned to this position is Professional Non-Union. Please click the link below for more details.
Click here for benefits details
This role is expected to accept applications for at least three business days and may continue to be posted until a qualified applicant is selected or the position has been cancelled.
Requisition ID: 64275
