Job Summary
We are seeking a Data Engineer with strong technical expertise and a DevOps mindset to contribute to engineering development and operations, from pre-production through production. The role involves building and maintaining data pipelines using Azure Cloud, Azure Databricks, and Snowflake, while promoting engineering best practices, knowledge sharing, and continuous improvement. The Data Engineer will participate in production releases, pipeline modernization, deployment automation, production monitoring, and on-call support as needed. The ideal candidate will take ownership of software development, release pipelines, production monitoring, security, and support while collaborating with engineering teams to deliver reliable and secure data solutions.
Key Responsibilities
• Create, maintain, and enhance data pipelines using Azure Cloud, Azure Databricks, and Snowflake.
• Document engineering solutions, transfer knowledge to other engineers, and support future enhancements and maintenance.
• Create secure data views and publish them to the Enterprise Data Exchange through Snowflake for consumption by other teams.
• Modernize and migrate data pipelines using Databricks Delta Live Tables (DLT) and Unity Catalog.
• Use existing CI/CD processes to deploy data pipelines.
• Adhere to personally identifiable information (PII) encryption and masking standards.
• Support improvements to existing CI/CD processes and deployment practices.
• Monitor production environments and troubleshoot pipeline failures.
• Serve as an escalation point and participate in on-call support rotations.
• Participate in discussions and initiatives to improve DevOps practices.
• Understand product releases and their potential impact on business operations.
• Participate in Agile ceremonies and contribute to engineering assignments using established procedures and best practices.
• Conduct research to support product troubleshooting and optimization.
• Participate in and contribute to the Engineering Community of Practice.
• Share technical knowledge and help other engineers improve their skills.
• Support production release activities and operational reliability across the development lifecycle.
Required Qualifications
• 5+ years of relevant engineering experience.
• Experience with Azure Cloud deployments and configurations.
• Experience with Azure Data Factory, Azure Databricks, Snowflake, and Storage Accounts.
• Experience with DBT/DLT and Medallion Architecture.
• Experience with CI/CD principles and best practices, including Azure DevOps and Repos.
• Experience working within a Data Engineering team and understanding of data engineering practices.
• Ability to learn and work quickly with emerging technologies, methodologies, and cloud/IT solutions.
• Experience with bug-tracking and task-management tools such as Jira.
• Experience managing outages, customer escalations, crisis situations, and similar circumstances.
• Ability to support production monitoring, troubleshooting, and on-call responsibilities.
• Strong collaboration and knowledge-sharing skills, with a commitment to engineering best practices.
Preferred Qualifications
• Completed bachelor's degree or diploma, or equivalent experience, in Computer Science, Software Engineering, Software Architecture, or a related field. Candidates with substantial and relevant industry experience are also eligible.
• Azure certification.
• Experience with Terraform and ARM templates.
Key Competencies
• Takes personal responsibility and recognizes the choices available in every situation.
• Demonstrates an entrepreneurial spirit and continuously innovates to achieve strong results.
• Communicates honestly and kindly while creating an environment where others can do the same.
• Leads with courage and embraces opportunities to achieve meaningful results.
• Builds trusting relationships and fosters connection by putting people first.
• Brings fun and joy to the workplace and maintains a positive, approachable attitude.
