Bachelor's degree in IT, Computer Science, or related field (Master's preferred)
5-7 years of experience as a Data Engineer in a complex environment
Strong collaboration and communication skills
Advanced SQL skills, especially T-SQL
Extensive Python development experience, particularly with Azure Databricks
Proficiency in ETL processes and Azure Data Factory
Familiarity with CI/CD tools like Azure DevOps and GitHub Actions
Responsibilities
Design data models integrating with enterprise frameworks using various modeling tools.
Plan architecture leveraging Azure frameworks to meet strategic goals.
Implement data asset management for scalable technology solutions.
Build and maintain automated ETL pipelines on Azure platforms.
Ensure compliance with data regulations using Azure Purview.
Resolve conflicts in data models for consistency and governance.
Partner with Data Owners to design solutions adhering to data management best practices.
Benefits
Opportunity for professional growth in a cutting-edge Azure environment
Collaborative work culture with cross-functional teams
Exposure to innovative data solutions and technologies
Access to continuous learning resources and development programs
Engagement in high-impact projects influencing analytics and business strategy
Full Job Description
Sr. Data Engineer Atlanta, GA FULLTIME
ESSENTIAL JOB FUNCTIONS:
Design data models for key program processes and data entities, ensuring integration with enterprise conceptual and physical data models using hackolade or any other modelling tools.
Plan data architecture and design by leveraging Azure architectural frameworks, including CAF and Azure Well-Architected Framework, to align with solution concepts, strategy, and target audience.
Implement a disciplined approach to managing data assets and convert architectural principles into scalable technology solutions.
Build and maintain reliable, automated, and scalable ETL pipelines on Azure data platforms, supporting critical business functions and analytics, with a strong focus on Azure Databricks and Azure Onelake.
Ensure regulatory compliance for data assets, including consumer privacy laws and data retention policies, using Azure Purview for data governance and cataloging.
Resolve conflicts between data models, ensuring consistency with the enterprise model and alignment with data governance best practices.
Partner with Data Owners and Data Stewards to design solutions that adhere to data management principles and best practices.
Experience in implementing Master Data Management is a plus.
Lead the design of data architecture across multiple domains and sub-domains within the Azure ecosystem.
Coordinate new development activities to ensure they are consistent and well-integrated with the established Azure ecosystem data architecture.
Review and approve data model designs, focusing on interoperability, analytics, and compliance with the portfolio information model.
Required to draft reports and prepare presentations for senior leadership. These reports and presentation must be clear, concise, unambiguous, engaging and convincing, which will demand exceptional communication skills.
Who We Are Looking For EDUCATION & EXPERIENCE:
Bachelor's degree (B.A. / B.S.) in Information Technology or Computer Science or related field or the equivalent combination of education and experience, Master of Science (M.S.) in Data Analytics, Computer Science or related quantitative field a plus.
Must have had at least 5-7 years of working experience working in a data engineering department, as a Data Engineer or Sr/lead Data Engineer in a fast-paced environment and complex business setting
Proven ability to collaborate with formal and informal groups, prioritizing team needs and involving others in decision-making.
Effective communication and teamwork skills, with the ability to work with cross-functional teams.
Strong problem-solving and critical thinking abilities, with a focus on innovative cloud-based solutions.
Advanced SQL skills, particularly T-SQL, with experience in relational database management on Azure SQL Database.
Extensive experience in Python development, especially for data projects using Azure Databricks.
Proficiency in ETL processes and pipeline orchestration using Azure Data Factory.
Familiarity with version control systems and CI/CD tools, particularly Azure DevOps and GitHub Actions.
Ability to interact with all management levels to assess business requirements and deliver stable, cost-effective technology solutions.
Strong troubleshooting and problem-solving skills, with attention to detail and accuracy.
Experience in partnering with technical and cross-functional teams to deploy machine learning models in production using Azure Client and Databricks.
