Responsibilities
As a Staff Data Engineer, you'll use an advanced analytical, data-driven approach to drive a deep understanding of our fast-changing business and answer real world questions. You'll work with stakeholders to develop a clear understanding of data and data infrastructure needs, resolve complex data-related technical issues, and ensure optimal data design and efficiency.
We are looking for:
- 8+ years of experience related to data engineering
What is the work?
• Designs data patterns that support creation of datasets for analytics; implements calculations, cleanses data, ensures standardization of data, maps / links data from more than one source
• Performs data validation and quality assurance on work of senior peers and write automated tests
• Maintains / streamlines / orchestrates existing data pipelines end to end
• Builds / supports complex data warehousing pipelines into cloud platforms GCP/AWS, data integrations, data streaming solutions, predictive model implementations with prompt engineering for AI Models, building out data quality checks into data pipelines that are on BigQuery or Databricks
• Works with peer staff engineers to perform design reviews and swiftly change patterns according to latest tools and technologies
• Designs and develops real-time streaming requirements by using structured streaming from either pub-sub or another streaming table on datalake
• Builds / supports more complex data pipelines including application programming interfaces (APIs)
• Identifies complex data from upstream sources to enable new capabilities.
• Engages in testing of the technical solutions to ensure data integrity and system functionality of own work and broader system, E2E
• Builds large-scale batch and real-time data pipelines with big data CDC processing frameworks
• Designs / develops data integrations to support application engineering and system integration
• Designs / develops / maintains large data pipelines; diagnoses / solves production support issues
• Creates documentation and training related to technology stacks and standards within assigned team
• Designs / implements monitoring capabilities based on business SLA and data quality
• Uses / contributes to refinement of Digital Engineering-related tools, standards, and training
• Engages / collaborates with external technical teams to ensure timely, high-quality solutions
• Engages with shared services teams and vendors when necessary
• Works closely with Product, Data Science, Application, and Analytics teams to develop a clear understanding of data and data infrastructure needs; assists with data-related technical issues; ensures optimal data design and efficiency
• Performs full SDLC process, including planning, design, development, certification, implementation, and support
• Interacts with Product, Business, Analyst stakeholders to confirm data quality, discuss requirements, and support data testing
• Peer reviews with team members; learns / adapts from peer review of own code
• Contributes to overall design, architecture, security, scalability, reliability, and performance
• Mentors / provides support to Senior Data Engineers
• Builds more complex data models to deliver insightful analytics; ensures highest standard in data integrity
• Has knowledge in machine learning concepts
What is your background?
• A related degree or work experience, preferably a Bachelors degree in related work stream
• 8+ years of experience related to data engineering
• Experience in Lean Startup or Agile development methodologies
• Experience working in large scale cloud infrastructure, large data sets, and mission critical SLAs
• Experience in Advanced SQL / Python.
• Have a data engineering certification from any of the cloud technologies
Do you have what it takes to be an H-E-B Staff Data Engineer?
• Advanced knowledge of Lean Startup / Agile methods
• Strong understanding business intelligence, analytics and reporting, and application integration
• Knowledge of data architectures such as data warehouse, data lake, and data mesh and when to apply
• Strong working understanding of data architecture and data modeling best practices and guidelines for various data and analytic platforms
• Strong working understanding of coding standards and design principles / patterns
• Strong prioritization skills
• Strong verbal / written communication and data presentation skills
• Ability to deliver on ambiguous projects with incomplete information
• Ability / willingness to learn new technologies as they emerge
• Ability to calmly work under pressure
• Ability to work a flexible schedule as needed
• Ability to collaborate across multiple teams at multiple work locations
• Ability to work within a team, and willingness to take feedback from peers and mentors
Can you...
• Function in a fast-paced environment with the ability to work from home and / or office
• Work extended hours; sit for extended periods
JDENGINEERING
Last revised: 11/01/2024
JDENGINEERING
