General Summary:
The Data Engineering Lead is responsible for the development and execution of data solutions that support product and technology initiatives including general application development activities, such as unit testing, code review, code deployment and technical documentation. This role also collaborates with Product and Engineering teams to design solutions and enable new data capabilities.
Key Responsibilities
- Lead and contribute to backend and ETL development effort of our data platform.
- Lead and provide hands-on new development as well as enhancement of existing data processes
- Design, maintain, and tune extraction, transformation, and load (ETL) processes using PL/SQL, SQL, Python, or Spark
- Provide architectural guidance and development/build standards for the team
- Promote collaboration through activities including design sessions, design reviews, and pair programming, etcetera
KEY DUTIES
- Develop and maintain data engineering solutions for the enterprise data platform
- Analyze business requirements and work with teammates to formulate supporting design and design documentation
- Other duties as assigned
KEY COMPETENCIES
- Thorough understanding of, and support for, Agile development methodologies
- Ability to design, collect, and analyze large datasets
- Ability to communicate technical concepts and designs to cross-functional and offshore teams who have varying levels of technical experience
- Proven data engineering, problem solving, and analysis skills
- Strong demonstrable SQL and Python skills
- High-level written and verbal communication skills
- Ability to think strategically
- Ability to adapt to changing conditions and lead others through change
- Analytical and problem-solving ability and orientation
- Demonstrated organizational, prioritization, and time management skills
- Attention to detail
- Ability and willingness to travel nationally to remote offices and partners approximately 10% of the time
Required Education, Certifications, and Experience
- Bachelor’s degree in Computer Science, Mathematics, or Statistics
- 5+ years of experience of ETL development in a big data environment
- 5+ years working in an agile development environment
- Technical writing experience in relevant areas, including queries, reports, and presentations
Preferred Qualifications
- Experience in a diverse set of Amazon Web Services data services including:
- EC2
- S3
- Athena
- Redshift
- Aurora MySQL
- RDS
- Lambda
- Step Functions
- Glue
- Development experience with Python, PySpark, or R
- Significant Experience developing in Oracle databases
Key Differentiators
- Application, system or data architecture experience
- Machine learning experience