We are looking for a talented engineer to join our growing data engineering team. The ideal candidate has significant experience in leading a small group of engineers that build scalable data platforms that enable business intelligence, analytics, data science, and data products. You must have strong, hands-on technical expertise in a variety of technologies and the proven ability to fashion robust, scalable solutions. You should have a passion for continuous improvement quality.
We embrace a wide variety of technologies and work very closely with data scientists and business stakeholders to deliver end to end solutions. If you are interested in a fast paced environment, the latest technologies, and fun data problems come join us!
- Design and develop big data applications using a variety of different technologies.
- Develop logical and physical data models for big data platforms.
- Automate workflows using Apache Airflow.
- Write data pipelines using Apache Hive, Apache Spark, Apache Kafka.
- Create solutions on AWS using services such as Kinesis, Lambda, and API Gateway.
- Provide ongoing maintenance and enhancements to existing systems, and participate in rotational on-call support.
- Learn our business domain and technology infrastructure quickly and share your knowledge freely and proactively with others in the team.
- Mentor junior engineers on the team
- Lead daily standups and design reviews
- Groom and prioritize backlog using JIRA
- Act as the point of contact for your assigned business domain
- 7+ years of hands-on experience with developing data warehouse solutions and data products.
- 4+ years of hands-on experience developing a distributed data processing platform with Hadoop, Hive, Spark, Airflow, Kafka, etc.
- 2-3 years of hands-on experience in modeling and designing schema for data lakes or for RDBMS platforms.
- Experience managing a small team of engineers
- Experience with programming languages: Python, Java, Scala, etc.
- Experience with scripting languages: Perl, Shell, etc.
- Practice working with, processing, and managing large data sets (multi TB/PB scale).
- Exposure to test driven development and automated testing frameworks.
- Background in Scrum/Agile development methodologies.
- Capable of delivering on multiple competing priorities with little supervision.
- Excellent verbal and written communication skills.
- Bachelor's Degree in computer science or equivalent experience.
Nice To Have
- Experience building machine learning pipelines or data products.
- Familiarity with AWS or GCS technologies.
- Be passionate about or have contributed to open sourced engineering projects in the past.