Data Engineers are responsible for the data infrastructure. The build systems designed for reliable data delivery and ingestion. Data Engineers develop and maintain internal and external facing APIs. They care deeply about data cleanliness and developing robust data pipelines.
What they will do:
They will be responsible for production grade data pipelines, databases, and data availability.
Minimum Requirements:
- Production grade, big data ETL pipeline experience.
- Building and maintaining distributed services.
- Hadoop, spark, apache beam, dataflow, hive, storm, etc.
- Real-time big data streaming.
- Microservice architecture design and development.
- Strong understanding of data structures and algorithms.
Plus, but not required:
- Previous experience in tech industry (GOOG, AMZN, FB, NFLX, Spotify, etc).
- Experience building and deploying distributed sql & nosql databases (i.e., Cassandra).
- Experience scraping large scale public data-sets in real time.
- Distributed, GPU-based cluster architecture.
- Experience with large-scale ML engineering.
- Experience building frontend systems.
