Skills you'll need:
- 5+ Years experience in software engineering including, 2+ building batch and microbatch pipelines, 2+ with Spark, experience leading technical projects and with Linux/Unix
- Competency both Scala and Python
- Experience with Data Workflow frameworks such as Azkaban, Airflow, Pinwheel, Oozie
- Experience with data warehousing using a columnar or distributed databasetechnology.
- Solid SQL - You can write multi-table joins and aggregates comfortably
- Experience mentoring junior engineers into high performance coders.
Skills not required but highly desired:
- Experience with a relational columnar technology such as SparkSQL, Redshift/parAccel, Greenplum, or Vertica
- Familiarity with Amazon Web Services (AWS), Unix and open-source technologies
- Configuration Management experience
- Competitive salary, great benefits, and equity
- Flexible hours, no vacation policy, and paid maternity and paternity leave
- Social groups and sports teams
- Free lunch and dinner every weekday in the office, all-day snacks, and a full bar
- Free monthly MetroCards/PATH cards