A successful ad-tech healthcare company is looking for a Senior Data Engineer to join their team in NYC. The ideal candidate will have 5+ years of experience as a Data or Software Engineer.
This position can be fully remote and can pay up to $144K, DOE.
Some of the cutting-edge technologies they have recently implemented are Kafka, Spark Streaming, Presto, Airflow, and Kubernetes. Required Skills & Experience
- 5+ years of software engineering experience with Python
- Fluency in Python, Experience in Scala/Java is a huge plus
- Knowledge and exposure to distributed production systems i.e Hadoop is a huge plus
- Knowledge and exposure to Cloud migration is a plus
What You Will Be Doing
- Airflow - for job scheduling
- Docker - Packaged container image with all dependencies
- Graphite/Beacon - for monitoring data flows
- Hive - SQL data warehouse layer for data in HDFS
- Impala- faster SQL layer on top of Hive
- Kafka- distributed commit log storage
- Kubernetes - Distributed cluster resource manager
- Presto - fast parallel data warehouse and data federation layer
- Spark Streaming - Near time aggregation
- SQL Server - Reliable OLTP RDBMS
- Sqoop - Import/Export data to RDBMS
- Design, build and maintain reliable and scalable enterprise level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
- Optimize jobs to utilize Kafka, Hadoop, Presto, Spark Streaming and Kubernetes resources in the most efficient way
- Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc.)
- Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
- Collaborate within a small team with diverse technology backgrounds
- Provide mentorship and guidance to junior team members
- Competitive Salary: Up to $144K/year, DOE
You will receive the following benefits:
- Medical Insurance & Health Savings Account (HSA)
- Paid Sick Time Leave
- Pre-tax Commuter Benefit
Applicants must be currently authorized to work in the United States on a full-time basis now and in the future.
Workbridge Associates, part of the Motion Recruitment network, provides IT Staffing Solutions (Contract, Contract-to-Hire, and Direct Hire) in major North American markets. Our unique expertise in today’s highest demand tech skill sets, paired with our deep networks and knowledge of our local technology markets, results in an exemplary track record with candidates and clients.