Lead Data Engineer in Charlotte, NC

$80K - $100K(Ladders Estimates)

Red Ventures   •  

Charlotte, NC 28202

Industry: Media

  •  

Less than 5 years

Posted 37 days ago

The mission of our data team at Red Ventures to make data easy-to-use for everyone. We are hiring a Lead Data Engineer who is excited to overcome challenges, work with the latest AWS technologies, and continue making our data better.

Our Data Engineers are a key part of building our data infrastructure; this includes using Spark for streaming applications and/or as an ETL tool, Data Aggregation, User Defined Functions, and using the Spark UI to analyze behavior and performance.

You will lead a team of highly skilled engineers who design, develop and automate high-quality, scalable solutions across the entire data lifecycle, from raw data to powerful insights and analytics. Basically, we are using Spark as our universal program on how we are transferring data across multiple platforms.

What We're Looking For:

  • Spark, working in RDDs and DataFrames/Datasets API (with emphasis on DataFrames) to query and perform data manipulation
  • Spark Structured Streaming
  • At least 1 to 2 years experience functioning as a lead or senior resource on a team.
  • Experience building large scale Spark applications, ideally with either Batch processing and/or Streaming processing
  • Scala would be ideal but a solid knowledge of Java is also acceptable
  • Experience in SparkSQL (Broadcast Joins)
  • Experience with cloud computing platforms, we use AWS (Kinesis, S3, Lambda, DynamoDB)
  • Has experience with ANSI SQL relational database (Oracle, SQL, Postgres, MySQL)

Even Better:

  • Linux common working knowledge, including navigating through the file system and simple bash scripting
  • General knowledge of distributed systems and distributed data processing frameworks
  • Experience with Storm or Cassandra is a plus
  • Knowledge about agile software processes

Who We Are:

Founded in 2000, Red Ventures is a portfolio of growing digital businesses that bring consumers and brands together through integrated e-commerce, strategic partnerships and many proprietary brands including Bankrate, AllConnect.com and Reviews.com.

Valid Through: 2019-10-7