Senior Data Engineer

RadiumOne   •  

San Francisco, CA

Industry: IT Consulting/Services

  •  

5 - 7 years

Posted 269 days ago

This job is no longer available.

Responsibilities

  • Create and maintain optimal data pipeline architecture
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater reliability, etc.
  • Build the infrastructure required for optimal ETL of data from a wide variety of data sources using SQL and Hadoop.
  • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
  • Work with stakeholders including the reporting team, account management, and data science.
  • Assist with data-related technical issues and support their data infrastructure needs.
  • Enhance monitoring of our ETL pipelines.

Qualifications

  • Minimum of 5 years of experience in a Data Engineering role.
  • Experience building and optimizing big data data pipelines, architectures and data sets.
  • Strong analytic skills related to working with unstructured data sets.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Understanding of message queuing, stream processing, and highly scalable data stores.
  • Experience supporting and working with cross-functional teams.
  • Experience using the following software and languages:
    • Hive
    • Hadoop
    • Yarn
    • Spark
    • Kafka
    • MySQL or MariaDB
    • SQL
    • Python
    • Unix shell script
  • Bachelor’s degree in Computer Science or similar field, advanced degree preferred.