Big Data Engineer

Throtle

$130K — $146K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 4+ years of experience in Big Data technologies like Spark and SQL databases
  • Strong proficiency in programming with Scala, Java, Python, or Shell
  • Expertise in data modeling and database design
  • Experience building and maintaining ETL transformations
  • Knowledge of fuzzy logic matching tools
  • Familiarity with cloud-based infrastructure, especially AWS
  • Strong analytical abilities and problem-solving skills

Responsibilities

  • Design and implement data pipelines using Spark, Hadoop, and related technologies
  • Develop complex SQL queries for data extraction and optimize performance
  • Create software components integrating identity graphs and client data
  • Monitor and plan capacity for processing environments
  • Maintain technical documentation for data workflows
  • Ensure data integrity through thorough research and analysis
  • Collaborate with teams to meet business requirements and enhance software scalability

Benefits

  • 401(k) and matching contributions
  • Company-sponsored events and parties
  • Flexible work schedule
  • Free food and snacks in the office
  • Comprehensive health, dental, and vision insurance
  • Generous paid time off including parental leave
  • Opportunities for training and professional development
Full Job Description
Replies within 24 hours

Benefits:
  • 401(k)
  • 401(k) matching
  • Company parties
  • Competitive salary
  • Dental insurance
  • Flexible schedule
  • Free food & snacks
  • Health insurance
  • Paid time off
  • Parental leave
  • Training & development
  • Vision insurance


Job Summary

Please consider joining our technology team at Throtle as a Big Data Developer. In this role, you will be part of the team that is responsible for transforming and maintaining several billion records that make Throtle's data onboarding solution work. You'll need to be able to evaluate incoming data, perform specialized hygiene, and standardize it for consumption by multiple processes. You'll be working in a fast-paced, high volume processing environment where quality and attention to the details are paramount.

Duties/Responsibilities

  • Design and implement data pipelines and transformations using big data technologies such as Spark, Hadoop, and related ecosystems
  • Develop and maintain complex SQL queries for data extraction and reporting, and optimize query performance and scalability
  • Design and develop software components that integrate with our identity graphs, client data, and keying processes
  • Participate in capacity monitoring and planning
  • Develop and maintain technical documentation
  • Ensure data integrity and quality in development process, including conducting data research and analysis to identify trends and patterns
  • Work with cross-functional teams to ensure software systems meet business requirements and are scalable for future growth, including developing predictive models and enhancing data quality
  • Position will require that the individual understands all regulations and laws applicable to their assigned roles and responsibilities. Additionally, the individual will be responsible for the development, implementation, and regular maintenance of policies and procedures that govern the work of assigned roles and responsibilities, including compliance with the security requirements of ePHI.


Required Skill and Abilities

  • Experience with large-scale data processing and analytics platforms, including Hadoop, Spark,or related ecosystems
  • Knowledge of distributed computing concepts and experience with cloud-based infrastructure
  • Experience with fuzzy logic matching and tools
  • Experience with AWS infrastructure


Education and Experience

  • 4+ years of experience working with Big Data technologies, including, Spark, and SQL databases
  • Proficiency in programming languages such as Scala, Java, Python, Shell or a combination thereof
  • Expertise in data modeling, database design, and development
  • Experience in building and maintaining data transformations (ETL) using SQL, Python, Scala, or Java
  • Ability to analyze, troubleshoot and performance tune queries
  • Ability to identify problems, and effectively communicate solutions to peers and management
  • Strong analytical and problem-solving skills to address complex software and data challenges
  • Ability to work effectively with cross-functional teams and communicate technical designs and solutions to non-technical stakeholders


Flexible work from home options available.

Compensation: $130,000.00 - $146,000.00 per year

Similar Jobs

More Jobs at Throtle

  • Big Data Engineer
    $130K — $146K *
    Red Bank, NJ 07701 (Monmouth County)
    Information Technology
    Hybrid

More Information Technology Jobs

Find similar Big Data Engineer jobs: