Data Engineer

AM Best Company$90K — $130K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's Degree in a relevant field
  • 4+ years experience as a Data Engineer or similar role
  • Active Databricks Data Engineer/Analyst certification
  • Proficient in Python with 4+ years continuous experience
  • Strong SQL and PL/SQL skills with 3+ years experience

Responsibilities

  • Lead the transition to the Databricks platform
  • Design and develop data pipelines for data extraction and loading
  • Optimize data pipelines for enhanced performance and scalability
  • Implement data quality checks within data pipelines
  • Maintain documentation for data mappings and architecture
  • Create proof-of-concepts for new processes and technologies
  • Troubleshoot and resolve data-related issues

Benefits

  • Flexible and hybrid work arrangements
  • Paid time off and paid company holidays
  • Medical plan options and prescription drug plan
  • Dental and vision plan options
  • Flexible spending and health savings accounts
  • 401(k) retirement savings plan with company matching
  • Educational assistance program
Full Job Description
Benefits Summary

  • Flexible and hybrid work arrangements
  • Paid time off/Paid company holidays
  • Medical plan options/prescription drug plan
  • Dental plan/vision plan options
  • Flexible spending and health savings accounts
  • 401(k) retirement savings plan with a Roth savings option and company matching contributions
  • Educational assistance program


Overview

The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. The ideal candidate will engage with cross-functional teams to gather data requirements, propose enhancements to existing data pipelines and structures, and ensure the reliability and efficiency of data processes.

Responsibilities
• Develop and maintain scripts and tools using Python, PowerShell, and R
• Design, write, and optimize SQL queries for performance and scalability
• Help modernize 'legacy' solutions to realign with our current code base and tech stack
• Assist in redevelopment, improvement and ongoing maintenance of existing data, analytics, and reporting solutions
• Ensure accurate and efficient data integration of diverse data sources and formats
• Enhance and support database functions and procedures
• Optimize data access and data processing workflows for performance, scalability, and efficiency
• Implement data quality checks and validations to ensure the accuracy, consistency, and completeness of data
• Identify and resolve performance bottlenecks, investigate and troubleshoot data related issues, and provide solutions to address defects
• Seamlessly transition between production support and development tasks based on business needs
• Deploy and manage code utilizing engineering best practices in non-prod and prod environments

Qualifications
• Bachelors Degree in computer science, data science, software engineering, information systems, or related quantitative field.
• Minimum 4 years of experience working as a Python Developer, Solutions Engineer, Data Engineer, or similar roles.

Skills
• 4+ years of solid continuous experience in Python
• 3+ years of solid experience writing SQL and PL/SQL code
• 3+ years of experience working with relational databases (solid understanding of Oracle preferred)
• 2+ years of experience scripting with PowerShell
• Experience programming in R
• Experience with web application frameworks including Shiny, Dash, Streamlit
• Experience with CI/CD utilizing git/Azure DevOps
• Knowledge of alternative storage formats including Parquet/Arrow/Avro
• Ability to collaborate within and across teams of different technical knowledge to support delivery of solutions
• Expert problem-solving skills, including debugging skills, enabling the determination of sources of issues in unfamiliar code or systems

Pluses, but not required: Any work experience in the following:
ETL / ELT tools: Spark, Kafka, Azure Data Factory (ADF)
NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar
Languages: SAS, Java, Scala, .Net

Similar Jobs

More Jobs at AM Best Company

  • Data Engineer
    $90K — $130K *
    Oldwick, NJ 08858 (Hunterdon County)
    Information Technology
    In-Person
  • Instructional Designer
    $70K — $95K *
    Oldwick, NJ 08858 (Hunterdon County)
    Education, Government & Non-Profit
    In-Person
  • Sr. Analyst - Industry Relations
    $80K — $110K *
    Oldwick, NJ 08858 (Hunterdon County)
    Finance & Insurance
    In-Person

More Information Technology Jobs

Find similar Data Engineer jobs: