Data Engineer

OpenDataJobs

$90K — $130K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years of hands-on data engineering experience on AWS
  • Proficient in Spark ETL processes using Glue and EMR
  • Skilled in Python and PySpark programming
  • Experience in designing S3 data lakes for warehouse integration
  • Strong understanding of event orchestration and data quality management
  • Familiarity with infrastructure-as-code practices
  • Ability to work within a regulated environment ensuring accuracy and auditability

Responsibilities

  • Build and maintain data ingestion, processing, and storage pipelines on AWS
  • Design and implement a data lake structure serving the data warehouse
  • Develop automated data quality checks and validation processes
  • Ensure effective monitoring of data pipelines and business logic
  • Collaborate with teams to explore new capabilities and business lines
  • Participate in all phases of data engineering, from experimentation to deployment
  • Own deliverables and contribute to team outcomes rather than just tasks

Benefits

  • Fully paid medical, dental, and vision insurance
  • Matched 401(k) plan with immediate vesting
  • Unlimited paid time off for work-life balance
  • Sponsored professional certifications to enhance skills
  • Opportunity to work on meaningful projects that impact real-world challenges
Full Job Description
Your first assignment is to build the pipelines that move a federal agency's data from source to warehouse on Amazon Web Services (AWS): ingest, process, store, and keep it clean and trustworthy at scale. The work is real, hard, and it matters. It is also where you start, not the shape of your career here: we hire people, not seats, and we move our best to where the hardest problems are. What you'll build • Ingest-process-store pipelines on AWS: Spark-based extract, transform, and load (ETL) with Glue, Amazon EMR, Lambda, and Step Functions, in Python and PySpark. • A data lake that feeds the warehouse: S3 design (Parquet, partitioning, lifecycle) into Redshift, with event orchestration, secrets and monitoring, and data quality, validation, and lineage built in. • In time, the firm itself: new capabilities, tools, and lines of business you help spin up. Who you are You are a data engineer who is serious about the craft and eager to go deeper. You care as much about whether the data is trustworthy as whether it arrives, and you do your best work alongside people who push you. You experiment, fail, learn, and repeat quickly. You would rather own an outcome than be handed a task. What you bring • Hands-on data engineering on AWS: Spark ETL (Glue, EMR), Python and PySpark, and S3 data-lake design feeding a warehouse. • The reliability craft around it: event orchestration, data quality and lineage, monitoring, and infrastructure-as-code. • The judgment to build in a regulated environment where accuracy and auditability are not optional. What you'll need U.S. citizenship (this initial engagement may be staffed by U.S. citizens only) and the ability to obtain a Public Trust determination. This is a hybrid role based in the Washington, DC metropolitan area, and it requires commuting into DC regularly. Everything else, the years, the certifications, the specific tools, we ask in the application and get into during the interview. What we offer Fully paid medical, dental, and vision; a matched 401(k) with immediate vesting; unlimited paid time off; and sponsored certifications. Beyond that, work worth your best years.

Similar Jobs

More Jobs at OpenDataJobs

  • Business Analyst
    $80K — $110K *
    Washington, DC 20011 (District Of Columbia County)
    Finance & Insurance
    In-Person
  • Software Engineer
    $90K — $130K *
    Washington, DC 20011 (District Of Columbia County)
    Information Technology
    In-Person
  • Software Developer
    $90K — $130K *
    Washington, DC 20011 (District Of Columbia County)
    Information Technology
    In-Person
  • Database Developer
    $90K — $120K *
    Washington, DC 20011 (District Of Columbia County)
    Information Technology
    In-Person
  • XBRL / Arelle Analyst
    $80K — $120K *
    Washington, DC 20011 (District Of Columbia County)
    Finance & Insurance
    In-Person

More Information Technology Jobs

Find similar Data Engineer jobs: