Full Job Description
Zantech is looking for a talented Data Engineer to contribute to the success of our Technical Infrastructure and Platform Support project for an On-Site role based out of Arlington, VA.
The Data Scientist owns the day-to-day health of the automated data pipelines that feed the Data and Analytics Platform from internal Navy systems and external sources. The Data Engineer implements the established data architecture by integrating new data sources, performing data cleansing and quality assurance, and resolving pipeline failures within an AWS GovCloud environment.
Responsibilities include, but will not be limited to:
- Maintain, monitor, and optimize automated ETL/ELT pipelines covering 10 to 20 data sources and 1 to 20+ TB of data, with ongoing growth of 1+ TB per year.
- Build and modify pipelines using AWS Glue, Step Functions, Lake Formation, Athena, and RDS.
- Write modular, parameterized pipeline code so that data structures and refresh frequencies can change without rebuilding the core pipeline architecture.
- Support three ingestion velocities: scheduled batch processing, time-based interval micro-batches, and user-triggered on-demand refreshes for time-sensitive financial submissions.
- Ingest and export data in open, non-proprietary formats (Parquet, CSV, JSON).
- Perform data cleansing, validation, and quality assurance as new sources are integrated.
- Serve as a primary resolver for Tier 3 tickets involving pipeline failures, refresh errors, and database performance issues.
- Implement Row-Level and Column-Level Security controls at the data layer in coordination with the Data Architect.
- Document pipeline configurations, data lineage, and operating procedures.
Required Experience or Knowledge of the following technologies/functions:
- Active Secret security clearance.
- Demonstrated experience building and maintaining ETL/ELT pipelines in a cloud environment.
- Proficiency in Python and SQL.
- Experience with data cleansing, data quality validation, and integrating structured and unstructured data from multiple sources.
- DoDM 8140.03 qualifying certification (e.g., CompTIA Security+ CE) prior to start.
Preferred Qualifications:
- Hands-on experience with AWS Glue, Step Functions, Lake Formation, Athena, and RDS.
- Experience with Infrastructure as Code (Terraform or CloudFormation) applied to data pipelines.
- Experience in AWS GovCloud or DoD IL4/IL5 environments.
- Familiarity with Advana/Jupiter, Qlik, or Power BI data sources.
- AWS certification (e.g., AWS Certified Data Engineer - Associate).
- Experience supporting federal financial or acquisition data.
Required Security Clearance:
- Active DoD Secret or higher clearance, per contract requirements.
