Active DoD Secret clearance and U.S. citizenship required.
Strong experience in data engineering, ETL/ELT development, or data warehousing.
Proficient in PySpark and/or Spark SQL with solid SQL fundamentals.
Experience with Databricks and Medallion Architecture (Bronze, Silver, Gold).
Familiarity with Delta Lake features like ACID transactions and schema evolution.
Experience designing batch and streaming ingestion pipelines.
Excellent communication and organizational skills.
Responsibilities
Build, operate, and maintain ingestion pipelines within Databricks.
Develop and support Bronze, Silver, and Gold data pipelines using Medallion Architecture.
Migrate data and business logic from legacy systems to cloud environments.
Gather and analyze technical and business requirements.
Design and develop batch and streaming ingestion pipelines.
Monitor data jobs and troubleshoot production issues.
Document technical processes and collaborate in an Agile environment.
Benefits
Paid time off (PTO)
Medical, dental, and vision coverage
Life insurance
Long-term disability insurance
401(k) plan
Additional optional benefits
Full Job Description
Overview
TSGi is seeking a Senior Databricks Data Engineer to support a large-scale federal technology modernization program. This is a remote position supporting the development and migration of cloud-based data solutions.
The Data Engineer will build and operate ingestion pipelines and Bronze, Silver, and Gold data layers within Databricks, helping migrate data and business logic from legacy environments into a modern cloud architecture.
An active DoD Secret clearance and U.S. citizenship are required for this position.
Responsibilities
Build, operate, and maintain ingestion pipelines within Databricks.
Develop and support Bronze, Silver, and Gold data pipelines using Medallion Architecture.
Migrate data and business logic from legacy systems into modern cloud-based environments.
Gather and analyze technical and business requirements.
Design and develop batch, incremental, and streaming ingestion pipelines.
Monitor data jobs and troubleshoot pipeline and production issues.
Provide post-go-live production support and break/fix assistance.
Document technical processes, data workflows, and procedures.
Collaborate with technical teams and stakeholders in an Agile environment.
Qualifications
Per our Federal Government Contract, Active DoD Secret clearance and U.S. citizenship required.
Strong experience in data engineering, ETL/ELT development, or data warehousing.
Strong PySpark and/or Spark SQL skills with solid SQL fundamentals.
Experience with Databricks and Medallion Architecture, including Bronze, Silver, and Gold layering.
Experience with Delta Lake, including ACID transactions, schema evolution, time travel, and MERGE/upsert patterns.
Experience designing batch and incremental/streaming ingestion pipelines.
Familiarity with Agile processes.
Excellent written and verbal communication skills.
Strong interpersonal, organizational, and prioritization skills.
Databricks certification or additional hands-on Databricks experience preferred.
DoD, Navy, or other federal experience preferred.
AWS GovCloud experience preferred.
Experience migrating code from non-accredited to accredited AWS environments is a plus.
Compensation:
W2 Hourly rate starts at $88. Pay rates are based on experience, qualifications, location, and certifications, and are subject to change at any time.
Benefits vary by compensation type and may include paid time off (PTO), medical, dental, and vision coverage, life insurance, long-term disability insurance, a 401(k) plan, and additional optional benefits.
