Position Overview
SiloSmashers is hiring for a Data Engineer to support digital transformation efforts for a government customer. The selected candidate will serve as part of a team responsible for architecting, developing, optimizing, and maintaining high-performance data ingestion, transformation, streaming, and analytical pipelines within Databricks-enabled Lakehouse environments. This role is expected to collaborate closely with the federal technical teams, and cross-task platform, SRE, governance, privacy, and development personnel.
Required Skills
• Foundational experience designing and implementing Databricks Lakehouse solutions, including:
o Basic Delta Lake table design, schema evolution, and familiarity with medallion architecture (bronze/silver/gold).
o Hands on experience with Databricks Workflows/Jobs, Auto Loader, COPY INTO, Structured Streaming, and introductory Databricks SQL Warehouse usage. Understanding of common performance considerations (e.g., cluster sizing, partitioning, basic caching strategies).
o Practical experience building batch or streaming pipelines using Python and PySpark, along with cloud native services in AWS or Microsoft Azure.
• Experience implementing secure cloud data integrations or platform configurations such as external locations, storage credentials, secret scopes, or identity federation basics.
• Experience working with Git based workflows and CI/CD tooling (e.g., GitHub Actions, Azure DevOps), plus familiarity with Databricks Repos, Asset Bundles, or basic Terraform/IaC concepts.
• Introductory experience or interest in AI/ML enabled engineering, such as Databricks Model Serving, Genie, Agent Bricks, or equivalent cloud native ML platforms.
Mandatory Qualifications & Requirements
• Ability to obtain and maintain a Public Trust clearance.
• Eight (8) or more years of experience (2 of which much be within the last five (5) years) in data engineering, ETL/ELT development, or distributed data processing within cloud environments or platforms (e.g. AWS, Azure, GCP, Snowflake, Databricks, etc.).
• Bachelor's degree in computer science, information systems, engineering, or related field; or four (4) or more years of equivalent experience within the last six (6) years in lieu of a degree.
• Databricks Data Engineer Associate Certification.
• At least one foundational cloud certification (e.g., AWS Cloud Practitioner, AWS Developer Associate, Azure Fundamentals, or equivalent).
• Preferred but not required: Databricks Lakehouse Architect, Generative AI Associate/Professional, or related ML/AI certifications.
• This position requires regular, physical attendance at customer sites.
