Job Family:
Data Engineering & Architecture Consulting
Travel Required:
Up to 10%
Clearance Required:
Ability to Obtain Public Trust
What You Will Do:
- Develop and maintain scalable data ingestion, transformation, and curation pipelines using Databricks (Delta Lake, Delta Live Tables, Auto Loader) and AWS services, supporting consistent delivery of analytics-ready datasets across the Data Platform.
- Implement standardized batch and near real-time data pipelines that integrate legacy systems and cloud-native data sources, contributing to enterprise-wide data access, reuse, and platform consistency.
- Support full data pipeline lifecycle activities, including intake, requirements analysis, source profiling, and technical implementation, aligning development work with defined intake processes, SLAs, and governance.
- Build production-ready data pipelines that meet defined technical and documentation standards, including participation in validation, testing, and release processes to enable reliable and compliant production deployments.
- Apply data quality checks, validation rules, and observability practices to improve pipeline reliability, support monitoring, and contribute to platform stability and operational performance targets.
- Integrate pipelines with AWS services (e.g., S3, streaming frameworks, APIs) and enterprise data tools to support secure, scalable data movement and interoperability across the ecosystem.
- Contribute to governed data engineering practices by implementing metadata capture, lineage tracking, and supporting access control patterns aligned to enterprise data governance standards.
- Support analytics and reporting use cases by preparing curated datasets and enabling consumption through SQL-based access, dashboards, and enterprise BI tooling.
What You Will Need:
- Bachelor’s degree is required
- Minimum FOUR (4) years of experience in data engineering, with hands-on development of data pipelines in cloud or distributed data environments.
- Strong proficiency in Python, PySpark, and SQL for building and maintaining scalable ETL/ELT pipelines.
- Experience working with Databricks and Delta Lake to support ingestion, transformation, and curated data layer development.
- Working knowledge of AWS data services (e.g., S3, IAM, VPC) and integration patterns supporting secure and scalable data architectures.
- Experience implementing data quality checks, monitoring, and basic performance optimization techniques for pipeline efficiency and reliability.
- Familiarity with data governance concepts, including metadata, lineage, and access control frameworks in regulated environments.
- Experience working within Agile delivery environments and contributing to CI/CD-enabled development workflows.
What Would Be Nice To Have:
- Experience supporting enterprise data platforms or federal data modernization initiatives, particularly in highly regulated environments.
- Exposure to streaming technologies such as Kafka, Kinesis, or EventBridge for near real-time data processing.
- Familiarity with Databricks Unity Catalog and governance capabilities (RBAC, data masking, auditing).
- Experience using metadata/catalog tools such as Informatica EDC or similar platforms.
- Understanding of DevSecOps practices, including automated testing, deployment, and environment promotion across dev/test/prod.
- Exposure to performance optimization techniques (e.g., partitioning, Z-ordering, clustering) for large-scale data processing workloads.
What We Offer:
Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.
Benefits include:
Medical, Rx, Dental & Vision Insurance
Personal and Family Sick Time & Company Paid Holidays
Position may be eligible for a discretionary variable incentive bonus
Parental Leave and Adoption Assistance
401(k) Retirement Plan
Basic Life & Supplemental Life
Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
Short-Term & Long-Term Disability
Student Loan PayDown
Tuition Reimbursement, Personal Development & Learning Opportunities
Skills Development & Certifications
Employee Referral Program
Corporate Sponsored Events & Community Outreach
Emergency Back-Up Childcare Program
Mobility Stipend
