Azure Data Engineer (Databricks, Python, Snowflake, Microsoft Fabric)
Location
- Frisco, TX
- Seattle, WA
- Atlanta, GA
Work Arrangement
- 100% Onsite (5 Days per Week)
Preferred Candidate Background
Candidates with full-time experience at leading technology companies are highly preferred, including:
- Google
- Meta
- Amazon
- Apple
- Netflix
- Microsoft
- Nvidia
- Salesforce
- Oracle
- IBM
- Cisco
- Adobe
- Palantir
- Snowflake
- Databricks
- X (Twitter)
- LinkedIn
- Uber
- Airbnb
Required Experience
SkillMinimum Experience
Microsoft Fabric
4+ Years
Data Engineering
4+ Years
Databricks
4+ Years
Python
4+ Years
Snowflake
4+ Years
Key Responsibilities
- Develop scalable, production-ready data engineering solutions.
- Design, build, and maintain end-to-end data pipelines for ingestion, transformation, storage, and reporting.
- Implement data ingestion using:
- Batch processing
- Change Data Capture (CDC)
- Streaming
- API integrations
- Design and optimize physical data models in Snowflake and Databricks.
- Work with:
- Delta Lake
- Unity Catalog
- Apache Iceberg
- Medallion Architecture (Bronze, Silver, Gold)
- Develop semantic models using Microsoft Fabric, including OneLake and Fabric Intelligence.
- Build Power BI dashboards, reports, and visualizations.
- Implement testing strategies including:
- Unit testing
- Integration testing
- Data quality validation
- Data reconciliation
- SLA monitoring
- Provide production support (KTLO) for critical data pipelines.
- Maintain technical documentation, data lineage, runbooks, and data dictionaries.
- Follow DevOps best practices, including:
- Git
- CI/CD pipelines
- Code reviews
- Automated testing
- Participate in architecture reviews, technical design sessions, and stakeholder meetings.
- Collaborate across engineering teams and contribute to continuous improvement initiatives.
Required Qualifications
- Bachelor's degree in Computer Science, Information Systems, Software Engineering, or a related field (or equivalent experience).
- 4+ years of enterprise Data Engineering experience.
- Strong SQL and Python programming skills.
- Hands-on expertise with:
- Databricks
- Delta Lake
- Unity Catalog
- PySpark
- Spark SQL
- Snowflake
- Microsoft Fabric
- OneLake
- Experience developing Power BI reports and dashboards.
- Strong understanding of:
- Data Modeling
- Lakehouse Architecture
- Dimensional Modeling
- Data Quality Frameworks
- Data Pipeline Testing
- Experience with Git, CI/CD pipelines, and DevOps best practices.
- Excellent communication, collaboration, and stakeholder management skills.
Preferred Qualifications
- Experience with Apache Iceberg or other modern table formats.
- Experience supporting:
- HR Analytics
- Workforce Analytics
- People Analytics
- Experience integrating with HR platforms such as:
- Workday
- ServiceNow HR
- Familiarity with:
- Apache Kafka
- Azure Event Hub
- Delta Live Tables
- Spark Structured Streaming
- Experience supporting AI/ML data platforms.
- Working knowledge of:
- Teradata
- Oracle
- SQL Server
- Microsoft Azure certifications are highly desirable.
- Understanding of:
- Enterprise Lakehouse Architecture
- Data Governance
- Metadata Management
- GDPR
- CCPA
- Employee Data Privacy and Compliance
