Overview:Job Title: Senior Data Engineer Location: Irvine, CA
Job Type: Full-Time
Job SummaryWe are seeking a highly skilled
Senior Data Engineer with deep expertise in Databricks, Apache Spark, Airflow/Astronomer, and cloud-native data platforms. The ideal candidate will be responsible for designing, building, and supporting scalable data engineering solutions while ensuring platform reliability, performance, automation, and operational excellence.
This role requires hands-on engineering experience, strong DevOps capabilities, and the ability to independently own technical delivery in a highly collaborative engineering environment.
Required Skills- Strong expertise in:
- Databricks
- Apache Spark
- Data Engineering
- Platform Engineering
- Hands-on experience with:
- Airflow
- Astronomer
- DAG Design & Orchestration
- Strong knowledge of:
- Data Pipeline Architecture
- Data Quality Frameworks
- Scalability & Performance Optimization
- Experience with:
- CI/CD Pipelines
- Infrastructure as Code (IaC)
- Automation Frameworks
- DevOps Best Practices
- Strong platform administration experience including:
- Monitoring
- Reliability Engineering
- Operational Readiness
- Incident Management
- Experience working with cloud-based data platforms
- Strong problem-solving and troubleshooting skills
Key Responsibilities- Design, develop, and maintain scalable data pipelines and data processing frameworks
- Build and optimize Databricks-based data engineering solutions
- Perform Spark performance tuning and platform optimization
- Develop and manage Airflow/Astronomer workflows and enterprise orchestration solutions
- Implement data quality, monitoring, and reliability frameworks
- Design and maintain CI/CD pipelines for data platform deployments
- Automate infrastructure provisioning and operational processes using Infrastructure as Code
- Ensure platform stability, availability, and operational excellence
- Collaborate with architects, data engineers, DevOps teams, and business stakeholders
- Troubleshoot and resolve complex platform and data engineering issues
- Drive engineering best practices, standards, and continuous improvement initiatives
Preferred Qualifications- Experience supporting enterprise-scale data platforms
- Knowledge of cloud-native architecture and distributed systems
- Exposure to DataOps and Platform Engineering practices
- Strong communication and stakeholder management skills
