Business Area: Engineering
Seniority Level:Mid-Senior level
Job Description:We are seeking a
Staff Software Engineer with deep expertise in data infrastructure and a
strong focus on Apache Airflow to lead the orchestration and workflow engineering experience within Cloudera Data Engineering (CDE).
In this high-impact, Individual Contributor (IC) role, you will be a key technical driver shaping how thousands of enterprise customers worldwide design, schedule, and monitor complex, multi-tenant data pipelines. You will help execute the architectural vision for scaling Airflow natively within Kubernetes across multi-cloud and on-premises environments, ensuring it seamlessly integrates with Spark and modern Lakehouse (Iceberg) architectures.
As a Staff Software Engineer you will:- Drive the multi-year technical roadmap and architectural vision for enterprise-grade Apache Airflow within Cloudera Data Engineering.
- Design and optimize highly available, secure, and multi-tenant Airflow environments capable of managing complex dependencies and scheduling workflows across thousands of distributed cluster nodes.
- Gain and apply deep technical knowledge across the CDE stack, with a primary focus on extending Airflow, integrating it with Spark and Iceberg, and contributing back to the open-source community where applicable.
- Foster a culture of technical excellence through hands-on technical mentorship, rigorous design reviews, and robust architectural guidance for distributed workflow execution.
- Collaborate closely with product management, engineering leaders, and cross-functional partners to deliver critical features that make workflow automation seamless and intuitive for CDE customers.
We are excited if you have (Required Experience):- BS or MS in Computer Science or a related technical field.
- 10+ years of professional software engineering experience, with a heavy emphasis on data infrastructure, workflow orchestration, and distributed systems.
- Deep, production-proven experience architecting, scaling, and extending Apache Airflow (e.g., custom operators, hooks, providers, Celery/Kubernetes executors, and security models).
- Strong understanding of Python (essential for Airflow development) and at least one compiled language such as Java, Scala, C++, or Go.
- Deep expertise in cloud-native architectures, specifically running stateful and stateless workloads on Kubernetes (EKS, AKS, GKE) and/or private cloud environments (OpenShift, Rancher).
- Strong understanding of large-scale systems design, scheduling algorithms, resource allocation, and performance tuning.
- Exceptional communication skills, an open-minded attitude, and a passion for clean code, technical quality, and mentorship.
Why this role matters:This is your opportunity to build cloud-native solutions that are deployable anywhere whether in massive clusters on any cloud provider or in private data centers. You'll work with cutting-edge technologies like Trino, Spark, Airflow, and advanced AI inferencing systems to shape the future of analytics. Your code will directly influence how data engineers, analysts, and developers worldwide find value in their data.
We believe in the power of open source. You'll collaborate with project committers, contributing upstream to keep technologies like Apache Hive and Impala evolving. You'll harden these engines for rock-solid security, optimize them for peak performance, and make them effortlessly run across all environments. Join us and help build the trusted, cloud-native platform that powers insights for the most data-intensive companies on the planet.
This role is not eligible for immigration sponsorship or relocation.What you can expect from us:- Generous PTO Policy
- Support work life balance with Unplugged Days
- Flexible WFH Policy
- Mental & Physical Wellness programs
- Phone and Internet Reimbursement program
- Access to Continued Career Development
- Comprehensive Benefits and Competitive Packages
- Paid Volunteer Time
- Employee Resource Groups
#LI-SZ1
#LI-REMOTE
