Your role and responsibilities
About the Role:
We are a team of passionate engineers who love solving complex distributed storage and query engine problems. Through the Tableflow and Lightning Tables project, we are building the next generation of storage and query solutions for Kora to enable streaming data to be consumed both as a table and as a stream seamlessly, and queried with the performance of a purpose-built analytical engine. As a Staff Software Engineer on the team, you will focus on advancing key layers of our system, including multi-tenant, highly available compute infrastructure, the core components responsible for table materialization and background maintenance, and the query engine that serves analytical workloads over that data. You will tackle challenges such as developing strategies for efficient table materialization and maintenance at massive scale across all of Confluent Cloud, building a fast and scalable query engine, and ensuring high availability and scalability for background storage tasks and query serving.
What You Will Do:
- Storage Engine Development: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale.
- Query Engine Development: Design and implement components for query planning, execution, and optimization for fast analytical queries.
- Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers background computational tasks for Kora's long-term storage and query serving. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives.
- Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage and query features and infrastructure with the broader organization.
Required education
Bachelor's Degree
Preferred education
Master's Degree
Required technical and professional expertise
What You Will Bring:
- BS, MS, or PhD in computer science or a related field, or equivalent work experience
- Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.
Preferred technical and professional experience
What Gives You an Edge:
- Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss
- Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark SQL, DuckDB, ClickHouse)
- Experience/knowledge with public clouds (AWS, Azure or GCP)
OTHER RELEVANT JOB DETAILS
IBM offers a competitive and comprehensive benefits program. Eligible employees may have access to:
- Healthcare benefits including medical & prescription drug coverage, dental, vision, and mental health & well being
- Financial programs such as 401(k), cash balance pension plan, the IBM Employee Stock Purchase Plan, financial counseling, life insurance, short & long- term disability coverage, and opportunities for performance based salary incentive programs
- Generous paid time off including 12 holidays, minimum 56 hours sick time, 120 hours vacation, 12 weeks parental bonding leave in accordance with IBM Policy, and other Paid Care Leave programs. IBM also offers paid family leave benefits to eligible employees where required by applicable law
- Training and educational resources on our personalized, AI-driven learning platform where IBMers can grow skills and obtain industry-recognized certifications to achieve their career goals
- Diverse and inclusive employee resource groups, giving & volunteer opportunities, and discounts on retail products, services & experiences
We consider qualified applicants with criminal histories, consistent with applicable law.
This position was posted on the date cited in the key job details section and is anticipated to remain posted for 21 days from this date or less if not needed to fill the role.
IBM will not be providing visa sponsorship for this position now or in the future. Therefore, in order to be considered for this position, you must have the ability to work without a need for current or future visa sponsorship.
The compensation range and benefits for this position are based on a full-time schedule for a full calendar year. The salary will vary depending on your job-related skills, experience and location. Pay increment and frequency of pay will be in accordance with employment classification and applicable laws. For part time roles, your compensation and benefits will be adjusted to reflect your hours. Benefits may be pro-rated for those who start working during the calendar year.
