Qualifications

• Experience with Apache Iceberg, Spark, Flink, Kafka, dbt, Trino, Snowflake, and Databricks.
• Familiarity with cloud platforms: AWS, Azure, or Google Cloud.
• Proficiency in programming languages: Python, Scala, SQL, and Java.
• Experience with containerization and orchestration tools: Kubernetes, Docker, and Terraform.
• Knowledge of workflow orchestration tools like Airflow, Dagster, or Prefect.
• Exposure to data cataloging systems: Polaris, Unity Catalog, Glue Catalog, and Apache Nessie.

Responsibilities

• Define the architecture and standards for enterprise lakehouses.
• Design and implement data layers based on Apache Iceberg.
• Develop frameworks for batch, streaming, and event-driven data ingestion.
• Build and optimize data pipelines using Spark, Flink, Kafka, and Python.
• Manage and implement data quality, metadata management, and governance processes.
• Optimize system performance through effective partitioning and compaction.
• Enable AI functionalities by developing semantic layers and GenAI workloads.

Benefits

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage including Dental & Vision.
• Family Support through Maternal & Parental Leaves.
• Insurance Options including Home, Auto, and Identity Theft Protection.
• Professional Growth benefits including Certification and Training Reimbursement.
• Paid Time Off including Sick Leave and Holidays.
• Legal and Financial Assistance programs including Student Loan Refinancing.