• Define and own a cloud-native, enterprise-scale data architecture that supports batch, streaming, analytics, and AI/ML workloads.

• Develop and execute a multi-year data platform modernization roadmap, balancing delivery speed, risk, cost efficiency, and business continuity.

• Drive architectural decisions across Snowflake, Databricks, and cloud-native services, ensuring scalability, security, and performance.

Platform & Engineering Leadership

• Partner closely with data engineering, analytics, infrastructure, security, and data science teams to design scalable ingestion, transformation, and serving layers.

• Define standards and best practices for data ingestion, transformation, integration, and data modeling across diverse structured and semi-structured sources.

• Establish robust data governance, quality, lineage, and access control frameworks aligned with enterprise and regulatory needs.

• Optimize performance and cost efficiency across Snowflake and Databricks environments, including workload isolation, query optimization, and storage strategies.

DevOps, Reliability & Security

• Lead the adoption of DevOps, CI/CD, and Infrastructure as Code practices (e.g., Terraform) for data platforms.

• Define patterns for environment management, observability, disaster recovery, and security-by-design.

• Ensure the platform meets enterprise standards for data privacy, compliance, and resilience.

AI / ML & Future Readiness

• Design the data foundation required for advanced analytics, ML, and LLM workloads, including feature stores, vector storage, and real-time data access.

• Stay ahead of emerging trends in AI, ML, and data platforms, translating innovation into practical, production-ready architectures.

• Influence the evolution toward lakehouse, data mesh, and data product–oriented architectures where appropriate.

Leadership & Influence

• Provide technical leadership and mentorship to senior data engineers and architects.

• Act as a trusted advisor to engineering leadership and business stakeholders.

• Clearly communicate complex architectural concepts to both technical and non-technical audiences.

Required Qualifications

• 10+ years of experience in data architecture, data engineering, or platform engineering roles.

• Deep hands-on expertise with SQL, Spark, and Python, and strong understanding of distributed data processing.

• Strong experience designing and operating data platforms on AWS, Azure, or GCP, including compute, storage, networking, and security.

• Solid background in data governance, metadata management, lineage, and cataloging (e.g., Purview, Collibra, Alation).

• Experience with real-time and streaming architectures (Kafka, Kinesis, Pub/Sub) and orchestration tools (Airflow, dbt).

• Demonstrated ability to lead cross-functional teams and complex migration programs.

• Exceptional communication skills and executive-level presence.

Nice to Have

• Snowflake, Databricks, or cloud provider certifications (AWS, Azure, GCP).

• Hands-on experience building ML and AI pipelines on Databricks or similar platforms.

• Experience with multi-petabyte-scale data platforms.

• Familiarity with data mesh, lakehouse, and domain-oriented data product concepts.

• Exposure to LLM pipelines, vector databases, or retrieval-augmented generation (RAG) architectures.

What we offer:

• Comprehensive medical, dental, vision, disability, life insurance

• Health Savings Account (HSA), Flexible Spending Account (FSAs) and Commuter benefits

• Voluntary supplemental health coverage and life insurance

• 401K match and ESPP

• Paid time off and paid sick leave

• Paid parental and pregnancy leave

• Family-forming benefits (IVF, Preservation, Adoption etc.)

• Emergency backup care (Child/Adult/Pets)

• Employee Assistance Program (EAP) with counseling sessions available 24/7

• Free legal services that provide legal advice, document creation and estate planning

• Employee bonus referral program

• Student loan refinancing assistance

• Employee 1:1 coaching, perks and discounts program

If you are hired in Belmont, California, the compensation range for this position is between $184,600 and $225,000 for full-time employees, in addition to eligibility for variable pay, equity, and benefits. Benefits may include, but are not limited to, health and wellness, 401k, ESPP, vacation, parental leave, and more! The salary may vary depending on your location, skills, and experience.