About the RoleWe are seeking a
Staff / Principal Data Engineer & Platform Architect to define the technical vision, architectural strategy, and technical execution for our enterprise-wide, next-generation event-driven data platforms. In this role, you will act as the principal authority bridging ultra-high-throughput streaming topologies, stateful distributed workflow orchestration, and federated data governance.
You will spearhead our platform architecture centered around
Apache Kafka for real-time event distribution and
Temporal for durable execution and distributed state orchestration. A core mandate of this position is establishing enterprise data contracts, driving schema evolution frameworks, and designing resilient Lakehouse/warehouse architectures capable of serving high-concurrency analytical and operational workloads. As a technical leader, you will set the engineering standards, mentor senior talent, and align multi-team distributed systems initiatives with executive business priorities.Key Responsibilities
- Architecture & Strategy: Define the multi-year technical roadmap and reference architecture for our real-time streaming infrastructure, Lakehouse ingestion engines, and distributed orchestration layers.
- Distributed Stream Topologies: Architect and govern multi-tenant, enterprise Apache Kafka ecosystems (producers, consumers, Kafka Connect, Schema Registry), ensuring sub-second latencies, exactly-once processing guarantees, and disaster recovery.
- Data Contracts & Governance: Formulate and enforce company-wide data governance, schema design patterns, and automated contract testing (Avro, Protobuf, JSON Schema) across polyglot microservices, streaming pipelines, and storage boundaries.
- Durable Orchestration & Transactional Integrity: Architect long-running, fault-tolerant workflow topologies using Temporal, standardizing complex distributed transaction patterns (e.g., Saga patterns, compensation logic, distributed state machines, human-in-the-loop flows).
- Lakehouse & Analytical Engine Architecture: Direct the design and operational scaling of large-scale distributed computing workloads (Apache Spark / PySpark) and modern cloud Lakehouse platforms (Snowflake, Databricks, BigQuery) using medallion architectures and dimensional/semantic models.
- Operational Excellence & FinOps: Implement comprehensive observability platforms (tracing, telemetry, automated drift detection, data quality gates) while establishing cloud infrastructure cost optimization and capacity planning frameworks.
- Technical Leadership & Mentorship: Act as the technical authority across platform and product teams; establish engineering excellence standards, drive RFC and Architecture Review Board (ARB) processes, and elevate senior engineering talent across the organization.
Required Experience- 8+ years of professional experience in data engineering, distributed systems, or backend platform architecture, with 2+ years acting in a Lead, Staff, or Principal/Architect capacity.
- Durable Workflow & State Machine Architecture: Deep architectural experience with Temporal (or Cadence), including deterministic execution modeling, activity orchestration, workflow versioning strategies, signal/query handling, dynamic task routing, and cross-boundary error recovery.
- Mastery of Streaming Systems: Deep foundational expertise with Apache Kafka internals (broker sizing, partition strategies, consensus mechanisms, custom connector development, consumer group rebalancing, and stateful stream processing via Kafka Streams or Flink).
- Enterprise Data Contracts & Schema Engineering:
- Authority in schema specification frameworks (Apache Avro, Protocol Buffers/gRPC, JSON Schema).
- Hands-on leadership implementing schema registries (Confluent, AWS Glue) with automated CI/CD schema linting, breaking-change prevention, and backward/forward compatibility strategies.
- Experience standardizing automated data contract verification tooling (e.g., Great Expectations, Pandera, Pydantic, dbt tests, contract-testing frameworks).
- Distributed Computing & Core Engineering: Advanced engineering capabilities in Python, Go, or Java/Scala, emphasizing distributed design patterns, concurrency models, and resilient microservice design. Extensive scale experience with Apache Spark / PySpark internals and query plan optimization.
- Modern Lakehouse & Data Modeling: Expert-level knowledge of dimensional data modeling, Data Vault, or semantic layer design, with practical experience scaling multi-terabyte/petabyte systems on Snowflake, Databricks, or BigQuery.
- Cross-Functional Influence: Demonstrated track record of aligning distributed engineering teams around unified architectural patterns, authoring architectural decision records (ADRs), and influencing executive stakeholders.
Infinitive is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $90,000 - $154,00.00.
