The search

Our client is seeking multiple technical leaders to contribute at both the Staff and Lead Engineer level. These are high-visibility, high-impact opportunities with a talent-dense, homegrown, bootstrapped, globally competitive Canadian tech success story. The company finds itself squarely in the middle of the disruption of search by AI/LLMs, the fragmentation of the media landscape and in a chapter of rapid, transformational growth.

The shared mandate is to modernize how massive volumes of data are processed, modeled, and made available to products and customers. The work spans petabyte-scale batch and streaming pipelines, low-latency APIs, analytical platforms, and the architecture that connects raw event data to reliable, production-grade data products.

Staff Engineer candidates may bring a stronger centre of gravity in backend and API systems or in data processing and platform engineering. For the Lead Engineer search, the client is seeking a hands-on technical and people leader with the affinity and aptitude to guide a distributed team through transformation.

Role overviews

In our client's context, both Staff Engineers and Leads remain hands-on. This is an intentional and informed cultural practice tailored to the challenge ahead of them. Leads are expected to understand the systems deeply, make informed architectural decisions, and help their teams execute.

As a Staff Engineer, you will be a primary owner and technical leader for data-intensive systems spanning processing, platform, and product-facing services. You will drive cross-team architecture, define reusable platform patterns, lead complex multi-system initiatives, and influence the long-term evolution of data processing and access across the organization. Depending on your background, your work may lean toward one or both of the following domains:

Data products and APIs: This team builds and operates high-scale services exposing aggregated and near real-time data for reporting, analytics, and customer-facing products; shaping API contracts, query abstraction and execution, caching, response design, performance, scalability, and reliability. This team operates as a full-stack data engineering group, owning the path from data generation and aggregation through query execution, APIs, and user-facing delivery. On this team you can expect to:

• Design high-scale APIs for multidimensional reporting and analytical workloads
• Define contracts for metrics, dimensions, filters, schemas, and aggregations
• Improve dynamic query generation, caching, response shaping, and execution performance
• Connect APIs to batch, streaming, and near real-time data sources
• Build primarily in Go or a comparable backend language such as Java
• Improve observability, incident response, reliability, and consumer experience
• Lead initiatives involving Product, Data Platform, ML, and Application Engineering

Data processing and platform: This team builds and evolves large-scale batch and streaming pipelines that transform raw event data into clean, canonical, business-ready datasets; improving data models, workflow orchestration, correctness, observability, runtime, and compute efficiency. On this team you can expect to:

• Build large-scale Spark pipelines and streaming systems using Kafka and Flink
• Transform raw event data into canonical, production-grade datasets
• Design partitioning, storage, aggregation, and query strategies for enormous datasets
• Support historical backfills, incremental processing, and real-time availability
• Help decompose centralized processing into domain-oriented streams and workloads
• Move appropriate services and processing patterns from Hadoop toward Kubernetes
• Improve data quality, correctness, observability, throughput, and compute efficiency
• Participate in the long-term design of globally distributed data infrastructure

As a Lead Engineer you will lead a distributed team of data and software engineers responsible for high-scale, low-latency data processing pipelines, analytical platforms, and reporting products. In practice this means:

• Set the technical direction for the team's products, systems, and architectures while remaining comfortable working directly with the code and technology
• Hire, mentor, and develop engineers, creating an environment of accountability, collaboration, and meaningful technical growth
• Partner with Product to establish roadmaps, goals, forecasts, and measurable technical and business outcomes
• Manage scope and sequencing across more opportunities than available resources, communicating progress and risks early and coordinating dependencies across teams
• Own the quality, health, and outcomes of the team's platforms, including operational response, technical debt, reliability, and continuous improvement

The shared mandate

• Platform architecture: Define and evolve patterns across edge, transport, compute, storage, modeling, query execution, and serving
• Batch and streaming systems: Build systems supporting historical processing, backfills, incremental updates, and near real-time availability without creating unnecessary architectural complexity
• Data products and interfaces: Provide stable, flexible access to metrics, dimensions, filters, aggregations, and canonical datasets through well-designed APIs and analytical platforms
• Performance and economics: Optimize query planning, data layout, partitioning, caching, execution, throughput, and infrastructure use so capacity does not need to grow linearly with data volume
• Correctness and trust: Design for deduplication, late-arriving data, schema evolution, data contracts, reconciliation, and the integrity of business-critical reporting and billing datasets
• Reliability and observability: Own monitoring, alerting, incident response, root-cause analysis, workflow health, and the mechanisms that make system behaviour visible
• Cross-functional influence: Work across Data Engineering, Data Systems, Application Engineering, Product, analytics, machine learning, and experimentation teams to turn platform capabilities into business outcomes

Tech stack

• Processing and streaming: Spark, Scala, Kafka, Flink, Airflow
• Data services and APIs: Go, Java, REST, gRPC, SQL
• Compute and storage: Kubernetes, Hadoop/HDFS, Ceph
• Query and analytics: Trino, Vertica, Druid and StarRocks
• Reporting and BI: Looker, Superset, Redash

The precise mix will vary by team and assignment. Depth in comparable technologies and the ability to reason from first principles matter more than matching every tool.

Key responsibilities

• Vision and strategic direction: Identify high-leverage technical opportunities and translate them into a coherent direction for the platform and business
• System design and delivery: Lead the design and implementation of distributed data systems operating under heavy workloads and demanding latency requirements
• Data platform evolution: Help move the organization from centralized, batch-oriented systems toward workload-aware architectures combining batch, streaming, and asynchronous processing
• API and query architecture: Define durable interfaces, schemas, query abstractions, and serving patterns for flexible access to large-scale multidimensional data
• Pipeline and model ownership: Build canonical datasets and processing workflows supporting reporting, billing, products, analytics, experimentation, and machine-learning use cases
• Operational excellence: Improve observability, service health, incident response, workflow reliability, technical debt management, and post-incident learning
• Organizational leadership: Coordinate across teams, unblock dependencies, communicate trade-offs clearly, and build alignment around technical and product outcomes

Your know-how

• You have significant experience designing, building, and operating data-intensive systems at scale
• You have a deep understanding of distributed-systems fundamentals, including partitioning, consistency, failure modes, state, throughput, latency, and cost trade-offs
• You have experience with large-scale data processing and modeling (the client leans on Spark currently)
• You have experience with streaming technologies such as Kafka and Flink, including incremental processing, late data, deduplication, and replay
• You have experience designing stable data contracts, schemas, canonical datasets, and transformation layers for business-critical use cases
• You have experience with large-scale storage, data warehouse, or analytical query systems and a strong grasp of query performance, data layout, and workload-aware optimization
• You have experience with workflow orchestration and the production deployment or operation of services on Kubernetes
• You have a track record of end-to-end ownership: navigating ambiguity, debugging complex cross-system issues, making sound trade-offs, and improving systems after they enter production
• You have an excellent command of English and experience collaborating with neighbouring engineering disciplines and stakeholders beyond R&D

For API-oriented work, you will call backend engineering skills (ideally but not necessarily with Go) and experience with REST/gRPC design, versioning, performance, and high-concurrency services.

Additional know-how for the Lead Engineer roles

• You have 3+ years of experience leading and managing distributed teams
• You have an affinity for mentorship and creating an environment that nurtures a balance of innovation and accountability
• You have experience partnering with Product Management and internal stakeholders to shape roadmaps, forecasts, deliverables, timelines, and measurable outcomes
• You are experienced making tradeoffs and experience exercising judgment to prioritize a broad portfolio when initiatives outnumber resources, including thoughtful sequencing and proactive communication of progress, delays, and risk