8+ years of software engineering experience, including 4+ years in observability/monitoring platforms at scale.
Deep expertise in open-source observability tools like Prometheus, Elasticsearch, and Loki.
Experience with OpenTelemetry for diverse service environments.
Proficiency in Go and strong experience in Python for building performant systems.
Strong knowledge of distributed systems and Kubernetes, including sharding and replication.
Experience in observability cost management and capacity planning at scale.
Proven ability to set technical direction and drive cross-team adoption.
Responsibilities
Own the architecture and roadmap for the observability platform across metrics, logs, traces, and events.
Design telemetry pipelines for high-frequency data from Wafer-Scale Engines and Kubernetes clusters.
Partner with teams to define SLOs, build alerting strategies, and enhance root cause analysis tools.
Integrate critical hardware health signals into a unified observability layer with software metrics.
Implement instrumentation libraries and standards to ensure rich observability is standard across services.
Mentor senior engineers and influence engineering practices to foster operational excellence.
Benefits
Opportunity to work at the forefront of AI advancements.
Collaborative environment with cross-functional teams.
Focus on professional growth and mentorship.
Access to cutting-edge technology and tools.
Culture that values operational excellence and innovation.
Full Job Description
Responsibilities
Set Technical Direction: Own the long-term architecture and roadmap for the observability platform across all pillars (metrics, logs, traces, and events). Drive adoption across the engineering organization through RFCs, design reviews, and close partnership with stakeholders.
Build for Cerebras-Scale: Design telemetry pipelines that handle high-cardinality, high-frequency data from thousands of Wafer-Scale Engines, large Kubernetes clusters, and the storage and networking fabric connecting them. Solve ingestion and query performance problems that break conventional approaches.
Drive Reliability Across the Organization: Partner with Engineering, SRE, Hardware, and ML Infrastructure teams to define SLOs, build alerting strategies, and create tooling that accelerates root cause analysis and reduces MTTR. Make it easy for any engineer to understand system health and diagnose failures.
Bridge Hardware and Software Observability: Work with engineering teams accross the stack to surface critical hardware health signals into a unified observability layer alongside software metrics.
Shape Developer Experience: Design and implement instrumentation libraries and standards (OpenTelemetry) that make rich observability the default for every service, not an afterthought. Establish conventions that scale across teams.
Mentor and Grow Engineers: Raise the technical bar across the organization. Mentor senior engineers, influence engineering practices across team boundaries, and foster a culture of operational excellence.
Qualifications
8+ years of software engineering experience, with 4+ years building or operating observability/monitoring platforms at significant scale (millions of active time series, petabytes of log data).
Deep expertise in the open-source observability ecosystem. You have operated, tuned, and extended systems like Prometheus, Thanos/Cortex/Mimir, Elasticsearch/ClickHouse, or Loki and understand their internals, failure modes, and cost characteristics.
Experience with OpenTelemetry for instrumentation across a polyglot services environment.
Proficiency in Go preferred, with strong experience in Python. You write performant, concurrent systems code, not just glue scripts.
Strong distributed systems and Kubernetes expertise: sharding, replication, consistency models, monitoring K8s itself and the workloads running on it, and the tradeoffs involved in building reliable data pipelines at scale.
Experience with observability cost management and capacity planning at scale.
Track record of setting technical direction and driving adoption across multiple teams. You operate effectively with ambiguity and can translate business needs into technical strategy.
Find out more about what it's like to work at Cerebras here!
Apply today and become part of the forefront of groundbreaking advancements in AI!
