About this RoleCrusoe Cloud is seeking a Senior Staff Network Operations Engineer to own production reliability across our global network, including edge, backbone, data center fabric, and GPU cluster interconnects. You will drive incident response, root cause analysis, and the operational excellence initiatives that keep our hyperscale AI infrastructure healthy at scale.
This is a senior production ownership role, not architecture, not pre-sales, not purely automation. You will set operational standards, define SLIs and SLOs, mentor Staff and Senior engineers, and serve as the senior escalation point during high-severity events. This is the role that keeps the network up.
What You'll Be Working On- Own Production Reliability: Serve as the senior technical owner for uptime of Crusoe's global edge, backbone, data center, and GPU cluster network, directly affecting the availability of AI workloads running on hundreds of thousands of GPUs.
- Lead Incident Response: Own end-to-end response for high-severity network events, including rapid mitigation, stakeholder communication, and postmortem documentation that prevents recurrence.
- Drive Root Cause Analysis: Lead RCAs for production incidents, identify systemic issues, author remediation plans, and track them to closure.
- Define SLIs and SLOs: Partner with Architecture and Site Reliability to define network reliability metrics and service level objectives, backed by real-time dashboards and alerting.
- Set Operational Standards: Author and maintain runbooks, escalation playbooks, and SOPs used by the broader operations team.
- Improve Observability: Drive continuous improvement of Crusoe's network monitoring stack including streaming telemetry, SNMP, NetFlow, and tools such as Kentik, Grafana, Prometheus, and ThousandEyes.
- Build Operational Automation: Write Python-based auto-remediation tooling that reduces toil and accelerates mean time to resolution for known failure modes.
- Mentor and Multiply: Provide technical guidance to Staff and Senior engineers. Drive post-incident learning and build a culture of operational excellence across the team.
What You'll Bring to the Team- 12+ years of production network engineering experience with a demonstrated focus on large-scale operations, incident response, and reliability in hyperscale or internet-scale environments.
- Observability and Monitoring: Hands-on experience with streaming telemetry, SNMP, NetFlow, sFlow, and tools such as Kentik, Grafana, Prometheus, ThousandEyes, and Arbor.
- GPU Cluster and RDMA Networking: Hands-on experience operating RDMA/RoCE (v1 and v2) lossless fabrics for GPU and HPC workloads, including PFC, ECN, and DCQCN tuning. Required at this level.
- Demonstrated Technical Leadership: Proven track record owning production reliability at scale, leading RCAs that drove systemic change, and setting operational standards the broader org executes against.
- Hyperscale Operational Depth: Comfort operating 10K+ device fleets across multi-region environments with 24/7 on-call responsibility. You have been the senior escalation point during critical network events.
- Protocol Fluency: Expert hands-on knowledge of BGP, EVPN-VXLAN, IS-IS, OSPF, MPLS, QoS, and TCP/IP across production DC fabric environments at scale.
- Hardware Platform Depth: Expert knowledge of Arista (EOS), Juniper (Junos), and NVIDIA/Mellanox platforms in leaf-spine CLOS architectures across multi-vendor environments.
- Operational Automation: Proficiency in Python for auto-remediation scripts, diagnostic tooling, and operational workflows that reduce toil and accelerate incident resolution.
- SLI and SLO Ownership: Experience defining and owning network reliability metrics and service level objectives in partnership with engineering and product leadership.
- Education: Bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience in hyperscale or internet-scale environments.
Benefits:- Competitive compensation
- Restricted Stock Units
- Paid time off & paid holidays
- Comprehensive health, dental & vision insurance
- Employer contributions to HSA account
- Paid parental leave
- Paid life insurance, short-term and long-term disability
- Professional development & tuition reimbursement
- Mental health & wellness support
- Commuter benefits (parking & transit)
- Cell phone stipend
- 401(k) Retirement plan with company match up to 4% of salary
- Volunteer time off
Compensation:Compensation will be paid in the range of $225,000 - $275,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicant's education, experience, knowledge, skills, and abilities, as well as internal equity and alignment with market data.
