Description & Requirements As Lead Infrastructure Engineer, you will own the GPU fleet our researchers train on including capacity, scheduling, diagnostics, and support. You will set technical direction for GPU operations and infrastructure architecture. You will additionally lead Infrastructure as Code setup, granting permissions, and debugging infrastructure problems.
This is a hybrid role, working three days per week in Redwood City, Montreal, or Vancouver.
You will report to the Head of Data and Infrastructure.
Responsibilities:
- You will own GPU fleet operations across our AWS estate.
- You will build the scheduling layer from zero.
- You will diagnose GPU and node failures fast and completely and drive hardware evidence and replacement through AWS support and capacity-block channels.
- You will run researcher support as a first-class product including holding office hours, owning the support channel, and driving the recurring causes out of existence with self-service tooling, preflight checks, and documentation
- You will instrument the fleet including utilization, queue depth, job success rate, and cost per experiment metrics.
- You will partner with our external compute and lab partnerships as a technical contact, and with EA's central infrastructure groups on shared services and escalation.
- You will author runbooks, decision records, and onboarding docs.
Qualifications:
- 8+ years of experience operating production infrastructure, with deep, current, hands-on AWS depth - EC2 GPU fleets, EKS, IAM and cross-account security, VPC and networking, S3 and FSx
- Experience scheduling, diagnosing, and managing GPUs in AWS specifically
- Experience operating GPU fleets at 1000+ GPU scale
- Expertise in scripting and automation with Python, PowerShell, bash, or equivalent
- Expertise in infrastructure as code (Terraform or equivalent)
- Familiarity with a GPU scheduling or orchestration layer (like Slurm, Kubernetes with Kueue or Volcano, Ray, dStack or SkyPilot)
- Observability practice including Grafana, Prometheus, or equivalent
We want to connect you with job opportunities that align with your interests, skills, and expertise. When you create an EA Careers Account and are logged into the portal, you can click "Get Recommendations" to view a curated list of job openings. These recommendations are enhanced by automated processing, including artificial intelligence, and take into account your skills and experience. However, all employment decisions are made by our hiring teams, not by automated systems.
Pay Transparency - North AmericaCOMPENSATION AND BENEFITS The ranges listed below are what EA in good faith expects to pay applicants for this role in these locations at the time of this posting. If you reside in a different location, a recruiter will advise on the applicable range and benefits. Pay offered will be determined based on a number of relevant business and candidate factors (e.g. education, qualifications, certifications, experience, skills, geographic location, or business needs).
PAY RANGES* British Columbia (depending on location e.g. Vancouver vs. Victoria) *$169,500 - $242,600 CAD
* California (depending on location e.g. Los Angeles vs. San Francisco) *$193,100 - $296,500 USD
Pay is just one part of the overall compensation at EA.
In the US, we offer a package of benefits including paid time off (3 weeks per year to start), 80 hours per year of sick time, 16 paid company holidays per year, 10 weeks paid time off to bond with baby, medical/dental/vision insurance, life insurance, disability insurance, and 401(k) to regular full-time employees. Certain roles may also be eligible for bonus and other incentive programs.
For Canada, we offer a package of benefits including vacation (3 weeks per year to start), 10 days per year of sick time, paid top-up to EI/QPIP benefits up to 100% of base salary when you welcome a new child (12 weeks for maternity, and 4 weeks for parental/adoption leave), extended health/dental/vision coverage, life insurance, disability insurance, retirement plan to regular full-time employees. Certain roles may also be eligible for bonus and other incentive programs.
Post To
External careers site, Internal careers site
LinkedInID
1449
