Site Reliability Engineering Lead

Join Graphcore, a leading AI supercomputing company, as a Site Reliability Engineering Lead. In this role, you will build and lead a new SRE organization responsible for the production operation of a rapidly scaling AI supercomputing platform. You will define the operating model, establish production readiness and incident-management practices, and ensure reliability and operability are engineered into the platform from the outset. This is a unique opportunity to establish the reliability function for a new platform from the ground up.

Establish the reliability function for a new AI supercomputing platform from the ground up, including defining the operating model and establishing production readiness.

Build and develop the Site Reliability Engineering (SRE) team, including defining roles, interviewing and hiring team members, and establishing career expectations.

Responsibilities

• Lead the development of Service Level Objectives (SLOs), operational health indicators, alerting standards, and reliability reporting appropriate for a large-scale production infrastructure service.
• Significant experience leading or building an SRE, Production Engineering, Infrastructure Reliability, or comparable function supporting large-scale, highly available production infrastructure
• Experience building and operating sustainable 24x7x365 production support or on-call organizations
• Experience taking a new or rapidly evolving platform through production readiness, launch, stabilization, and ongoing operation
• Demonstrated ability to lead, hire, mentor, develop, and retain a growing team of strong technical engineers at various levels of capability

Requirements

• Strong judgment regarding when to solve an immediate operational problem and when to invest in eliminating the underlying failure mode
• Strong written and verbal communication skills, particularly during incidents and when communicating technical risk to senior leadership
• A strong systems engineering background with a working understanding of Linux, networking, storage, distributed systems, automation, and production infrastructure
• Ability to operate effectively at both the leadership and hands-on technical levels, moving comfortably between organizational design, architecture discussions, troubleshooting, and incident response
• Ability to manage conflicting priorities under a pressured environment
• Strong incident leadership experience, including managing high-severity, multi-team production incidents under significant time pressure
• Strong understanding of modern Site Reliability Engineering principles, including SLOs, incident management, observability, automation, toil reduction, capacity management, and production readiness
• Candidates are not expected to have experience in all the areas below. Experience in several would be particularly valuable:
• High-performance networking technologies such as InfiniBand, RDMA, RoCE, or large-scale Ethernet fabrics

Large-scale parallel or distributed storage systems

• Workload schedulers and orchestration platforms such as Kubernetes, Slurm, or comparable systems
• Production observability and service-level monitoring for complex distributed infrastructure
• Operational relationships and escalation processes involving datacenter operations and infrastructure/hardware/networking vendors
• Custom or early-generation hardware, firmware, or environments where hardware and software are developed concurrently
• Rapid capacity expansion, datacenter migration, or transitions between temporary and permanent production environments