Staff Site Reliability Engineer
- Menlo Park, United States
- Lead/Staff
- 10+ years
- Bachelor's degree
- Posted
Skills
Required
- site reliability / DevOps
- software engineering
- orchestration
- load shedding / admission control
- AWS / GCP / Azure
- GPU
- metrics
- Kubernetes
- infrastructure automation
- Python / Go
- containerization and orchestration
- Docker
- Terraform
Preferred
- ML
- inference serving
- model deployment
- Triton
- KServe
- Ray Serve
- scheduling
- GPU management
- pipeline
- infrastructure
Join our team as a Staff Site Reliability Engineer, where you'll take ownership of managing a large fleet of GPU-backed models. You'll design and build our GPU management and scheduling platform, implement admission control, and develop cloud orchestration systems. This is a senior role for someone with over 10 years of experience in site reliability and software engineering.
Responsibilities
- Design and build the GPU management and scheduling platform, including the metrics pipeline and admission control logic.
- Implement autoscaling that adjusts the number of model replicas in response to real-time demand and utilization.
- Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure, and develop infrastructure automation and deployment pipelines.
Requirements
- Strong software engineering fundamentals — you build orchestration and scheduling systems in Python and/or Go, not just configure off-the-shelf tools
- 10+ years of professional experience across site reliability / DevOps engineering and software engineering
- Strong knowledge of containerization and orchestration (Docker, Kubernetes)
- Experience with monitoring and logging stacks (ELK, Grafana, Datadog, or similar)
- Excellent problem-solving skills and the ability to work both independently and collaboratively
- Computer Science Degree Required from a top CS program
- Experience designing systems that make decisions from operational metrics — collecting signals, interpreting them, and driving control loops such as autoscaling, load shedding, or admission control
- Hands-on production experience with at least one major cloud platform (AWS, GCP, or Azure)
- Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD, or similar)
- Familiarity with secrets management and security tooling (HashiCorp Vault, AWS KMS, Azure Key Vault)
- Strong communication and interpersonal skills
- Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators
- Experience implementing HIPAA and SOC 2 compliance
- Experience with Kubernetes autoscaling internals (HPA/VPA, custom metrics, custom controllers)
- Experience operating in an HPC environment
- Bachelor's or Master's in Computer Science, Computer Engineering, or a related field
Preferred
- Familiarity with ML inference serving and model deployment (e.g. Triton, KServe, Ray Serve, or similar)