Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud

About Zyphra

• Our research methodology is grounded in methodical, step-by-step approaches to ambitious goals. Both deep research and engineering excellence are equally valued
• We strongly value new and crazy ideas and are very willing to bet big on new ideas
• We move as quickly as we can; we aim to minimize the bar to impact as low as possible
• We all enjoy what we do and love discussing AI

Responsibilities

• You’ll Work Across:
• Training clusters
• Administration and automation of our Linux-based cluster environments
• User onboarding/offboarding, access control, and security auditing

Monitoring system resources, job scheduling, and cluster health

• Bring-up and validation of new hardware (compute, networking, storage)
• Supporting and improving developer workflows (e.g., VSCode remote development, Docker)
• Enabling and supporting large-scale training jobs

Inference cloud

• Deploying, operating, and scaling inference services across heterogeneous accelerators
• Managing container orchestration, CI/CD, and rollout pipelines for model deployments
• Building observability for production services, including metrics, logging, alerting, and dashboards

Owning service reliability: uptime, incident response, and postmortems

• Capacity planning, autoscaling, and utilization/cost tracking across hardware types
• Securing production systems through secrets management, network policy, and tenant isolation
• You’ll own critical systems end to end. Your work will directly affect developer productivity, training throughput, and the performance and reliability of our inference platform.

Requirements

• Experience with Linux system administration, user and access management, and automation
• Demonstrated expertise in scripting languages for system tooling and automation (bash, Python, etc.)
• Experience with containers (Docker) and scheduling/orchestration systems (Slurm, Kubernetes)
• Experience running production services, including monitoring and alerting (e.g., Prometheus, Grafana), deployment pipelines, and incident response
• Experience building tooling for cluster validation and reliability (accelerator, networking, and storage health checks)
• Experience setting up and managing developer tools and third-party services (e.g., cloud storage providers, Docker Hub, Slack, Gmail, Telegraf, experiment trackers)
• Excellent debugging and troubleshooting skills across compute, storage, and networking
• Strong communication skills and ability to collaborate across technical and non-technical teams

Preferred

Experience with infrastructure as code (e.g., Ansible, Terraform)

• Prior work supporting ML/AI infrastructure, including GPU management and workload optimization
• Experience deploying and operating LLM serving stacks (e.g., vLLM, SGLang, Ray)
• Experience with multiple accelerator platforms (e.g., NVIDIA CUDA, AMD ROCm, TPUs)
• Experience with high-performance networking (InfiniBand, RoCE) and distributed storage
• Experience working with cloud platforms such as AWS, Azure, or GCP
• Familiarity with Apptainer and with integrating containers into Slurm and Kubernetes

Benefits

• Comprehensive medical, dental, vision, and FSA plans
• Competitive compensation and 401(k) plan
• Relocation and immigration support on a case-by-case basis
• In-office snacks and meals provided
• Unlimited PTO and company holidays
• In-person team in San Francisco with a collaborative, high-energy environment