Staff + Senior Software Engineer (Scaling)
- Seattle, United States
- Lead/Staff
- Posted
Skills
Required
- distributed systems
- machine learning systems
- infrastructure
- machine learning
- software engineering
- LLM inference optimization
- batching
- caching
- Kubernetes
- cloud infrastructure
- AWS
- GCP
- Azure
Preferred
- load balancing / request routing / traffic management systems
- Python / Rust
- traffic management
- maximize compute efficiency
Join our Inference team as a Staff + Senior Software Engineer (Scaling). You will be responsible for building and scaling the critical systems that serve our AI model, Claude, to millions of users worldwide. Your work will involve designing and maintaining distributed systems, developing intelligent request routing and load balancing systems, and maximizing compute efficiency across multiple cloud providers. You will also have the opportunity to contribute to new inference features and support inference for new model architectures.
Results-oriented, with a bias towards flexibility and impact
We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed
Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself prematurely and to submit an application if you're interested in this work
About Anthropic
Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide, ensuring high performance and reliability.
We think AI systems like the ones we're building have enormous social and ethical implications. We think this makes representation even more important, and we strive to include a range of diverse perspectives on our team
Responsibilities
- Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators and multiple cloud providers to maximize compute efficiency.
- Build and operate production-grade deployment pipelines for releasing new models to users, contributing to new inference features and supporting inference for new model architectures.
- Care about the societal impacts of your work
Requirements
- Desire to learn more about machine learning systems and infrastructure
- Willingness to pick up slack, even if it goes outside your job description
- Significant software engineering experience, particularly with distributed systems
- Thrive in environments where technical excellence directly drives both business results and research breakthroughs
- Familiarity with LLM inference optimization, batching, and caching strategies
Experience with high-performance, large-scale distributed systems
- Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure)
- Experience implementing and deploying machine learning systems at scale
Preferred
- Proficiency in Python or Rust
- Experience with load balancing, request routing, or traffic management systems
Pay
USD 320,000 - 485,000
