Senior Manager of Engineering (AI Inference)
- San Francisco, United States
- Lead/Staff
- 2+ years
- Bachelor's degree
- Posted
Skills
Required
- inference
- large language models
- AI/ML
- software engineering / low-level optimization / ML infrastructure
- LLMs
- ML-focused
- Python / C++
- Docker
- Kubernetes
Preferred
- CUDA
- ML
- balancing team leadership
- tailor deployments to
- optimizations
- AI/ML pipelines
Join Crusoe, a company focused on advancing AI strategies and making a meaningful impact in the world. As a Senior Manager of Engineering, you will lead a team dedicated to optimizing large language models for production. This role combines team leadership, technical direction, and hands-on coding, with a customer-facing aspect that ensures performance gains deliver clear business and customer value.
Responsibilities
- Lead an engineering team focused on optimizing large language models for production, balancing team leadership with hands-on technical work.
- Collaborate with customer engineering teams to tailor deployments to their needs, ensuring performance gains deliver clear business and customer value.
- Drive the delivery of projects from early experiments to production optimizations, establishing performance goals and partnering with product and engineering leaders.
- A working knowledge of AI/ML pipelines and the full path of developing and deploying ML models
Requirements
- A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field
- Strong communication skills, particularly when explaining hard technical topics to customers and teammates
- Familiarity with methods for optimizing LLMs for high throughput / low latency inference
- Strong technical, hands-on experience in software engineering, low-level optimization, or ML infrastructure, with a continued desire to stay close to code and architecture
- Clear interest and hands-on experience with large language models
- A firm grasp of how GPUs are built and how they behave
- 2+ years of experience directly managing and leading an engineering team in a high-performance or ML-focused environment
- Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python
- Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level
- A track record of making software systems run faster, especially for large language models
- A strong command of software engineering fundamentals, with a record of building and shipping AI/ML inference systems
- Experience with Docker and Kubernetes
Preferred
- Experience with CUDA or comparable technologies
- Prior work building or tuning AI/ML projects, particularly in a customer-facing setting
Pay
USD 250,000 - 300,000
