Software Engineer (Inference Platform)

Join Baseten, a remote-first company, as a Software Engineer focused on building the distributed runtime for large-scale LLM inference. You'll work across the stack, ensuring models are fast, reliable, and cost-efficient. You'll have the opportunity to own projects end-to-end and make a significant impact on how developers use AI models at scale. Benefits include unlimited PTO, full healthcare coverage, and a learning and development budget.

Déboguer et renforcer des systèmes de production complexes couvrant Kubernetes, les runtimes distribués, le réseau et les charges de travail GPU pour améliorer la fiabilité et l'évolutivité.

This role is ideal for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users

Responsibilities

• Construire l'infrastructure et les systèmes d'orchestration qui déploient et exécutent l'inférence LLM distribuée à grande échelle, y compris le routage, l'autoscaling, la planification et la gestion du runtime.
• Concevoir, construire et exploiter les API de modèle, en mettant l'accent sur les capacités d'inférence avancées : sorties structurées, appel d'outils/fonctions et service multimodal.

Requirements

• 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs where reliability, latency, and scale are first-class concerns
• Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience
• Excellent written communication and collaboration skills, including writing clear design docs and working across functions
• A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations
• Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering. Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus
• Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals
• Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and SLO management
• A strong sense of developer experience. You think about how systems are used, not just how they work
• Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo
• Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways
• Experience with distributed scheduling, autoscaling, or service orchestration
• Experience operating GPU workloads in production
• A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems

Preferred

• Familiarity with observability tooling, CI/CD systems, or release automation

Pay

USD 180,000 - 360,000