jobapplier

Staff Software Engineer (Inference / Compute Infrastructure Engineering)

Together AI

Skills

Required

  • Go
  • Rust
  • software engineering
  • Python
  • event-driven systems
  • designing and building software
  • message queues / event streams / pub/sub
  • Kafka
  • NATS
  • SQS
  • cron-driven
  • durable workflow orchestration
  • Temporal
  • Cadence
  • run long-lived, manifest-driven workflows

Preferred

  • Kubernetes
  • GPU scheduling
  • controller/CRD patterns
  • bin-packing
  • right-sizing
  • GPU infrastructure

Join our team as a Staff Software Engineer focused on Inference and Compute Infrastructure Engineering. You will design and build a Kubernetes-native control plane for our GPU inference fleet, creating a manifest-driven API and systems for efficiency. You will own the software you build, from delivery to operation and support in production. Strong candidates will have experience with Kubernetes, GPU scheduling, and a product mindset.

Responsibilities

  • Concevoir et mettre en œuvre un plan de contrôle natif Kubernetes pour provisionner et exécuter une flotte d'inférence GPU.
  • Construire des systèmes pour maintenir l'efficacité de la flotte, y compris la défragmentation et la logique de rééquilibrage.
  • Développer des API déclaratives et un plan de contrôle pour permettre à l'équipe d'inférence de demander, de mettre à l'échelle et de démanteler des clusters d'inférence.
  • Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered
  • A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship
  • A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship
  • Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines)

Requirements

  • Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living
  • Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts
  • Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution
  • Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure
  • Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE)
  • Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization

Preferred

  • Systems programming in Rust or Go

Every job at Together AI