Join our team as an LLM Inference Systems Engineer, where you will be responsible for building the distributed serving and high-performance networking layer behind our large language models.
Experience benchmarking new GPU platforms and turning results into production architecture decisions
Responsibilities
You will work closely with the Model, Infrastructure, and Application engineering teams to design, implement, and operate disaggregated LLM serving architectures.
Your expertise in optimizing multi-node inference across modern GPU systems and diagnosing complex distributed failures will be crucial to the success of this role.
Conception, mise en œuvre et exploitation d'architectures de service LLM désagrégées, y compris les pools de pré-remplissage et de décodage indépendants.
Optimisation de l'inférence multi-nœuds sur des systèmes GPU modernes en utilisant NVLink, NVSwitch, NVLS, InfiniBand, RoCEv2, GPUDirect RDMA, NCCL, UCX.
Clear written and verbal communication across research, infrastructure, and product-facing teams
Requirements
A Master's or PhD is required, along with experience in deploying production workloads on Kubernetes and a strong understanding of LLM inference.
Production de services sur Kubernetes avec des vérifications de santé, une mise à l'échelle automatique, des déploiements sûrs, des métriques, un traçage et des diagnostics exploitables.
Master's or PhD Required
Experience deploying production workloads on Kubernetes and operating them with clear reliability and observability standards
Strong Python skills and working proficiency in C++ or another systems language
Ability to profile and debug performance across application, runtime, kernel, network, and infrastructure layers
Strong understanding of LLM inference, including tensor/pipeline/data parallelism, continuous batching, KV-cache management, and prefill/decode behavior
Production experience building or operating distributed systems, high-performance computing systems, or large-scale ML inference platforms
Hands-on experience with GPU communication and networking technologies such as NCCL, NVLink/NVSwitch, InfiniBand, RoCE, RDMA, UCX, or equivalent systems
Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM
CUDA, Triton, custom kernel, or low-level GPU performance experience
Contributions to open-source inference, networking, or distributed-systems projects
Preferred
Experience with NVLS, SHARP, GPUDirect RDMA, UCX, RDMA congestion control, or GPU-cluster topology optimization
Experience with speculative decoding, multi-LoRA serving, quantization, or cache-aware routing
jobapplier