Senior Staff Inference Performance Engineer

Join d-Matrix, a leading AI company, as a Senior Staff Inference Performance Engineer. In this role, you will be responsible for the end-to-end performance of our inference workloads, ensuring they perform at least as well as they do on bare metal.

Responsibilities

• You will diagnose performance problems across the full serving path, isolate each problem to a specific layer and owner, and protect throughput, latency, and stability targets for orchestrated workloads.
• You will also assess what it takes to scale from single deployments to fleet operation and build lightweight telemetry and benchmarks.
• This position offers equity, health care, flexible time-off, paid paternity leave, a 401k retirement plan, and a remote & hybrid working model.
• Posséder la performance de bout en bout des charges de travail déployées sur la plateforme orchestrée, en veillant à ce qu'elles fonctionnent au moins aussi bien que sur du matériel nu.
• Diagnostiquer les problèmes de performance sur l'ensemble du chemin de service, de l'orchestration aux internes d'inférence, en isolant chaque problème à une couche spécifique.
• Écrire des déclarations de problème minimales et reproductibles pour les constatations inter-couches afin que les équipes responsables puissent agir sans répéter l'enquête.
• Sound judgment about ownership, with examples of correctly handing off problems that were not yours to fix

Requirements

• Clear technical writing that hardware, firmware, or platform teams can act on without context from your part of the stack
• 8+ years of hands-on performance or systems engineering, including resolving hard regressions that crossed multiple layers of a stack
• Proficiency in Python and C/C++, sufficient to instrument, debug, and patch code across the stack
• Deep systems fundamentals, including Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths
• Experience running performance-sensitive workloads on Kubernetes, including identifying when scheduler defaults, network overlays, or orchestration abstractions cost latency or throughput, and closing the gap
• Track record of finding performance overhead through profiling in unfamiliar, sparsely documented systems, valued more than experience with any specific vendor stack or accelerator
• Fluency with AI and agentic tools for investigation, code search, log triage, and drafting instrumentation, with the discipline to verify output against real data
• Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM
• Hands-on experience building or tuning speculative decoding, disaggregated serving, or heterogeneous serving architectures
• Experience tuning KV-cache management, request and batch scheduling, or weight prefetch and residency in an LLM serving system

Preferred

• Working knowledge of LLM inference serving, including batching, KV-cache, prefill/decode, quantization, and speculative decoding
• GPU programming and profiling experience (such as CUDA, Nsight, NCCL), valued as transferable background
• Experience with pre-production or bring-up hardware, or in organizations where root causes routinely cross team boundaries
• Rust experience
• Startup experience or comfort in environments where priorities shift quickly

Pay

USD 175,000 - 265,000