Inference Optimization Engineer

Join our team as an Inference Optimization Engineer, where you'll work at the intersection of research and systems to make state-of-the-art visual models fast and efficient at inference time. You'll profile model architectures, develop new approaches to improve inference latency and throughput, optimize GPU kernels, and investigate communication and memory movement strategies. This role requires a strong understanding of deep learning models, performance tradeoffs, and experience with relevant technologies.

Responsibilities

• Collaborer avec l'équipe de recherche pour rendre les modèles visuels rapides et efficaces lors de l'inférence.
• Identifier les goulets d'étranglement et mettre en œuvre des optimisations à travers l'architecture du modèle, les algorithmes d'inférence, les temps d'exécution, les noyaux et l'exécution distribuée.
• Développer et mettre en œuvre de nouvelles approches pour améliorer la latence d'inférence, le débit, l'efficacité de la mémoire et l'utilisation du GPU.
• Strong understanding of how modern deep learning models execute on hardware, including the relationship between compute, memory, communication, and performance
• Ability to reason across abstraction layers rather than treating model architecture, framework, runtime, kernel, and hardware boundaries as fixed
• Curiosity about how models work internally and a willingness to modify or rethink existing approaches when the performance problem calls for it
• Contributions to ML systems, inference runtimes, compilers, GPU libraries, or performance-focused open-source projects

Requirements

• Ability to communicate technical ideas clearly and collaborate closely with research scientists and engineers
• Strong intuition for performance tradeoffs across latency, throughput, memory, numerical precision, model quality, and complexity
• Deep technical ability in efficient ML inference, ML systems, GPU computing, or adjacent research, demonstrated through research, production engineering, open-source contributions, or unusually ambitious independent work
• Comfort working on ambiguous problems where the bottleneck, and sometimes even the right question, is not known in advance
• Experience profiling ML workloads, identifying bottlenecks, forming hypotheses, and driving measurable performance improvements
• Experience with some combination of PyTorch, CUDA, Triton, TensorRT, vLLM, SGLang, or comparable technologies
• Strong programming fundamentals in Python, C++, or another systems-oriented language
• We don’t expect every candidate to have experience across the entire stack. Exceptional depth in one or more relevant areas, combined with the ability and curiosity to reason across the others, matters more to us than checking every box
• Experience optimizing large generative, multimodal, vision, or video models
• Multi-GPU or multi-node model execution, including tensor, pipeline, sequence, or other forms of parallelism

Model compilation or graph optimization experience

• Research or publications in efficient ML, ML systems, GPU computing, compilers, or related areas
• Experience optimizing diffusion, autoregressive, transformer, or other large generative architectures
• Experience with profiling tools such as Nsight Systems or Nsight Compute
• Deep knowledge of GPU architecture, memory hierarchy, and hardware-aware optimization

CUDA, Triton, CUTLASS, or other GPU kernel development

• Quantization, sparsity, caching, speculative execution, or other efficient inference techniques
• Experience with attention optimization, kernel fusion, memory-efficient execution, or custom operators
• Experience optimizing communication or data movement between accelerators