Research Intern (Frontier Speculative Decoding, Winter 2027)

About Together AI

Join Together AI as a Research Intern in the Frontier Speculative Decoding team. You will work on cutting-edge research questions in speculative decoding, train and fine-tune draft models, and implement optimized versions for serving engines. This internship offers a unique opportunity to collaborate with industry-leading engineers and researchers, gain hands-on experience in large model training, and contribute to the advancement of AI technology.

Responsibilities

• Research and prototype new efficient speculative decoding algorithms and draft model architectures.
• Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
• Implement optimized version for serving engines to show efficacy in real world workloads.

Requirements

• Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field
• Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
• Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
• Strong knowledge of Machine Learning and Deep Learning fundamentals

Preferred

• Prior research experience in speculative decoding, efficient inference, or efficient machine learning
• Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)
• Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)
• Experience with knowledge distillation, post-training, or reinforcement learning
• Contributions to open-source machine learning projects