Research Intern (Frontier Speculative Decoding, Summer 2027)

About Together AI

Join Together AI as a Research Intern in the Frontier Speculative Decoding team. You'll work on cutting-edge research questions in speculative decoding, train and fine-tune draft models, and implement optimized versions for serving engines. This internship offers hands-on experience in machine learning and deep learning, with opportunities to communicate findings and bring successful ideas into production.

Responsibilities

• Research and prototype new efficient speculative decoding algorithms and draft model architectures.
• Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
• Implement optimized version for serving engines to show efficacy in real world workloads.

Requirements

• Strong knowledge of Machine Learning and Deep Learning fundamentals
• Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
• Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field

Preferred

• Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
• Prior research experience in speculative decoding, efficient inference, or efficient machine learning
• Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)
• Experience with knowledge distillation, post-training, or reinforcement learning
• Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)
• Contributions to open-source machine learning projects