jobapplier

Software Engineer (LLM Infrastructure)

Fireworks AI

Skills

Required

  • infrastructure / machine learning systems
  • LLM
  • software engineering
  • context length
  • disaggregated
  • prefill
  • KV
  • cache memory estimation
  • programming
  • Python
  • Go

Preferred

  • ML / MLOps
  • open source inference engine
  • vLLM / Sglang / TRT-LLM
  • open-source infrastructure
  • AI
  • CI/CD
  • ML infrastructure and tooling
  • PyTorch
  • MLflow
  • Kubernetes

About Fireworks AI

Join Fireworks AI, a fast-growing generative AI platform, as a Software Engineer on our AI Infrastructure team. You will design and build the core systems that power our platform, ensuring reliability, performance, quality, and availability. Collaborate with various teams to deliver infrastructure solutions and tackle challenges at the forefront of AI infrastructure.

Responsibilities

  • Contribuer à la conception et au développement d'une infrastructure backend évolutive qui prend en charge l'entraînement distribué, l'inférence et les pipelines de données.
  • Construire et maintenir des services backend essentiels tels que le pipeline CI/CD LLM, le plan de contrôle et les systèmes de service de modèle.
  • Collaborer avec les équipes de performance, de formation et de produit pour traduire les besoins de recherche et de produit en solutions d'infrastructure.
  • Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)

Requirements

  • Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)
  • Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
  • Strong programming skills in Python, Go, or a similar language
  • 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
  • 5+ years of experience in software engineering, with a focus on infrastructure or machine learning systems

Preferred

  • Experience with open source inference engine like vLLM, Sglang, or TRT-LLM
  • Experience in building large scale ML/MLOps infrastructure
  • Contributions to open-source infrastructure or ML projects

Pay

USD 175,000 - 220,000

Every job at Fireworks AI