Senior AI Platform Engineer

As a Senior AI Platform Engineer, you will design, build, and operate key components of our AI enablement stack, including GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. This is an engineering-focused role where you will own one area of the platform while contributing to the rest of the stack. You will also have the opportunity to work in a flexible hybrid work environment and enjoy a range of benefits, including uncapped vacation, retirement plans, and health care coverage.

We don't expect you to arrive with every skill on this page

You'll also need solid data science and AI research fundamentals: you'll work closely with our Applied AI Research team, and you'll sometimes run experiments when a platform decision needs evidence

About Code Metal

Code Metal, a company focused on building AI-driven solutions for code transpilation, modernization, optimization, modeling, and simulation.

Responsibilities

• Concevoir, construire et exploiter des parties de la pile d'activation de l'IA, y compris le service d'inférence GPU, les passerelles de modèle, les infrastructures agentiques, l'ingénierie du contexte, l'observabilité et la gestion des expériences.
• Posséder un ou plusieurs composants de la plateforme de bout en bout, de la documentation de conception à l'exploitation en production, et contribuer à l'ensemble de la pile.
• Écrire un code propre et bien testé, déboguer des problèmes de production difficiles dans les composants que vous possédez, et examiner le code et les conceptions de vos coéquipiers.
• Agentic infrastructure: Build agent harnesses, orchestration primitives, and context-engineering services (memory, retrieval, and data discovery) that product teams compose into reliable, verifiable workflows
• Observability and experimentation: Instrument the stack end to end, for example, with OpenTelemetry traces and service metrics. Build evaluation harnesses and the experiment-tracking and artifact layer that lets engineers and researchers reproduce and compare results
• Model serving: Deploy, benchmark, and tune elastic, production inference for open-weight models on runtimes such as vLLM, SGLang, and TensorRT-LLM. Run the model gateways that teams use to reach self-hosted and commercial models
• Each engineer on the team brings depth in at least 1 of these areas:
• Model serving: deploying and tuning LLM inference, or running an LLM gateway or API gateway
• Observability and experimentation: instrumenting and operating services against SLOs, or building evaluation and experiment-tracking systems
• Experience owning a service or component in production, including debugging and improving its reliability

Requirements

• Production-grade Python and solid platform engineering fundamentals: API and service design, distributed systems, containers and orchestration (such as Kubernetes), CI/CD, and testing
• Agentic infrastructure: shipping agentic systems, or building context-engineering services such as retrieval-augmented generation, embeddings, and vector or hybrid search
• Solid data science and AI research fundamentals: how transformers and LLM inference work, experiment design, benchmarking, and model evaluation. Working familiarity with PyTorch and Hugging Face
• Experience writing design docs for focused projects, reviewing code, and mentoring or onboarding teammates
• Experience in more than 1 focus area

Preferred

Production experience in at least 1 focus area

• Typically 4+ years of software engineering experience, including 2+ years building and operating ML/AI or LLM systems in production
• Production experience running an LLM gateway or proxy such as SMG or Bifrost, or building an API gateway, including routing, auth, rate limiting, quotas, failover, and cost attribution
• Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, GPU profiling
• Experience with durable workflows, agent frameworks, or tool protocols such as MCP
• Experience fine-tuning language models, including distributed training across multiple GPU nodes
• Experience with experiment-tracking or artifact management systems such as MLflow or Weights & Biases
• Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace

Pay

USD 170,000 - 210,000