Staff Machine Learning Engineer (AI Platform & Agentic Apps)
- Menlo Park, United States
- Lead/Staff
- 10+ years
- Master's degree
- Posted
Skills
Required
- orchestration
- simulation environments
- Python
- building agentic systems
- context management
- multi-step planning
- eval / safety / agent tooling
- distributed-systems
- LLM-powered
- evaluation methodology
- golden datasets
Preferred
- Build trajectory-level evaluation systems
- integrations
- designing action-level guardrails
- tool-scoping models
- approval gates
- blast-radius controls
Join Robinhood as a Staff Machine Learning Engineer on the AI Platform & Agentic Apps team. In this role, you will design and build the core of Robinhood's agent harness, ship agentic applications, and build trajectory-level evaluation systems. You will also architect action guardrails as platform primitives and make evals and guardrails products that other teams adopt. This position requires 10+ years of experience as a Machine Learning Engineer or ML-focused software engineer, with a strong background in Python and distributed systems.
Responsibilities
- Design and build the core of Robinhood's agent harness, including orchestration, tool integrations, context and memory management.
- Ship agentic applications end to end on that harness, from an ambiguous problem to a production agent that takes real action.
- Build trajectory-level evaluation systems that score how an agent got to an answer, not just the answer, backed by simulation environments.
- Demonstrated expertise designing action-level guardrails — permission and tool-scoping models, approval gates, blast-radius controls, and sandboxing — for agents operating in systems where mistakes have consequences
Requirements
- Hands-on experience building agentic systems end to end — tool use, orchestration, context management, multi-step planning — on top of frontier models, in production
- Proven ability to build platforms, not just models: you have shipped eval, safety, or agent tooling that other engineering teams adopted, and you have the judgment to know when to build versus buy
- Deep expertise evaluating agents: you have built trajectory-level evals, tool-call scoring, and simulation environments, and you can articulate why final-answer accuracy is insufficient for systems that act
- 10+ years of experience as a Machine Learning Engineer or ML-focused software engineer, with strong Python and distributed-systems fundamentals and a track record of shipping LLM-powered systems to production at scale. A Master's degree in Computer Science or a related technical field, or equivalent professional experience
- Rigor in evaluation methodology: golden datasets, rubric and LLM-as-judge grading and their failure modes, statistical significance with small N, offline-to-online metric correlation, and eval data versioning and contamination control
Pay
USD 255,000 - 300,000
jobapplier