Member of Technical Staff - Evals Research Engineer
NeoSigma
- San Francisco, United States
- Lead/Staff
- Posted
Skills
Required
- machine learning
- deep learning
- LLM evaluation / agentic systems / reinforcement learning / post-training
- Python
- ML
- infrastructure
Preferred
- telemetry ingestion
- platform reliability
- sandbox infrastructure
- API
- distributed systems
- query performance
- backend
- APIs
- agent evaluation
- reward modeling
We are looking for a Backend Engineer to own the systems that ingest, evaluate, and serve agent behavior data at production scale.
About NeoSigma
Customer Obsession - We start with the customer and work backwards.
We aim to earn trust through consistent delivery, thoughtful listening, and by obsessing over customers.
Bias for Action - We set high standards and move quickly to meet them.
We prefer building and learning with customers over debating in the abstract, and we iterate based on real feedback.
Extreme Ownership - We take responsibility for outcomes, not just tasks.
About Us NeoSigma is a product-driven research lab building the intelligence layer that helps close the feedback loop between your customers, products, and AI systems. We are a small, intensely technical team of researchers and engineers who have trained frontier-scale models and widely used AI products and agents at Parallel Web, Essential AI, MIT, Apple, and Amazon. We are backed by world-class investors and leaders from Google DeepMind, OpenAI, Decagon and others.
Responsibilities
- You will work across telemetry ingestion, platform reliability, sandbox infrastructure, and the API surface that customers and integrations build on.
- Some weeks you will be deep in distributed systems and query performance .
- Other weeks you will ship a customer-facing feature across backend, APIs, and the data layer.
Design and build evaluation frameworks for agentic systems
- Research methods for agent evaluation, reward modeling, reward-hacking, post-training, and self-improving systems
- Develop rubrics, verifiers, reward signals, and benchmarks to measure agent performance, reliability, and generalisation
- Own research streams end-to-end and building infrastructure to ship working systems
- Publish research findings through technical blogs, papers, and open-source contributions
Requirements
Moving fast with high agency
- Strong foundations in machine learning, deep learning, and computer science Hands-on experience with LLM evaluation, agentic systems, reinforcement learning, or post-training
- Strong programming skills in Python and experience building reliable ML research infrastructure
- Strong research taste and the ability to identify and solve complex and meaningful problems
- Comfortable navigating open-ended research problems and quickly translating ideas into working implementations
- High standards for experimental rigor, engineering quality, and clear technical communication
Benefits
- Ownership means seeing problems through to completion and ensuring solutions truly work in practice.
- Benefits and perks
- Competitive salary plus meaningful equity package
- Comprehensive medical benefits and generous PTO Flexible work arrangements
Direct impact on company direction and technical decisions
- High ownership and the opportunity to make a career-defining impact
- Visa sponsorship As a founding member, you’ll help define the technical foundation of NeoSigma.
- Your scope will grow with the company, from owning core systems end-to-end to shaping architecture, hiring, and engineering culture.
- Intellectual Honesty - We operate with high trust and low ego. Ideas matter more than titles, and we communicate openly and directly while assuming good intent, even in strong disagreement.
- This role has a natural path toward technical leadership or engineering management as the team scales.
