jobapplier

Research Residency – Red Team

AI Security Institute

Skills

Required

  • machine learning / AI security / alignment / AI control / evaluations
  • ML
  • LLMs
  • adversarial
  • reinforcement learning / fine-tuning / evolutionary methods
  • optimisation
  • PyTorch / Hugging Face
  • open-source

Preferred

  • control
  • data poisoning
  • jailbreaks
  • AI safeguards
  • AI safety
  • security research
  • CTF
  • model internals
  • AI control literature
  • AI

The deadline for applying to this role is Tuesday 8th December, end of day, anywhere on Earth

We are looking for exceptional candidates currently studying for a PhD, or postdoctoral researchers who have submitted their PhD within the last 2 years, in machine learning, computer science, computer security or a related field.

About AI Security Institute

About the AI Security Institute

The AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We’re in the heart of the UK government with direct lines to No. 10 (the Prime Minister's office), and we work with frontier developers and governments globally.

We’re here because governments are critical for advanced AI going well, and UK AISI is uniquely positioned to mobilise them. With our resources, unique agility and international influence, this is the best place to shape both AI development and government action.

The Red Team conducts cutting-edge research to evaluate the safety measures of frontier AI systems, sharing our findings directly with leading AI companies, UK officials and allied governments. Our work shapes how the most powerful AI systems are deployed - you can read about it in the The New York Times

The Economist, and

  • The Telegraph. Demand for this work is growing faster than we can currently meet - we're scaling to address that gap.
  • The Red Team’s work is crucial to ensuring advanced AI systems are developed and deployed safely. We are composed of three specialised sub-teams:
  • Alignment – developing methods to stress-test whether models reliably act as intended (e.g.,
  • our evaluation of GPT-6 Astra, and our research into cheating behaviour in frontier model evaluations
  • Control – researching methods for stress-testing and evaluating the measures that identify and prevent agents taking harmful actions
  • Misuse - stress-testing safeguards against misuse of dangerous capabilities (e.g.
  • our safeguard testing with Anthropic and OpenAI, and recent work on Boundary Point Jailbreaking attacks on layered safeguard systems and principles for evaluating misuse safeguards
  • While each team has its own focus, we share methodologies, infrastructure and tooling across the wider team.
  • This is an exciting opportunity for motivated and talented researchers to join AISI's Red Team, contributing to priority research objectives within a fast-paced environment in government.

Responsibilities

  • Are familiar with loss-of-control threat models, such as deceptive alignment, research sabotage or reward-seeking
  • resident

on one of our sub teams, you

  • Work closely with research scientists to decide which questions matter most and how to test them
  • Work on a defined project with support from an experienced research engineer or scientist, for example new attack methods, automated red teaming, alignment evaluations, or attacks on AI monitors and agents
  • Help design and run attacks and evaluations on frontier AI systems to find weaknesses in their safety measures, such as jailbreaking safeguards, uncovering misaligned behaviour, or evading control monitors
  • Write code and help build tools to run experiments at scale, such as testing attack techniques across several models, with a focus on reliability, speed and reproducibility
  • Analyse experiment results carefully and turn them into clear findings for technical and policy audiences
  • Keep up with new attack techniques and the research literature, apply relevant insights to our work
  • Learn how frontier AI models are evaluated and red teamed in practice
  • Illustrative projects
  • Create and improve novel automated research approaches for developing better red-teaming algorithms, including evolutionary approaches like OpenEvolve and multi-agent approaches
  • Jailbreaking and attack pipelines, for example
  • Boundary Point Jailbreaking of Black-Box LLMs and
  • STACK: Adversarial attacks on LLM safeguard pipelines
  • Backdoors and data poisoning, for example
  • Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
  • Alignment
  • Build methods that
  • automate the identification of
  • model
  • behaviour, such as research sabotage or reward-seeking
  • research sabotage

cheating in evaluations

  • Develop methods to improve the realism of simulated evaluations (for example, our simulated supply-chain attack evaluation of GPT-6 Astra
  • Investigate, after the fact, what drove examples of misaligned behaviour in the wild (as in our investigation of AISI's July 2026 incident
  • Control
  • Develop better ways to evaluate AI monitors
  • working out how attacks on monitors can predict whether they’ll catch harmful agent behaviour before it happens in real deployments
  • of the
  • with the paradigm.

Requirements

  • Have research experience in adversarial machine learning, AI security, alignment, AI control or evaluations, or are familiar with this research
  • Have experience with optimisation methods such as reinforcement learning, fine-tuning or evolutionary methods
  • Have hands-on experience red teaming or jailbreaking AI systems, or have taken part in capture-the-flag, bug bounty or other security challenges
  • Have built something with LLMs, such as an evaluation, an agent or a fine-tuned model
  • Have used ML frameworks such as PyTorch or Hugging Face
  • Have shared research or technical work publicly, for example a paper, preprint, open-source tool or blog post
  • scientific

understanding

  • You may be a good fit if you have some of the following skills, experience and attitudes:
  • Experience running experiments or analysing data, for example in coursework or projects
  • Good research habits, such as keeping work reproducible, recording what you tried and being honest about uncertainty
  • Curiosity about how systems fail, and a willingness to try unusual approaches
  • Clear written and spoken communication, including explaining technical work to non-specialists
  • Openness to feedback, and a willingness to ask for help when you're stuck

Preferred

  • We additionally may factor in experience with particular areas like:
  • Published or co-authored research on adversarial ML, jailbreaks, data poisoning, AI safeguards, alignment, control or evaluations
  • Previous research experience in ML or AI safety, such as a research internship, a research assistant role or a substantial thesis project
  • A background in security research, such as finding and disclosing real vulnerabilities, or placing well in CTF competitions
  • Experience fine-tuning or modifying open-weight models, or working with model internals
  • Familiarity with the alignment or AI control literature, or with current methods for post-training and aligning LLMs
  • Experience with RL, supervised fine-tuning or other ML optimisation, including careful choices about what to measure
  • Experience in security analysis of agent scaffolds, sandboxes or ML infrastructure
  • Experience testing the security of AI agents, for example prompt injection or tool misuse
  • You want to lead or set research strategy from day one. You'll have real influence over your project and will work with experienced research leaders, who set our agenda during this residency
  • You prefer well-defined requirements and long planning cycles. Our priorities shift as the landscape of frontier AI changes rapidly, and projects often run to tight deadlines
  • You are uncomfortable in a collaborative research environment and prefer extended periods of solo working
  • Motivated candidates are encouraged to apply even if they don't meet all of the above criteria.
  • Selection

We will by default consider this a 6-month residency but are flexible to shorten or extend per candidate preferences and availability. Following an end of residency review, promising candidates may be offered the opportunity to join their preferred team in a full-time employee role.

We are particularly interested in candidates who have one or more of the following. You don't need all of them, and we'd encourage you to apply even if you only match a few:

  • The projects below are examples of the kind of work you might do. Our priorities change quickly, so your actual project may be different, and we're open to matching work to your interests and skills.
  • Misuse

Prompt injection and AI agent security, for example

Breaking agent backbones harmful or unsanctioned

limitations of

including both

tractable problems with current evaluations, and

Motivation to reduce risks from advanced AI and to support government work on AI security

This might not be the role for you if:

process

The interview process may vary from candidate to candidate, however, you should expect a typical process to include some technical proficiency tests, discussions with a cross-section of our team at AISI (including non-technical staff) and conversations with your team lead. The process will culminate in a conversation with members of the senior leadership team here at AISI.

Candidates should expect to go through some or all of the following stages once an application has been submitted:

Initial assessment

Initial screening call

Technical assessment

Research interview

Final interview with members of the senior leadership team

Easy Apply