Agentic AI Systems: A Safety Checklist for Builders
Practical questions to answer before giving a language-model agent access to tools, data or the internet.
Security and misuse
Structured analysis of how harms could arise and who could cause them.
Practical questions to answer before giving a language-model agent access to tools, data or the internet.
Dan Hendrycks, Mantas Mazeika, Thomas Woodside, arXiv, 2023
An accessible survey that sorts the most serious risks from advanced AI into four groups — malicious use, competitive race dynamics, organisational failures and loss of control — and discusses mitigations for each.
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, et al., arXiv, 2023
Argues that developers should evaluate models both for dangerous capabilities and for their propensity to use them, and that these evaluations should feed into training, deployment and governance decisions.
OWASP Foundation, OWASP, 2023
A community-maintained list of the most important security risks in applications built on language models, such as prompt injection and insecure handling of model output, with mitigations for each.
MITRE, 2021
A knowledge base of adversary tactics and techniques against AI-enabled systems, modelled on MITRE ATT&CK and illustrated with case studies.
Chaired by Yoshua Bengio, with an international expert advisory panel, UK Government (GOV.UK), 2025
A scientific assessment of the capabilities and risks of general-purpose AI and of the techniques available to manage them, written by an independent panel of experts nominated by many countries and international organisations.