Reading Group Notes: Concrete Problems in AI Safety, Ten Years On
Notes from our reading group on which of the paper’s five problems have become research fields — and which remain under-studied.
People, methods and education
Research design, reproducibility and rigorous experimentation.
Notes from our reading group on which of the paper’s five problems have become research fields — and which remain under-studied.
Most safety evaluations are written in English. What changes when we test in Sinhala, Tamil or Arabic?
Every resource in the library shows who reviewed it and when. Here is what that review involves.
A hands-on session on writing, running and interpreting a small language-model evaluation with an open-source framework.
A hands-on session on writing, running and interpreting a small language-model evaluation with an open-source framework.
Dan Hendrycks, Nicholas Carlini, John Schulman, Jacob Steinhardt, arXiv, 2021
A research agenda that organises machine-learning safety into four areas — robustness, monitoring, alignment and systemic safety — with concrete open problems in each.
Neel Nanda and open-source contributors, GitHub, 2022
A Python library for mechanistic interpretability of GPT-style language models that makes it easy to cache, inspect and edit internal activations.
UK AI Security Institute, 2024
A Python framework for writing and running reproducible evaluations of language models, including multi-turn and tool-using agent evaluations.