Skip to content
Technical AI Safety Knowledge Hub SAFE | Responsbile | Human Centered AI
Explore the Library
  • Home
  • About
    • Mission and principles
    • Profile of the Founder
    • People & Community
    • How to contribute
    • Contributors
    • Editorial policy
    • Corrections and takedown
  • Explore AI Safety
  • Knowledge Library
  • Research & Projects
    • Research projects
    • Insights and articles
  • Courses & Workshops
    • Learning pathways
    • Workshops
  • Events
  • Contact Us
  1. Home
  2. Explore AI Safety
  3. Robustness and Reliability

Technical AI Safety

Robustness and Reliability

Behaving safely under distribution shift, adversarial pressure and edge cases.

Everything 8 Library 5 Projects 1 Articles 2

Concrete Problems in AI Safety

Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané, arXiv, 2016

A foundational paper that reframed AI safety as a set of practical machine-learning research problems, from reward hacking to robustness under distributional shift.

Research paperIntroductoryAbout 2 hours
  • AI Alignment and Specification
  • Robustness and Reliability
  • Scalable Oversight

Unsolved Problems in ML Safety

Dan Hendrycks, Nicholas Carlini, John Schulman, Jacob Steinhardt, arXiv, 2021

A research agenda that organises machine-learning safety into four areas — robustness, monitoring, alignment and systemic safety — with concrete open problems in each.

Research paperIntermediateAbout 3 hours
  • AI Alignment and Specification
  • Research Methods
  • Robustness and Reliability

Goal Misgeneralization in Deep Reinforcement Learning

Lauro Langosco, Jack Koch, Lee Sharkey, Jacob Pfau, David Krueger, ICML 2022, 2021

Demonstrates agents that keep their capabilities out of distribution but pursue the wrong goal — for example, an agent that learned to run to the end of a level rather than to collect the coin that was always placed there…

Research paperIntermediateAbout 2 hours
  • Agent Safety and Control
  • AI Alignment and Specification
  • Robustness and Reliability

TruthfulQA: Measuring How Models Mimic Human Falsehoods

Stephanie Lin, Jacob Hilton, Owain Evans, ACL 2022, 2021

A benchmark of questions designed so that common human misconceptions produce wrong answers. In the original experiments, larger models were often less truthful, not more.

BenchmarkIntroductoryAbout 1 hour
  • Model Evaluations
  • Robustness and Reliability

Universal and Transferable Adversarial Attacks on Aligned Language Models

Andy Zou, Zifan Wang, J. Zico Kolter, Matt Fredrikson, arXiv, 2023

Shows that automatically optimised text suffixes can make safety-trained language models comply with harmful requests, and that such suffixes often transfer between models.

Research paperAdvancedAbout 2 hours
  • AI Security and Misuse
  • Robustness and Reliability

Related topics

  • Agent Safety and Control
  • AI Alignment and Specification
  • Mechanistic Interpretability
  • Model Evaluations
  • Scalable Oversight
Technical AI Safety Knowledge Hub SAFE | Responsbile | Human Centered AI

An independent platform for AI Safety research, education and governance. Summaries are editorial and are not legal or professional advice.

Explore

  • Start here
  • Knowledge Library
  • Research projects
  • Insights

Learn and take part

  • Learning pathways
  • Workshops
  • Events
  • People and community

Contact

  • contact@aigovernance.lk
  • Contact page
  • Who runs this platform
  • Contribute

With past support from

BlueDot Impact Amazon Web Services (AWS)

© 2026 Technical AI Safety Knowledge Hub. SAFE | Responsbile | Human Centered AI

  • Editorial policy
  • Corrections and takedown
  • Accessibility
  • Privacy policy
  • Contact