Evaluates many language models across a broad set of scenarios and several metrics at once — including calibration, robustness, fairness, bias and toxicity — rather than accuracy alone.
This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.
Why it matters
HELM’s taxonomy of scenarios and metrics is a useful checklist when designing any evaluation, and its coverage gaps — including limited coverage of South Asian languages — are a reminder of how much remains to be measured.
What you will learn
- Explain why multi-metric evaluation matters
- Identify gaps in scenario coverage, such as languages
Before you start
Suggested background. This is editorial guidance, not a requirement.
- Basic understanding of benchmarks
How to cite
Liang, P., Bommasani, R., Lee, T., et al. (2023). Holistic Evaluation of Language Models. Transactions on Machine Learning Research.