Brier Score

from blog Two-Wrongs, | ↗ original
The accuracy of a binary prediction (i.e. will an event happen or not) can be evaluated through the Brier score. For all the faults of the Brier score11 Does not account for question difficulty, does not penalise overconfidence as strongly as e.g. a log score, cannot account for mutually exclusive forecasts, which implies it also cannot handle forecasting full distributions., it’s the most standard score we have to compare the accuracy of predictions and predictors.