An Opinion is not a Take

The accuracy of a binary prediction (i.e. will an event happen or not) can be evaluated through the Brier score. For all the faults of the Brier score11 Does not account for question difficulty, does not penalise overconfidence as strongly as e.g. a log score, cannot account for mutually exclusive forecasts, which implies it also cannot handle forecasting full distributions., it’s the most standard score we have to compare the accuracy of predictions and predictors.

There is an experiment by LessWrong user dkl9 where they try to figure out how their mood is affected by music. Aside from the difficulties of running with no replicates11 Which is what always happens when one experiments on oneself only., they had a fairly good setup going. You can read their article for more details22 Some of those details are improvements on the basic design I’ve laid out here., but for the sake of this article, we can pretend the experiment worked like this:

Subscribers to the weekly Entropic Thoughts newsletter know that in every newsletter, I share one interesting flashcard from the past week. The newsletter has now run for six months, and this is a compilation of the flashcards that have been selected, for the benefit of non-subscribers.

This was such an interesting question. What’s the obvious way to figure out whether Bocelli sold out the Garden? You wait to see if the event organisers claim he has sold out. But what if they don’t say anything? You want another resolution method in that case, say, trying to buy tickets and seeing if there are any officially available.

The compass does not actually point to true north. This is because the magnetic north pole is not quite on the actual northernmost point of Earth – our slushy rock, and its magnetic field, are two separate things. The dynamics of spinning suggest that the magnetic field will be roughly aligned with the true north–south axis, but this is no guarantee.