The Models - Covid or Weather

Those of us who live in hurricane-stricken  areas watch weather models very closely.  The famous spaghetti models of  possible hurricane tracks are a huge source of merriment or some of us.  They keep wandering as the data firms up, and generally about 12 hours prior to landfall we have a good idea about where the storm is going.  Even then, the models fail sometimes.

Over the past couple of months, we've been watching public health models trying to predict what might happen during the covid crisis.  And, like the weather models, these models have been all over the place.  Initial estimates of 2.2 million US deaths are probably wrong at this point.  Woefully inaccurate.

Unlike weather models, public health models can scare the hell out of us, and motivate us to change behavior to influence the outcome.  That may be what is happening here.  But, we can't say that with certainty, simply because we can't measure something that never happened.

When I was taking statistics classes in college, we had something called a degree of confidence.  When using statistics we might assign a "95%  degree of confidence" to a model, but that other 5% was still hanging out there.  We were pretty confident, but we didn't know, because there is no way of knowing.  YOU CAN'T MEASURE SOMETHING THAT DID'T HAPPEN.

When this crisis is over, and someone tells you that he knows something or other, ask for the degree of confidence.  If he tells you that he knows cor certain that we saved lives, ask him how many lives.  Certainty is a powerful thing until you start trying to pin it down and if he can't pin it down, he's not certain.  See how that works?

We try to operate under certainty, an that's impossible.  There is always a degree of confidence, and if the modeler isn't willing to give a degree of confidence, then the listener is perfectly justified in assigning something I call a "degree of bullshit"  Confidence + bullshit = 100%.  There is always bullshit in every model.  We cannot measure what did not happen.