The 3-sigma bump reported by Fermilab on Wednesday has garnered a lot of attention. Understandable, since it might be a precious sign of particle physics beyond the Standard Model — but it’s also just a 3-sigma bump, and usually those go away.

Via Matt Strassler and Lisa Randall, here’s a set of plots that helps indicate exactly how hard this game really is. The plots can all be found on this web page at CERN. In comments Matt and Peter note that they were made by Tommaso Tabarelli de Fatis, as explained on Tommaso Dorigo’s blog. Here is the original plot from the CDF paper:

We’re looking at the number of events that produce a W boson and two jets, as a function of the energy of the jets. The bottom plot is all the data, while at the top they’ve subtracted off most of the Standard Model background, leaving only the predicted red curve from WW/WZ events. You see the extra little bump around 150 GeV, that’s what’s getting everyone so excited. It’s unlikely that the data are a good fit to the prediction; the “KS (Kolmogorov-Smirnov) probability” is given as 5×10^-5, which means that it’s not bloody likely.

But, just for giggles, let’s imagine that the energy of the jets wasn’t measured very accurately. Obviously the experimenter worked hard to get it right, and I would trust their judgment over my own any day, but you never know. Jets are complicated things with many particles in them, you can imagine being off by a bit.

So here is the same plot, just scaling the jet energy by two percent:

You see that makes a lot of the excitement go away. The KS probability is now 9×10^-2, which essentially means … the bump has gone away. If we scale by 4%, it goes away almost entirely:

Now the data fit the Standard Model very well. If we keep cranking up the supposed error, we stop fitting again, because the data at very low energies begin to go astray:

There is an animated version of the plot which is fun to look at. Nobody is saying (I don’t think) that this is certainly what’s going on; it’s just an illustration of how difficult this game really is, and why people shouldn’t get too excited about three-sigma events. A little bit of excitement is good, and descending deep into cynicism is bad, but patience is really warranted. Let’s collect more data and see what happens.