More data doesn't mean more truth

Silver opens with a paradox: the information age has produced an explosion of data, yet our collective forecasting record on the things that matter most — recessions, earthquakes, elections — hasn't obviously improved. The reason is that most of any dataset is noise, random and irrelevant fluctuation, and only a sliver is signal, the real underlying pattern.

He likens the danger to apophenia — the human tendency to see meaningful patterns in random noise, the same instinct that makes people see faces in clouds. Give a forecaster a mountain of data and no rigorous method, and they'll find spurious correlations that fit the noise perfectly and predict nothing.

His warning is aimed squarely at the modern faith that sheer data volume solves prediction problems. Volume without a disciplined model just gives you more raw material to fool yourself with — the real bottleneck was never data, it's the judgment applied to it.