1/9
Idea 01How to Lie with Statistics

A biased sample makes any survey result meaningless, no matter how large it is

Huff opens with the foundational statistical sin: drawing conclusions about a whole population from a sample that isn't actually representative of it, regardless of how impressively large that sample sounds. He uses the classic example of surveys conducted by mail or by phone in earlier decades that inadvertently oversampled wealthier or more educated respondents, since those were the people more likely to have phones, read certain publications, or bother responding to a mailed survey at all.

He stresses that sample size alone doesn't fix a biased sampling method — a million responses gathered in a systematically skewed way are still less reliable than a much smaller sample gathered properly at random, because the bias compounds rather than averages out no matter how many biased responses you collect.

Huff's broader point is that the credibility-boosting phrase "a survey shows" should immediately prompt the question of who exactly was surveyed and how they were selected, since that single detail can invalidate an otherwise impressive-sounding statistic entirely.

Takeaway: no sample size can rescue a survey from a fundamentally biased way of choosing who gets asked.

Reading: How to Lie with Statistics — Wisdomly