The Value of Experience
Ultimately, of course, your own experience counts more than anything else. When the weatherman says there's a 70 percent chance of rain and you don't see a cloud in the sky, you know that the weatherman is wrong. It may be raining somewhere, but it isn't going to rain on you. So you leave your umbrella at home.
McBride's statistics, for example, came from a press release issued by the Bureau of Labor Statistics (BLS). When it comes to certain kinds of objective information, the BLS is a highly reliable source. Formed in 1915, the BLS has access to resources that few other statistical researchers do. The BLS can, for example, list a few of its questions in the national census.
The BLS originally submitted surveys to four major industries. But during the Great Depression, a tremendous controversy arose over how many people were actually suffering from unemployment. There were no reliable ways to find out. So, in the early 1930s, President Hoover ordered the BLS to expand its range of survey questions. In 1936, they produced comprehensive wage estimates and information for the first time in American history.
Interest in these figures was so great that the government soon added the Current Employment Statistics (CES) to collect sample data directly from employers. Thirty years later, computer technology has kicked the whole thing up to a new level. The BLS and CES now rely on computerized phone interviews, touch-tone self-reporting, voice recognition systems, and electronic data interchanges to gather information on a monthly basis. These techniques have become common practice in gathering all kinds of statistical information. Most researchers derive their statistics from a "sampling" of the population. But is sampling a reliable technique?
When you're trying to find out how accurate a statistic may be, take a look at how many people were sampled. Ask yourself what percentage of the whole that sample represents. The best-selling book The Millionaire Next Door (Simon & Schuster, 1999) offered surprising information about the "real" lifestyles of millionaires based on interviews with "more than 500 millionaires." The authors, Drs. Thomas Stanley and William Danko, are serious researchers and their careful investigation has extended over twenty years of work in the field. There is little reason to doubt their findings--within that sample.
But there were 3.45 million millionaires in the United States when they began their research. By the time the book was published, the number of millionaires had grown to 7.1 million. If you were hoping to find out what life will be like when you become a millionaire, you're out of luck. It is virtually impossible to draw any widespread conclusions from such a small sample. By contrast, monthly BLS reports sample up to 390,000 people out of 276,000,000.
William Safire of The New York Times is quick to point something else out: Not that many people respond to surveys. Whether it is a call from the Bureau of Labor Statistics, a political pollster, or a generic research facility, only 65 percent of Americans are willing to reveal any information about their lives. Experts in the field have even confessed to Safire that the real figure dips as low as 35 percent.
Furthermore, those who do respond are not necessarily representative of the whole. Just as jury pools are sometimes made up of people who are retired, unemployed or inactive, those who are willing to answer a survey may well have different characteristics than those who won't take the time to answer. Keep the basic questions in mind: Who is the most likely to respond to a survey about job changes, income, or employment status? When they do respond, how honest will they be? And what will their answers say about you?
There's no way to know for sure. And that brings us to the most important problem with statistics: They give us the illusion that we know something, that we have all the facts. But we probably don't. Statistics appeal to our desire for certainty in a world where the only thing that's certain is uncertainty itself.
