Question

Dear Mitch,

Can you explain how statistics can be used to mislead?  We've heard that a lot, but my sister and I think that if the statistics come from a good source and are reported in something like the New York Times or USA Today or The London Times then they would probably be legitimate.  Are we missing something?

Robby and Sheila B.

Pasadena, CA

Answer

Dear Robby and Sheila,

When people say that statistics can be used to mislead, they are absolutely correct, even in cases where the source of the data is a very reliable one and the place it is being reported is also reliable, like the three legitimate newspapers you mention.

When things like statistics are used to mislead people, the journalist or politician or whoever it is who is doing the misleading is relying upon a common assumption that people make:  If the source of the information and the origins of it are careful, well-respected, and honest, how could they be wrong? 

The answer lies in the fact that the accuracy of the numbers (the statistics) involved and the quality of the newspaper are not the ones doing the deception; this 'deception' is usually done by a person or group motivated to prove or show something; usually it is the author of the article that uses the statistics, or the politician, who is trying to gain something (such as your vote to get into office).  That is who 'shapes' the information. 

By this ('shaping') I mean that the person is either implying or openly stating a conclusion from true facts that is not a logical conclusion, or not the only possible logical conclusion -- but is the conclusion they've selected because it portrays them in the most favorable light, or portrays their competition in the less favorable light.

EXAMPLE:  

I once heard a politician brag that when he first took office in a certain community there were only two banks serving the public of the entire area, and now that he has been in office for fifteen years, there are seven banks in the area, which is more than three times the original amount.  Therefore, he said, smiling, it is clear that he has had a positive effect on the local economy.  Otherwise, two banks would have continued to be sufficient, or close.

HOWEVER:  What the speaker left out was the fact that just about every town in that state had tripled its number of banks over that same fifteen year period, because that is how the nation grew economically, and it had nothing to do with that politician's work.  In fact, without him the town may have had an even bigger financial growth, as many other towns around his town did, though that is the kind of thing that cannot be determined because the same variables cannot be recreated years later.

EXAMPLE 2: 

"There is a strong relationship between the softness of asphalt covering a road and the number of heart attacks that occur as people walk up that road's big hill.  Therefore, soft blacktop should be abandoned in favor of a firmer type to prevent heart attacks." 

Right? Uh, no.  Wrong.

Why? 

Because here, instead of the simple cause-effect relationship that readers naturally assume, where one of the pieces of information (the softness of the road) causes the other (increased number of heart attacks), there is a third important piece of information, which is missing.  It turns out that both these pieces of information are "effects", and the "cause" was left out. 

QUESTION:  What causes blacktop to be soft? 

ANSWER:  A hot sun.  Yes, on a hot sunny day the road is softer and the air is hotter and presents more of a strain on the heart of people climbing a big hill.  So, even if the climbers were going up a hill covered with soft grass, they are likely to have a higher chance of heart attack than those who climb the same hill on a cooler day.

Hope this helps,

Mitch