Evaluating the Accuracy of Reported Information
We are inundated by information. One example of information overload is the stream of US election polls as if we are following a horse race. The Word Bank, the World Health Organization (WHO) and numerous UN agencies provide information regarding poverty, hunger in the world, access to clean water and sanitation as well as the incidence of malaria. A look at the NY Times any day will provide endless information. On a single day, they reported that 300 million people in India live without electricity and that India’s annual per capita carbon dioxide emission is 1.7 tons. What can we do with this information? Are the election polls useful? How do they affect the electorate? Importantly, how accurate is all this information?
Governments have been collecting data on their citizens for many centuries. The Egyptian Pharaohs conducted a census to find out the available labor force to build the pyramids, and in the Roman Empire, the five-yearly census was all about finding out who was available for military service and what wealth existed to be taxed. The ancient Babylonians collected data from their citizens nearly 6,000 years ago to understand how much food was required to feed their population. In the Bible, it says that Moses counted males who have reached the age of 20 and are able to bear arms.
DATA FROM POLLS
As a result of current interest, let’s examine election polls and data from some UN agencies. All polls depend on random sampling. If successful, a poll presents a snapshot at one point in time. Polls are getting more difficult to do. In the past, sampling was done by telephone on land lines, with repeated calls if necessary. Today the response rates are way down, to about 10 per cent. Those who respond may not be representative. Currently, about 40 percent have only cell phones with no land lines. So sampling is done some on land lines and some on cell phones. US federal law prohibits automated dialing devices to call cell phones. Numbers have to be dialed by hand, which is more time consuming and more expensive.
Households that use only cell phones tend to include minorities and younger voters and occur more frequently in metropolitan areas. Men are more likely to be cell-phone-only compared to women. How to take that into account in the sampling? Other difficulties in polling include the effect of the wording of questions and the truthfulness of people’s responses. Polling has had accuracies as well as inaccuracies.
WHEN POLLING GETS IT WRONG
A famous case was the Literary Digest poll in the 1936 presidential election between Roosevelt and Alf Landon. A random sample of telephone users was chosen. The result of the poll was a prediction of victory for Landon. Of course, Roosevelt won. The reason for the result was that the pollsters used data from land lines. Land line owners were then more Republican than Democrats.
Another failure was the presidential election in 1948. Major polling organizations predicted a landslide victory for Dewey. Truman won the election. Some may remember a picture of Truman with a big smile holding up a newspaper with the headline that Dewey won.
More recently, in the 2015 election in Britain, pollsters missed the result of the win by the Conservatives. Also in 2015, polls in Israel failed to predict the victory of Benjamin Netanyahu.
It is not clear how polls influence voters and politicians. We think polls have a negative effect. Instead of concentrating on serious issues, polling tends to enhance the popularity contest.
INFORMATION AND REAL LIFE
In 2015 the UN General Assembly voted to extend the Millennium Goals to the Sustainable Development Goals for the next 15 years. The plan consists of ambitious goals, such as eliminating crushing poverty, hunger, providing clean water and sanitation, improving maternal health and climate action. Development goals require reliable indicators and must be measurable so that progress can be monitored. Most of what we think of as facts are actually estimates.
We know less than we think we do. Around 1.2 billion people live in extreme poverty – maybe, maybe not. According to WHO, malaria deaths fell by 49% in Africa between 2000 and 2013. Perhaps. Maternal mortality in Africa fell from 740 deaths per 100,000 live births in 2000 to 500 per 100,000 in 2010. We are not sure. We have actual numbers on maternal mortality for just 16% of all births and on malaria for about 15% of all deaths. For six countries in Africa, there is basically no information at all.
According to WHO, more than two-thirds of the world’s population lives in countries that do not produce reliable statistics on mortality by age, sex and cause of death. WHO is perfectly aware of the inadequacy of available data.
What is much needed is support for countries to have strong health information systems. Without good data, we are working in the dark. Bad data are a recipe for bad decisions. Governments and agencies need reliable data to know where to put their money and effort. They also need to know if what they are doing is actually working.
The UN is very much aware of the need for improved data and has called for a “Data Revolution.” An independent advisory group of over 20 international experts has proposed methods to improve data collection. Their report, “A World That Counts,” has many suggestions for how to proceed. It will require increased funding. It is our opinion that people in their own locality must be helped and trained to establish and monitor the gathering of necessary data.
A timely quote from Artemus Ward:
It ain’t so much the things we don’t know that gets us into trouble. It’s the thing we know
that ain’t so.