650.823.3042
George W. Bush and Al Gore, Jr. ran for president of the United States. They virtually ran a dead heat. And it took the system an agonizing month and a half to sort it all out, globally showcasing a substantial amount of dirty laundry in the process. What went wrong? Who really won? How can we keep it from happening again?
The answers to these questions don’t come from the law. They don’t come from politics. Or even the US Constitution. They come from market research.
Think of the presidential election as a really big market research study. Any good researcher will tell you that the first step in conducting a successful market research project is to define your research objective in clear, measurable and actionable terms and then get the entire project team to understand and agree to the objective.
What was the objective here? Before the election, although never explicitly stated, most people would probably say the objective of our little research project (the national election) was to choose a president. But it turns out, this objective is not necessarily clear or measurable, although it has proven to be actionable in the extreme. After the election, the objective each candidate implicitly assumed shifted subtly. Bush’s lawyers based their arguments on the assumption that the objective was to win the election. Gore’s lawyers argued, assuming the objective was to discern the will of the people. Apples to oranges. Politics.
If the objective was to win the election, then the issue is simply who got the most votes within the rules of the game laid out prior to November 7. And Bush wins. Winning the election is clear, measurable and actionable. Also technical, rigid and perhaps irrelevant. What if the person winning the election is not the person most people wanted to be president? Are we happy with that? If not, if we say we want the objective to be to discern the will of the people and we also want the rules of the election to be such that the will of the people is discerned, then we are faced with some classic market research issues.
Aristotle claimed that what uniquely distinguished humans from all other animals was that humans were rational. Now, Aristotle was smart. Probably would have made a good engineer. Be he was exactly wrong. What really distinguishes humans from the other animals is that we are irrational, and sometimes, arational. That is to say, humans often behave illogically or in a way completely unrelated to logic (alogically).
The first issue is sampling error. Only about half of eligible voters actually vote in any election. Does that half accurately reflect the will of the other half? Probably not. It is often said low turnout favors the Republicans and high turnout favors the Democrats. You may say the voting half doesn’t need to reflect the will of the non-voting half. That if someone chooses not to vote, that’s his or her problem. Okay with me. But you’ve now changed your objective to discerning the will of the people that voted. And that’s really the Gore view, even though they never say it. The Bush view is one step farther out: their objective is to discern the will of the people that voted correctly (they would say legally). And we’re back to arguing about objectives (see how important it is to get that straight at the beginning?).
But the really big issue is measurement error. Measurement error is the difference between what the voter meant to do and what he actually did. So if I wanted to vote for Gore but I actually voted for Buchanan, that would be measurement error. If I wanted to vote for Gore but I actually failed to punch out a chad completely and was officially counted as a no-vote, that would be measurement error..
In an election, as in any research project, there are two types of measurement error: random and systematic. A random error affects all votes with equal probability and, therefore, should not affect the outcome. That is, it would be highly unlikely that it would affect one candidate more than the other. If all voters voted in exactly the same manner, say the old punch card system, all voter punch cards were handled in the same way and to the same degree and all votes were counted in the same machine (or at least in exactly the same way), there would still be errors in the counting. But those errors would be randomly distributed across the two candidates. The winning candidate would be extremely likely to reflect the will of the people (at least the people that voted).
But if there are differences in the way the voters vote (punch card vs. optical scanner), or the way the votes are counted (machine vs. hand), then the error terms are no longer necessarily random and equally distributed. These new error terms could favor one candidate over the other. For example, if voters favoring Bush more often voted using voting procedures yielding less no-votes than procedures that Gore voters used, there could be an error favoring Bush. Then Bush could win the election but not reflect the true choice of the people. Another example: if no-votes are hand counted by different people using different criteria, say Broward County vis-à-vis Palm Beach, then another systematic error could occur. And of course, if no-votes are counted in some counties and not others, then once again a systematic error term would have been introduced.
A brief sidebar: since there has been no substantive claim of any fraudulent or intentionally malicious behavior by either side, I will ignore dishonesty as an error source. Same for system or machine malfunction.
Why have the network computing devices that offer Web access for a fraction of the cost of a personal computer still not been embraced fully by the mass market? Familiarity is one factor. People resist true novelty. We are afraid of anything we don’t understand. Another factor may be that people relate to their televisions in a passive way. We are accustomed to letting the television do all the work. They don’t call us couch potatoes for nothing. We’ve earned that title the old-fashioned way and we won’t give it up without a fight. Accessing the Web from your TV requires an entirely new way of relating to that appliance. A way that takes more energy than we are used to spending in front of a TV (at least when it’s on). Humans are big on momentum. It takes a lot to change them. What it takes is a compelling answer to the universal question “What’s in it for me?”
I haven’t yet mentioned exogenous effects such as the networks calling Florida for Gore before precincts in the panhandle had closed, thereby discouraging up to 20,000 Bush supporters from voting at all. Or military absentee ballots where the military, not the voter, failed to get a postmark on the ballot, thus disqualifying an otherwise valid measure of voter intent. But these are just other examples of systematic error. Even inclement weather could introduce a biasing effect. Bad weather could be a source of systematic error if the weather affected voter turnout only in a predominantly pro-Gore or pro-Bush geographic area. It would be a source of random error if it affected pro-Gore and pro-Bush voter turnout equally. I’m sure you can imagine dozens of scenarios containing either or both systematic and random errors.
Those of us who have collected data for a living, whether it be in market research or any of a number of other fields, know there is always, always error in the data set. The key to getting the right answer is not eliminating all error. That is impossible. It is minimizing non-random or systematic error. Random error generally won’t mislead you, especially with a large sample size. Systematic error is much more likely to.
So what do we do here? Ideally, we would collect every vote in exactly the same way. And going forward, it’s obvious that we need to substantially revamp our voting procedures nationwide so that everyone within a given state votes in the same way as everyone else in that state. Fairness demands it.
But what should we have done with this election? With this data? Once the data is collected, it’s too late to change the data collection procedure. The cow is out of the barn, as my grandfather would say.
In market research, we recognize imperfections exist in data. We don’t generally throw out the study because of it. We first design studies with minimal systematic error. We next ensure that there is no question of fraudulent data. We assume the random error doesn’t affect the results. Next, we make every effort to clean and edit the data set to insure it is as accurate and as complete as possible. That would be analogous to doing handcounts (but everywhere and in exactly the same way). And finally, forced by practical realities, we assume, given an absence of malicious intent or mechanical malfunction, that any random or systematic errors that remained occurred equally often to both sides and cancelled each other out. At this point, we let the chips fall where they may.
Too bad politicians aren’t researchers.
Prior to founding MACRO, Mr. McCullough was the senior vice president for a leading international strategic consulting and design research firm. Before that, he was market research manager for a $300 million division of Levi Strauss & Company.
Mr. McCullough has spoken before the American Marketing Association, the Marketing Research Association and various universities on a wide range of marketing topics. He is a past board member of the American Marketing Association, San Francisco chapter, and has been quoted in Canadian Packaging Magazine. He has chaired and lectured at applied multivariate statistics seminars for several years and is a member of numerous professional associations.