The Herald Bulletin

February 27, 2012

Emmett Dulaney: Using a convenience sample can lead to meaningless results

By Emmett Dulaney
For The Herald Bulletin

ANDERSON, Ind. — You may not know this, but I seem to be considered something of an expert in the field of printing — particularly when it comes to the HP model M4345. This distinction was bestowed upon me without my needing to attend any special school, read any books on printers or even possess so much as an inkling of interest in the topic. The honor came solely through the virtue of the most recently relocated printer in our department being now placed closest to my office door.

Without anyone batting an eye, I am routinely asked how to duplex and staple in the strangest of ways, change the size of the printable area, clear jams that could have only been accomplished with the use of Gorilla Glue and perform a myriad of other tasks that I personally would never think to undertake. Never mind that my response more often than not is “I don’t know,” the same individuals come back time and time again seeking even more guidance. Why? Certainly not for my expertise, helpfulness, or even hint of concern. Rather, they always come because I am convenient.

A similar thing happens in the realm of market research when the person doing the research chooses not to ask those who might represent real customers, but instead asks those who are easiest to ask — family, friends and so on. Known as a “convenience sample” — and occasionally termed an “accidental sampling” — this leads to results that are meaningless. The respondents are chosen merely on the basis of proximity and bear nothing in common with those who really should be asked.

For example, suppose I come up with the brilliant idea of franchising some Redbox machines and putting them in the lobby of the university dorms. Instead of asking students who live in the dorms whether or not they think this a good idea, I instead ask my family and coworkers. Since the respondents don’t represent any potential future customers, I can collect hundreds of “absolutely” and “best idea I ever heard” responses only to find out the hard way (in terms of a sizable investment loss) that those living in the dorms would never rent a single movie because they already get everything they want online from Netflix.

There can be any number of reasons for conducting a convenience sample instead of one more closely representing the true market (appropriately referred to as a “representative sample”). Sometimes it is too difficult to reach that population (which brings up the question as to how they will then find you after you commit to the business). Sometimes, there is a fear that if you ask too many people, someone will steal your idea. Sometimes, there is a fear that you won’t be understood or scoffed at and not taken seriously.

There are times when those fears are legitimate, but far too often it is just laziness or a lack of understanding on the part of the researcher. Don’t just take my word for it. I asked some of my friends and they all agree.

Columns from Anderson University’s Falls School of Business usually appear Tuesdays. Today’s columnist is Emmett Dulaney, who teaches marketing and entrepreneurship.