KnowThis Blog Postings
The Quality and Ethics of Pay-to-Participate Market Research
There are many websites that recruit web surfers to participate in online research, such as answering surveys or participating in focus groups. Many of these sites are run by legitimate market research firms that use the Internet as a method for building research panels (a.k.a., consumer panels). Such panels consist of a large number of people who have agreed to take part, often on a regular basis, in the research conducted by the panel’s operator. In many cases the research is sponsored by marketing organizations that have contracted with research firms running the panels. Marketing organizations like panels as a research tool because these have the potential to offer quick access to many highly targeted respondents.
Research panels have been around for many years and, for just as long, research companies have offered incentives to those who agree to participate. In most cases incentives are in the form of coupons, sweepstakes, special discount offers and, sometimes, cash. The offer of incentives combined with the power of the Internet has lead to a surge in the number of people willing to participate in research panels.
One of the reasons for the increase in participants is the emergence of websites that act as intermediaries by directing respondents to a panel’s site. These paid research intermediary (PRI) sites promote themselves as one-stop shops for locating hundreds of opportunities to take part in incentive-based online research. Of course, many PRI sites also suggest that “easy money” can be made by those taking part in online research to the point of stating that someone can earn thousands of dollars a month. For their services some of these intermediary sites charge respondents a fee to establish an account before providing access to the listing of research sites.
Research Value of PRI Sites
The emergence of PRI sites has raised several concerns among marketing organizations who are contemplating using research panels. One issue is whether these sites are contaminating panels with their sometimes aggressive promotional methods. This raises the question: Are PRI sites filling panels with members drawn to participate strictly because of the incentives being offered and who have little interest in providing good information?
While appearances may suggest that PRI sites have a negative influence on research panels, from a market research perspective this may not be the case. In fact, PRI sites may provide a useful service for research companies seeking to build research panels. In a short time PRI sites have the potential to deliver a large number of participants to a panel. But the number of participants, while crucial for operating a panel, is not the sign of a good panel. Instead, any research panel that bills itself as offering quality research must be scrutinized for the type of respondents that make up its panel. To be considered a useful research tool, a panel must be representative (i.e., share the same characteristics as those in market who are not part of the survey) of the market to be researched. Quality panels must also ensure that members are providing truthful information and not just answering questions or engaging in behavior just to get paid.
To ensure quality research, operators of legitimate panels will attempt to screen those forwarded by PRI sites. The level of screening is key to whether the research panel is one of quality. The best screening seeks to confirm a person’s intentions and verify their profile by such methods as follow-up phone calls, direct email correspondence or require completion of a highly detailed questionnaire. But screening efforts are expensive leading less reputable panels to simply accept what a person enters in their profile when they register (e.g., when first directed to the panel via a PRI site). Obviously, it would seem easy for someone whose primary purpose is to make money to scam panels that do little screening by simply entering bogus information when they register.
For panels doing little screening it may be a stretch to consider what they do as contributing to producing quality market research since their base of respondents may be neither representative of the market nor offer truthful responses. It is safe to say these sites are almost never involved in highly scientific research and, consequently, the information they produce is not very useful. Instead, much of this research is used to produce “eye-candy” statistics, charts and graphs that may appear on promotional materials, advertising or websites for the company but are not grounded in good research techniques.
Thus, when it comes to research quality, marketing organizations looking to use a panel for research should not be as concerned with whether a member of a research panel was recruited through PRI sites but whether the panel is making efforts to effectively screen those directed there by a PRI site. Marketers should not use a panel that undertakes little screening if they intend to conduct good research.
Ethical Business Activity
In addition to questions concerning PRI sites value in benefiting market research, ethical issues have also been raised in regards to PRI sites charging for their services. The ethical concerns of PRI sites are arguably more concerning than whether these sites contribute to quality research since such concerns are directed at PRI sites’ own business practices. But evaluating ethical behavior can be a tricky thing since what is right and wrong differs depending on such factors as: nationality, culture, and even industry. Because the issue is a complex one, let’s evaluate the ethics of PRI sites in terms of commonly accepted standards in the U.S. Keep in mind things may be different in other parts of the world.
To understand the ethical issues surrounding PRI sites, it is useful to look to the Council of American Survey Research Organizations (CASRO), a trade group whose mission includes promoting ethical practices within the survey industry. A scan of the Membership Directory shows that it contains most of the major market research firms operating in the United States as well as many that operate internationally.
CASRO members agree to abide by the organization’s Code of Standards and Ethics for Survey Research. Reading this code should be a requirement of all market researchers, whether they are members of CASRO or not. The code includes information on market researcher’s responsibilities to clients and respondents. Some of the most notable statements include:
- Information gained from respondents should not be used “for individual marketing efforts and that no action can be taken toward an individual respondent simply because of his or her participation in the survey.”
- Market researchers are in violation of the code if they use “deceptive practices and misrepresentation, such as using research as a guise for sales or solicitation purposes.”
- In relation to survey research that recruits respondents or distributes surveys via email, the code prohibits researchers “from using any subterfuge in obtaining email addresses of potential respondents, such as collecting email addresses from public domains, using technologies or techniques to collect email addresses without individuals’ awareness, and collecting email addresses under the guise of some other activity.”
- If a researcher hires others to do the work, the research must ensure that the subcontractors “are aware of and agree to maintain and respect Respondent confidentiality whenever the identity of Respondents or Respondent-identifiable information is disclosed to such entities.”
Reading these statements, as well as the rest of the code, it would seem PRI sites are running afoul of the CASRO ethics code especially since they are making money off the participants which appears to be a violation of the no “individual marketing efforts” statement. Yet there are at least two reasons why PRI sites do not cross the line on ethical policies established by CASRO. First, these sites are not per se research sites; they are agents that bring research companies together with respondents. While their mechanism for locating respondents may be questionable since they are charging customers to participate, this itself may not necessarily be a violation of the code, especially if these sites are not violating privacy issues laid out in the code.
Second, most sites that engage in pay-to-participate methods to attract respondents are probably not members of CASRO, so technically speaking, CASRO’s ethics policy would not apply to them. This is not to suggest that these sites are not involved in unethical practices, some may be. In fact, there have been several reports in the past about such sites being spam centers through their collection of users’ email addresses or worse perpetrating fraud by gaining access to registrants’ credit card information.
Overall, reputable PRI sites are probably not performing unethical business activity and they may actually help in the market research process. The lesson for marketers looking to use research panels is to focus more attention on the quality of the panel than whether panel members were recruited through PRI sites.
Comments