How to Evaluate Market Research Reports

1. What was the purpose for doing the research?

The first question that should be asked examines the reasons the research was undertaken.  For studies produced by market research firms one might believe their main purpose for doing research is to generate revenue.  While it is certainly true that a profit motive exists, it would be a mistake to conclude that this is the main purpose of their research.  More likely, the purpose of a market research company’s report is to conduct good research that people want to buy.  Producing poor quality reports only leads to lower revenue as buyers become less satisfied with work produced by the market research company and refuse to purchase additional reports.

Research created by non-research companies is more likely to have an ulterior motive.  One sign that the quality of research is questionable is when a research report appears to paint a nearly perfect picture of an industry, company or product.  In such cases it is highly likely that the main purpose for doing the research is to support promotional efforts and not to produce quality information.  Research designed strictly as a promotional piece invariably leads to bias in research design.  Such reports often are constructed in ways that, intentionally or unintentionally, contain elements, such as a poorly designed survey questions or a poorly chosen group of respondents, that help sway results in the favor of researcher (e.g., own company) or a group related to the researcher (e.g., members of industry association). 

While by itself the answer to this first question is not automatically indicative of poor research, an issue arising here should make the answers to the following questions that much more important.