Evaluating Company-Produced Research Reports: Is the Research Done Right?

Students studying marketing often find the most challenging and time-consuming assignments are those involving a written report. These assignments can feel especially burdensome if a good bit of information is needed on an industry, company or product. To this end, many students will turn to Google or some other Internet search engine looking for suitable material. Many times, search results will include a list of research reports produced by non-marketing research companies. That is, these reports are presented on websites of companies whose primary business is not that of undertaking research, but of selling a product or service. For these companies, the research report has become an essential marketing tool that helps explain what they do. While most people do not equate research reports with promotion, the fact is many companies use it for promotional reasons.

The practice of distributing research reports has been used for many years in scientific and technology industries. However, in the past, these reports primarily served as background supporting materials to help establish credibility for a company?s products and services. Today companies are placing these reports at the front of their promotional activity. The release of company-produced research reports is often supported with a promotional press release highlighting key findings and encouraging anyone interested in visiting the company’s website to see the full report. Though, in most cases, accessing the report requires completion of a form containing contact information that can later be used by the company for follow-up purposes.

While the type of research used in preparing a report can include such techniques as experimentation, personal observation or website visitor tracking, the majority of reports present the results of survey research (i.e., people completing questionnaires). Reports are presented in formats such as white papers, slide presentations, videos, infographics, webinars and flash graphics presentations. These reports are often produced using high-quality graphs and charts backed up by carefully created narratives that proudly emphasize the company?s strengths.

For students (and anyone else looking to use information found in these reports), it is important to understand what it is they are getting when reading and possibly citing a company-produced research report. While companies may claim the research supports their products, many of these claims may, in fact, be more fluff than substance since they are not grounded in doing research the right way. Doing research right takes a good deal of knowledge of research methods and a lot of time. By ?right? we are talking about using scientific methods that have been tested and refine over hundreds of years and hold up to statistical analysis. Unfortunately, most company-produced research is not scientific and, consequently, may not be as important as the company would have its readers think.

One of the biggest concerns when reading a research report is whether what is being presented is indeed relevant and worth considering. To be relevant, the research must overcome several obstacles, particularly regarding research validity and reliability.

  • Research Validity – This refers to a series of hurdles that determine whether the research is measuring what it claims to be measuring. For instance, if a company says its report is measuring how people prefer its products over competitors? products, is the research designed in a way that tests this?
  • Research Reliability – This is measured in terms of whether the results of the research can be applied to a wider group than those who took part in the study. For example, if a company?s research study reports on the results of a few focus groups with a total of 40 people participating is the information obtained from these people sufficient to conclude how the entire target market for a product, which may number in the millions, feels about the company?s products?

So how can a non-researcher judge whether a research report is based on sound research design? Well, there is never a guarantee that research is good (think of all the mistakes companies make introducing new products), but there are certainly clues when research may not have been done right. In Part 2, we will explore these clues.