How to Evaluate Market Research Reports

6. Are differences between groups meaningful?

Once the research is obtained it must be analyzed.  A very large percentage of market research reports use a simple approach, called descriptive analysis, when presenting results.  Descriptive analysis is mostly limited to indicating how many participants responded in a certain way (e.g., how many liked or dislike a product).  But in many cases, to really be helpful, research should also indicate whether there are differences between respondent groups.  For instance, are North American, European and Asian customers different in how they rate a certain issue?  To compare groups requires analysis techniques that are more advanced than simply showing that more North American customers selected answer A, more European customers selected answer B, and more Asian customers selected answer C. 

But using more advanced analysis requires a valid research design that includes good sampling techniques, well-structured questions, and good respondents.  It may also require experience using advanced statistical tools.  Reports that offer indications of more advanced analysis will suggest their results are “statistically significant” when comparing differences between groups.  If this research has met the requirements of the first five questions, then very likely the information is valid and its implications are worthy of consideration.