By: Barnard C. Beins, Ph.D., Ithaca College

In choosing potential dating partners, women are more selective than men, right? Did we actually need research to know this?

Psychological research, especially research that finds its way into mass media, often evokes the dreaded response by non-psychologists that the outcome is so obvious that there was no need to conduct the study. This is nothing new. Over a century ago, E. C. Sanford wrote that “Somebody has defined psychology (with a touch of cynicism) as the science of what everybody knew beforehand, anyway” (1906, p. 118). The exercise I describe here is designed to illustrate to students that they need to go beyond obvious and cursory explanations about research findings.

To start, we should ask how obvious our results really are. Although we clearly have some insights into human behavior, our ability to predict outcomes in advance may not be as good as we would like to think.  For example, consider the possible outcomes listed below from five studies that I have taken from the published literature. (These are not “Gotcha” types of findings for which one would learn to pick the “less obvious” outcome.) Can you pick out the actual result (A, B, or C)?

I ask my students to identify the actual outcome; I subsequently group them in pairs to come up with an explanation for the results. As it turns out, they can come up with plausible reasons for the answer they choose, even when they are wrong. This exercise serves to let them know that predicting results is not as straightforward as they might think and that results might be obvious only in hindsight.

Available Materials

These materials that follow are part of my presentation at the 2020 Eastern Psychological Association convention. The entire set of scenarios, reference citations, and classroom materials (along with the presentation itself) are available on my website: http://barneybeins.com/epa/epa2020.html

Select the Actual Outcome

A	B	C
Short tests in school lead to worse performance.	Test length is unrelated to test performance.	Short tests in school lead to better performance.
Women wearing lipstick while taking a test score lower than women not wearing lipstick.	Women wearing lipstick while taking a test score about the same as women not wearing lipstick.	Women wearing lipstick while taking a test score higher than women not wearing lipstick.
People with high cognitive abilities (i.e., smart people) are less likely than others to form social stereotypes.	People with high cognitive abilities (i.e., smart people) are just as likely as others to form social stereotypes.	People with high cognitive abilities (i.e., smart people) are more likely than others to form social stereotypes.
When looking at reviews of online products, people pay more attention to how many people reviewed a product than to the rating of the product.	When looking at reviews of online products, people pay about equal attention to how many people reviewed a product as to the rating of the product.	When looking at reviews of online products, people pay less attention to how many people reviewed a product than to the rating of the product.
Women spend less time making eye contact with men they know are gay than with men they know are straight.	Women spend about the same amount of time making eye contact with men they know are gay as with men they know are straight.	Women spend more time making eye contact with men they know are gay than with men they know are straight.

How Well Do Students Perform?

            When I ask my students to give their best guesses on an expanded set of 15 studies, they are invariably at chance levels. The level of accuracy is pretty constant across sections and semesters.

            I should point out that students do score relatively well with some scenarios. For instance, 75% of students correctly predicted that people undergoing stress are likely to gain weight. And they were correct 57% of the time when speculating that the sleep of people with insomnia improves when they take vitamins. I point out to them that the purpose of this exercise is not to embarrass them, but to show the difficulty of our task as psychological researchers. In addition, when I have demonstrated this for Ph.D.-level psychologists, they were no better than students in their guesses.

            As such, why are outcomes so “obvious” after you know what actually happened? Our job as researchers (and as people) is to construct a good story about why people behave as they do. It is pretty easy to construct a plausible story in each of the scenarios above, (although as psychologists we call them interpretations) regardless of the direction of the outcome. Solid interpretations can be very convincing, even if you are describing an outcome that is opposite to the real result.

Goals of the Exercise

            I use this activity in my Research Methods class, although it is also effective in an introductory class. I like this demonstration because it helps students understand three important points. First, predicting research outcomes is difficult, although experts may be right more often than others in their specialty area. Interestingly, it appears that expertise is fairly narrow; when experts discuss matters even a little outside their specialty, their knowledge pretty quickly becomes similar to that of a layperson (Pigliucci, 2010).

We do research because we want to know with a level of confidence what people do and why they do it. And we won’t know that until we actually do the research.

            The second point is that we can come up with plausible explanations, no matter how research turns out. But just because we can generate an explanation, it doesn’t mean that we actually understand the phenomena we are investigating. After all, we can construct compelling stories that are the opposite of what actually occurred.

            A third, related point is that for most research results, there are multiple possible explanations for the outcome. When students generate explanations, they often generate different reasons to explain the same predicted results. This points out the need to plan further research so we can identify the best explanation and, as with most complex behavior, why we need to pay attention to combinations of variables. Students are often quite good at asking “What would happen if we manipulated this or that variable?”

How Well Did You Do?

How well did you predict the outcomes? The table below shows the actual outcome and the percent of my students who picked the real result.

Actual Outcome	Percent of Students Answering Correctly
Short tests in school lead to worse performance.	20%
Women wearing lipstick while taking a test score higher than women not wearing lipstick.	16%
People with high cognitive abilities (i.e., smart people) are more likely than others to form social stereotypes.	20%
When looking at reviews of online products, people pay more attention to how many people reviewed a product than to the rating of the product.	25%
Women spend more time making eye contact with men they know are gay than with men they know are straight.	14%

            Finally, to address the issue I posed at the beginning about women being more selective than men in choosing a dating partner, maybe the answer isn’t so obvious. If you change the parameters of the setting, women and men seem to show similar levels of selectivity (Finkel & Eastwick, 2009). This is why we do “obvious” research.

References

Finkel, E. J., & Eastwick, P. W. (2009). Arbitrary social norms influence sex differences in romantic selectivity. Psychological Science, 20(10), 1290-1295. https://doi.org/10.1111/j.1467-9280.2009.02439.x

Pigliucci, M. (2010). Nonsense on stilts: How to tell science from bunk. University of Chicago Press.

Ryan, R. M., Bernstein, J. H., & Brown, K. W. (2010). Weekends, work, and well-being: Psychological need satisfactions and day of the week effects on mood, vitality, and physical symptoms. Journal of Social and Clinical Psychology, 29(1), 95-122. https://doi.org/10.1521/jscp.2010.29.1.95

Sanford, E. C. (1906). A sketch of a beginner's course in psychology. Pedagogical Seminary, 13(1), 118-124.

The British Psychological Society has a digest in which they provided a list of 10 counterintuitive research findings: https://digest.bps.org.uk/2014/12/01/10-of-the-most-counter-intuitive-psychology-findings-ever-published/

Authors' Bio

Barney Beins is Professor of Psychology at Ithaca College. He is the author of Research Methods: A Tool for Life (Cambridge University Press) and co-author of Effective Writing in Psychology: Posters, Papers and Presentations (Wiley) and The Worth Expert Guide to Scientific Literacy (Worth Publishers). He has taught at Ithaca College since 1986. He is a Fellow of the American Psychological Society, the American Psychological Association, the Eastern Psychological Association, and the New England Psychological Association. He also served as president of the Society for the Teaching of Psychology and of the New England Psychological Association. He is the 2020 president of the Eastern Psychological Association.