By: Susan A. Nolan, Ph.D., MaryClare Colombo, Seton Hall University

Does COVID-19-related science have your mind reeling? Don’t wear masks! Wait, please do! You’re at higher risk if your blood is Type A! Not so fast!

Since the global pandemic altered just about every aspect of our lives, more people than ever are active consumers of science, and are noticing, in real time, the twisting turns of knowledge as it is created. In the past, the general public mostly read journalists’ interpretations of peer-reviewed articles, but now, we’re eager to see the results of every new study. Preprints, the drafts of research papers posted online before peer review, are now read by the public, sometimes without full realization that the paper has not been vetted (except perhaps on Twitter). Nonscientists are suddenly using the lingo of science, asking about samples and replications and even, although maybe not quite with this vocabulary, effect sizes. The open science movement, with its push toward increasing transparency, is making its mainstream debut!

The pandemic is a (devastatingly) perfect example of how current events, directly related to all of our lives, might drive student interest in understanding the previously murky scientific processes, and the clearer processes engendered by the open-science movement. Indeed, Grahe and colleagues (2019) argue that exposure to open science is not just a good idea, but one that fosters inclusion, helping all students feel connected to the scientific process. We can harness pandemic-related discussions about preprints and replications in the media, and use this as an opening toward discussing the full range of psychological science. And we can impart these lessons across the curriculum (see Morling & Calin-Jageman, 2020). Here, we outline ways to talk about open science in any course, including introductory psychology, and then introduce more advanced ideas suitable for a research or capstone course. This integration of open science can enhance students’ ability to think critically about science from all kinds of sources.  

Across the curriculum: An open science approach. Open science need not use the “scary” language of statistics. For instruction in lower-level courses, replication is at the center. At its core, the question is whether other researchers can repeat a study’s findings, especially one with an unexpected outcome. Indeed, Chopik and colleagues (2018) reported the benefits of instruction on replication, even at the introductory psychology level, including a better understanding of research generally and open science specifically.

Based on that premise, we’ll offer some examples of how instructors can embed open-science lessons into all of their courses in ways that encourage critical thinking.

• Initiate discussions related to behavioral observations from everyday life. What leads people to comply with public health directives such as mask-wearing, for example? How could we study this? If a study comes out, why is it important to wait for a replication?

• Initiate discussions related to news articles. Share news articles about psychology research, and ask what students can determine about the study, including the sample, from the news report. Explore with them how to dig deeper, including to find out whether this finding has repeated.

• Introduce crowdsourcing, with all of its pros and cons. When a study appears to be based on crowdsourced data, such as a community sample recruited through a platform like Amazon Mechanical Turk (https://www.mturk.com/), encourage students to consider why that sample might be better than an introductory psychology participant pool, why the self-selected nature of such a sample might be problematic, and how such methods might be unethical (e.g., extremely low pay).

• Introduce the open-science-badge initiative from the Center for Open Science (https://www.cos.io/our-services/badges). Show your students how to find out if a study they read in the news has any badges by looking at the peer-reviewed journal article. Ask students what they can learn from each of the badges, and why it’s important to have this kind of transparency.

Upper-level courses: Preregistration. For upper-level courses—whether research methods or laboratory courses—we can teach students about open science via the online preregistration process, such as the one on the Open Science Foundation’s website (https://osf.io/prereg/). Here, we’ll talk about a Seton Hall University course, Laboratory Research Experience (LRE), in which students attend a once-a-week class to learn practical research skills while also working in a faculty member’s laboratory gaining hands-on experience. Courses such as LRE teach students research basics such as finding and reading scientific sources, the steps in the research process, and how to present and write about research (Joh, 2019). Students can jump into the process at any point, charged with the typical tasks of a research assistant. In the lab, students can use open-science tools—particularly preregistration—as simple, yet powerful, classroom-to-laboratory bridges, while increasing critical thinking skills.

Students can understand two important concepts through preregistration—HARKing, or hypothesizing after the results are known, and p-hacking, manipulation of data and analyses to achieve statistical significance. Preregistration reduces these sketchy practices by detailing hypotheses, methodology, materials, and planned analyses in a time-stamped online log before data collection (Nosek et al., 2018). Students can complete a preregistration as a course project by using information from an Institutional Research Board application and asking questions of lab supervisors. In one study of a senior-level research course, students first reported unfamiliarity with preregistration, but later reported that completing a preregistration increased connections between methodology and statistics, prevented later changes in statistical decisions, and improved time and careful thought spent on research design (Blincoe & Buchert, 2020). In our own LRE course, undergraduate students took charge of the preregistration process, overseen by us (a professor and a graduate student), for an actual study; after completing this detailed assignment, they exhibited deep knowledge of the research process.

Preregistration could also be used as a classroom exercise to teach about ethical research considerations. Students could design a replication study and work through a preregistration assignment in groups. (The assignments in this case would not ultimately be uploaded to OSF.) Due to the precision required for preregistration, students must consider how study components fit together, why certain decisions are made, and what implications and restrictions arise. A discussion following the activity should highlight the importance of preregistration as it relates to research ethics and ask what students perceive as the benefit of completing the preregistration.

In summary, many aspects of open science are accessible to students at all levels of the undergraduate psychology curriculum. As the messy nature of the research process is increasingly visible to the general public, an understanding of open science can help our students to develop the critical mindset necessary to navigate the world.

References

Blincoe, S., & Buchert, S. (2020). Research preregistration as a teaching and learning tool in undergraduate psychology courses. Psychology Learning & Teaching, 19(1), 107-115. https://doi.org/10.1177/1475725719875844

Chopik, W. J., Bremner, R. H., Defever, A. M., & Keller, V. N. (2018). How (and whether) to teach undergraduates about the replication crisis in psychological science. Teaching of psychology, 45(2), 158-163. https://doi.org/10.1177/0098628318762900

Grahe, J. E., Cuccolo, K., Leighton, D. C., & Cramblet Alvarez, L. D. (2020). Open science promotes diverse, just, and sustainable research and educational outcomes. Psychology Learning & Teaching, 19(1), 5-20. https://doi.org /10.1177/1475725719869164

Joh, A. (2019). PSYC 2315/2316: Laboratory research experience [Syllabus]. Department of Psychology, Seton Hall University.

Morling, B., & Calin-Jageman, R. J. (2020). What psychology teachers should know about open science and the new statistics. Teaching of Psychology, 47(2), 169-179. https://doi.org/10.1177/0098628320901372

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114

Authors' Bios

Susan A. Nolan, Ph.D. is a professor at Seton Hall University, an author of statistics and introductory psychology textbooks, and President Elect of the Society for the Teaching of Psychology. Susan is co-editor of a new book on assessment of undergraduate psychology (https://www.apa.org/pubs/books/assessing-undergraduate-learning-psychology) and is on Twitter @Susan_A_Nolan.

MaryClare Colombo is a Master’s student in Experimental Psychology at Seton Hall University, where she is also pursuing a certificate in Data Analytics. She has several blossoming research interests including mental illness stigma and engagement in learning.