Amanda Mae Woodward
University of Minnesota Twin Cities

Open Science, or the practice of making research transparent and accessible, is becoming more prevalent in psychology research (Santoro, 2022; van der Zee & Reich, 2018). Journals, including Developmental Science and Psychological Science, accept registered reports and award authors badges for engaging in transparent research practices. As open science becomes more widely used, educating future researchers about the values and tools available is important. Graduate students, who have prior research knowledge, may benefit from guides and recommendations as they refine their research skills (Kathawalla et al., 2021). Undergraduate students, many of whom are just beginning to learn about research, may benefit from a more structured introduction to open science.  

Infusing open science into undergraduate courses can be beneficial to students planning to enter the field of psychology because they will be introduced to modern research methods and values. Undergraduate students who learn about open science may gain skills that will make them more competitive for graduate school, including programming and communicating research decisions effectively. Further, students may gain a deeper understanding of research workflows as well as a better appreciation of how to evaluate mixed evidence and the importance of replication.  

Of course, many undergraduate students do not go on to graduate school (APA, 2016). An introduction to open science can also be beneficial for these students. Activities that introduce undergraduate students to open science can help them refine skills, such as critical and analytical thinking skills, familiarity with software and databases, and evaluating evidence and making decisions, that are beneficial across a wide variety of careers (Naufel et al., 2018). All students, regardless of whether they go to graduate school, will come in contact with research findings in their daily lives. By helping them learn more about transparent and accessible research, these students will be better prepared to be informed consumers.

While introducing students to open science can increase students learning, it may feel like it is adding to the instructor’s burden (many of us struggle to find space to add material given requirements and needs of a single course!). To facilitate the inclusion of open science into the classroom, I have written about the methods I have tried in my courses below. They are listed by category and provide some reflection on the ease of including them in the semester.

 

Introductory Statistics Courses:

About My Course:

My introductory statistics course is a 4-credit course with a large lecture (~350 students) and a lab component. Students in this course learn both descriptive statistics and inferential statistics using R Programming. To introduce students to Open Science, I include the following: 

Pre-registration:

Pre-registrations involve describing your methods and your analyses prior to collecting or analyzing your data. There are several platforms for doing this, including the Open Science Framework (osf.io) and Aspredicted (aspredicted.org). Prior to covering inferential statistics, students in my course are presented with several scenarios, including those where the analyses are planned before data collection, those where data points are removed, or those where they are given no information. Students then discuss which type of evidence they would find more believable and whether they think sharing research plans ahead of time was a good or bad idea. After this discussion, I provide a brief recap of benefits and considerations with pre-registration and students explore the Aspredicted website. Then, I tell students that they will be expected to do a mock pre-registration for the inferential statistics we cover in class.     

Students base their mock pre-registration on the prompts for the practice problems I provide in class. Specifically, students are asked to 1) identify the research question, 2) identify the variables in the prompt, 3) describe the scale of measurement used, 4) determine the independent and dependent variables, 5) write their hypotheses, 6) identify the correct statistical test and explain why, and 7) explain what information they will base their conclusion on. 

Reflection:

This activity was relatively easy for me to include in my introductory statistics course. The pre-registration includes questions about all the information I typically want students to be able to identify. The main difference is that I am explicitly asking them to mention these pieces, rather than having an implicit expectation that they connect the scale of measurement and scenario to the statistic they calculate. I think pre-registrations help students in the course form connections between the wording of research questions, hypotheses, and analyses. 

 

Advanced Statistics Course:

About My Course:

I teach a smaller, more advanced course that focuses on using R for statistical analyses. This class has approximately 20 students enrolled who meet twice a week. Students work on a final analysis project using existing data.

Pre-registration:

For their secondary analysis project, students complete a pre-registration using the Open Science Framework template, which includes more questions. This step of their project helps them think through their specific research questions, what data they have access to, and what analyses would be appropriate for their project.

Pre-registration Reflection:

This pre-registration activity takes more effort and time than the alteration I made to my introductory course. However, it makes grading their final projects much easier and has led to more student meetings about the analyses they choose. Because this step is due before their data analyses, it gives us time to discuss different approaches to analyzing their data. It also makes them think through how to use the data (e.g., what should they do if they have missing values? Do they want to use summed scores or another approach?). By including this activity, I have shifted some of the work required to grade their final project to the beginning of the semester and I have noticed that their final projects tend to be of a higher quality. 

Data and Code Sharing:

In this course, there are several activities related to sharing code and data. First, I model sharing code and data by making all the course notes available via GitHub. I post the “blank shell” notes at the beginning of the class, and “commit” updates of the notes as we complete each learning outcome so students can see the updates in real time. Students in this course are expected to hand in their weekly assignments on GitHub so that they get the same experience of using GitHub. As part of their final project, students are expected to share the code they create and the necessary data on either GitHub or the Open Science Framework.

Beyond assignment submissions, students in this course are also expected to evaluate each other’s code and to provide feedback to code posted on GitHub. This helps them think through what is needed for code to be reproducible as well as to think through ways to make their own files more easily accessible.

Data and Code Sharing Reflection:

Teaching students to use GitHub takes more time and an understanding of how different operating systems work. However, there is very helpful documentation to get started on GitHub and detailed instructions for syncing R Studio with GitHub. Further, there are ways for students to take steps, rather than fully integrating their work with GitHub. For instance, they can download files from GitHub and then reupload it by pointing-and-clicking. Overall, students have appreciated the opportunity to learn about GitHub, even if it was challenging.  

 

Advanced Research Methods courses:

About the Course:

I teach an advanced undergraduate course on open science research methods. This course is approximately 10 students who meet twice a week. In this class, students learn about different aspects of open science and focus on applying what they learn to a replication project across the semester. 

Data code and sharing:

Students in this course use available materials to assess the methods used in a study. For instance, they can look at the study design and see how the variables were operationalized. Shared materials also allow students to understand how researchers transform raw data into the forms we often use in analyses because they can walk through the data cleaning code and the analysis script.

Shared materials also facilitate students completing a replication project in their research methods course. My students are currently completing a replication project through Project CREP, which offers a great set of resources on the Open Science Framework to facilitate this process. Students in my course have used these materials to 1) create a pre-registration, 2) to develop a Qualtrics survey to collect data, and 3) have started using available data to complete analyses in R or in JASP. 

Reflection:

Admittedly, this course is very easy to include open science topics in because it is the nature of the course. Students in the course have mentioned enjoying the activities mentioned above and have found connections between what they are learning and in their other courses. As the semester has progressed, I have seen the quality of student evaluations of open materials improve.

 

Content Courses: 

Though I have not included open science explicitly in my content courses, I believe some activities, like discussing replications and using open data, could be beneficial. Below are two examples related to cognitive development that I plan to use in the future. 

Discussing Replications:

Students in this course will read an article describing a failure to replicate (Oostenbroek et al., 2016) and a response from the original study’s author (Meltzoff et al., 2016). They will be prompted to think about the evidence presented in each paper and to identify factors that could have led to different results. The discussion would continue by having students think what other evidence they may need and how these papers relate to theories concerning imitation and social learning. 

Open Data:

There are several open data sources that include visuals. For instance, Wordbank (http://wordbank.stanford.edu/) is an open data source that allows you to examine children’s vocabulary growth around the world. An activity asking students to look at overall trends and as well as trends in specific groups would be one way to highlight a benefit of open data.

Conclusion:  

There are many ways to introduce students to open science as part of our courses. Introductions can be short or more in depth, depending on instructor preference and the amount of material covered in a semester. Including these activities across my courses have led to fruitful conversations about cognitive development and about the methods and statistics we use.   

 

References 

APA (2016, February). By the numbers: How do undergraduate psychology majors fare? Monitor on Psychology, 47(2), 11. 

Kathawalla, U. K., Silverstein, P., & Syed, M. (2021). Easing into open science: A guide for

graduate students and their advisors. Collabra: Psychology, 7(1).

Naufel, K. Z., Spencer, S. M., Appleby, D., Richmond, A. S., Rudman, J., Van Kirk, J., …

Hettich., P. (2019). The skillful psychology student: How to empower students with

workforce-ready skills by teaching psychology. Psychology Teacher Network, 29(1).

Retrieved from https://www.apa.org/ed/precollege/ ptn/2019/03/workforce-ready-skills

Santoro, H. (2022, January). Open Science is Surging. Monitor on Psychology, 53(1), 1.   

van der Zee, T., & Reich, J. (2018). Open Education Science. AERA

Open, 4(3). https://doi.org/10.1177/2332858418787466