Melissa C. Rothstein
The University of Rhode Island

Matriculating into a Behavioral Science Psychology PhD program at 21 years old, I eagerly joined the Health and Alcohol Related Problems (HARP) lab to work under the guidance of Dr. Amy Stamates. Fast forward to my third year of the program, I am not just a student but the instructor of record for an advanced statistics and research methods course at the University of Rhode Island. My passion for research methods and statistics, coupled with my steadfast dedication to ongoing learning, empowers me to connect with students and foster a dynamic and engaging learning environment. Despite the occasional confusion that I still blend in with undergraduates (and, at times, get mistaken for one), I’ve honed the skill of blazer camouflage - an invisibility cloak for looking my age in academia.

I currently teach Applied Methods in Psychological Research (a 400-level course), where students undertake the challenge of crafting a psychological manuscript comparable to a published journal article. With a class size typically comprising around 15-20 students, this manageable number enables me to provide personalized attention and facilitate hands-on learning experiences tailored to students needs and interests. Throughout the semester, students engage in the collection, cleaning, and analysis of empirical data. The culmination of their efforts results in a written manuscript, which is showcased orally as a presentation at the end of the semester. The assignments provided below have been personally developed, drawn from my undergraduate experience at SUNY Purchase College, or obtained from past instructors at the University of Rhode Island. 

The Data & The Data Cleaning 

We utilize Qualtrics to administer a survey to undergraduate students, encompassing various questionnaires chosen by students, covering topics such as happiness and exercise. To enhance the students’ practical skills, I incorporate demonstrations on employing Qualtrics effectively for survey administration. This includes guidance on designing well-structured questionnaires and navigating various features within the Qualtrics platform. Students are explicitly informed that the data collected is solely for educational purposes, as it lacks approval from the Institutional Review Board (IRB) for broader dissemination. Following the initiation of data collection, students undertake cleaning the dataset generated by Qualtrics and coding the questionnaires to prepare for subsequent analyses addressing their research inquiries. Instruction covers data cleaning techniques (e.g., addressing normality, outliers, and missing data) and coding procedures (e.g., sum scores, reverse scoring). Post data cleaning and prior to conducting analyses for their research questions, students are introduced to and explore preliminary analyses, including missing data analysis and reliability analysis.

The QMRI

QMRI’s (Question, Method, Results, Implications) serves as a valuable tool for conducting literature reviews and crafting the introduction section of a manuscript. I first encountered QMRI during my undergraduate years at SUNY Purchase College, where it played a pivotal role in my understanding of scientific writing. Now, I incorporate this assignment into my own course. More details can be found here: https://www.purchase.edu/live/files/1244-the-literature-reviewpdf. Students are provided with a template to answer key questions based on the journal articles they read and cite in their introductions. QMRI’s aid in paraphrasing and summarizing, proving particularly helpful for students in the manuscript writing process. The template comprises the following components:

Q: What is the research question/aims? What is the hypothesis?

M: What is the method (participants, measures, procedures)? What are the independent and dependent variables?

R: What are the results of the experiment in lay terms?

I: What are the implications of the results? Why is this experiment important?

The IRB Protocol Form

In my experience, students often gain theoretical knowledge about Institutional Review Boards (IRB) and ethics but lack practical immersion in the process of obtaining approval for an empirical study on human subjects. Consequently, a lab assignment in my course requires students to complete an IRB protocol form for their proposed study (even though gaining approval is not required for students to be able to carry out their research in the course). Research suggests that writing protocols has the potential to function as an educational tool in various domains, such as clarifying and refining research questions, conducting literature reviews, enhancing writing clarity, and ensuring adherence to ethical principles in research (Balon et al., 2019). In this assignment, students work in small groups, typically four to five members, to collaboratively fill out a protocol form and submit it for “IRB approval.” I review the protocols and provide feedback (though my feedback is not as thorough as what the IRB would provide). This exercise encourages students to thoughtfully consider the intricacies of their cross-sectional study, providing valuable insights into the steps researchers take to achieve ethical data collection.

The Peer Review

Peer review is an integral part of the learning process in my course because it offers valuable feedback from both their peers and the instructor. This feedback includes constructive criticism, insights, and suggestions aimed at enhancing the quality of students’ work. In fact, research shows that students who were more critical of their peers’ writing tended to achieve higher grades on their own writing (Yalch et al., 2019). All students receive training on how to provide insightful and professional peer review feedback before this assignment. Upon selecting research topics and receiving instruction on providing peer reviews, students are grouped based on shared interests. In these groups, students review each other’s work in two rounds: (1) introduction and method sections and (2) results and discussion sections. Each group consists of three students, with each student reviewing the work of two peers. Written reviews consist of 1-2 pages, encompassing a summary of the research, impression of the paper, and identification of any major or minor issues. Students are also asked to post comments and use tracked changes in the document while reviewing to provide more direct feedback. Additionally, students evaluate their reviewers based on the timeliness, professionalism, and helpfulness of the feedback received. This evaluation is factored into students’ grades to account for the feedback provided and received.

The Scaffolding

Using scaffolding for a complex assignment such as writing a psychological manuscript has been beneficial for both students and myself. Scaffolding involves organizing assignments and course materials systematically to align with course learning objectives and ensuring clear communication of goals and processes to students. More information on this approach can be found here: http://www.brooklyn.cuny.edu/web/aca_facultywac/Workshops-AssignmentScaffolding-120412.pdf. At the beginning of the semester, students are tasked with formulating research questions and hypotheses based on survey topics. Subsequently, they develop an outline for their manuscripts before progressing to drafting the content. I divide the manuscript assignment into three parts: (1) a draft introduction and method sections, (2) a draft of the results and discussion sections, and (3) the final manuscript encompassing the title page, abstract, introduction, method, results, discussion, references, tables, and figures. Grading is reserved solely for the final manuscript. Feedback is provided on the drafts, concurrent with peer reviews, to guide students in scientific writing and enhance their skills. Training students on manuscript writing is timely and requires a lot of thoughtful effort. My feedback is usually centered around creating coherent writing, using scientific language, and accurately reporting statistical analyses. To further support students in their learning journey, I encourage an iterative process for manuscript development via scaffolding as described previously. After the initial drafts and feedback stages, students are given an opportunity for revisions before submitting the final manuscript.

Furthermore, the scaffolding approach extends to collaborative learning experiences, where students engage in discussions and workshops focused on key elements of manuscript writing. These collaborative sessions foster a supportive environment for peer learning. By integrating scaffolding and iterative practices, the aim is to empower students not only in producing high-quality manuscripts but also in fostering a comprehensive understanding of the research and writing processes inherent in psychological studies.

The Statistics: Guess That Test

Throughout the semester, students are introduced to a range of statistical tests applicable to addressing their research questions. Since students have already acquired the mathematical foundations for these tests prior to the class in other research methods and statistics courses, the emphasis in this course shifts towards fostering a conceptual understanding. Before taking this course, students would have taken a Quantitative Methods in Psychology course, a Research Methods and Design in the Behavioral Sciences course, and related laboratory classes. The primary focus is on ensuring that students can confidently determine the appropriate statistical test for different research scenarios, emphasizing practical applications and decision-making in test selection. To enhance conceptual understanding, I employ interactive methods such as real-world examples and class discussions to allow students to apply their knowledge to practical situations. This reinforces their ability to discern the most suitable statistical test for a given research context. Here are a couple of example questions below: 

A psychologist is interested in assessing whether there is a significant difference in anxiety levels before and after a therapy intervention within the same group of participants. What statistical test should the psychologist use and why?

Solution: Paired-samples t-test, because it enables comparisons between related measurements (pre- and post-intervention anxiety levels) within the same group of participants, facilitating the assessment of if the therapy intervention led to a significant change. 

You are investigating the relationship between stress levels (measured on a Likert scale) on satisfaction with life (measured on a Likert scale). What kind of statistical test would you run? 

Solution: Linear regression, because this analysis is suitable for predictive modeling (predicting the value of the dependent variable based on the independent variable) and analyzing relationships between continuous variables. However, given our utilization of cross-sectional data, students commonly employ terms like 'relationship' or 'association' to characterize such connections, rather than using language indicative of prediction or causation.

The Last Class: All About Graduate School 

The majority of students enrolled in this class are actively in the process of applying to or preparing their applications for graduate school. As part of the curriculum, I dedicate the last class to discussing master’s and PhD programs in psychology to raise awareness and provide insight into the various opportunities available within the field of psychology graduate programs. I delve into key aspects of the application process, including important considerations when choosing between master’s and PhD programs, crafting a compelling personal statement, and securing strong letters of recommendation. The goal is not only to spread awareness but to provide students with the knowledge and resources needed for a successful transition to graduate studies in psychology. 

In my role as an instructor, I strive to create an inclusive and collaborative learning environment where students feel inspired and empowered to actively engage in the field of psychology. Recognizing the diverse backgrounds and perspectives within the classroom, I encourage open discussions and harness the wealth of collective experiences. In addition to teaching Applied Methods in Psychological Research, I actively mentor students from my class and lab in their individual research endeavors. This mentorship extends beyond the classroom, providing students with personalized support and fostering a sense of community within the psychology department. As I navigate the dual roles of student and instructor, I remain committed to fostering a learning environment where curiosity thrives, critical thinking is encouraged, and each student feels empowered to explore the domains of psychological research.

References

Balon, R., Guerrero, A.P.S., Coverdale, J.H., Brenner, A.M., Louie, A.K., Roberts L.W. …  (2019). Institutional review board approval as an educational tool. Academic Psychiatry, 43, 285-289. https://doi.org/10.1007/s40596-019-01027-9

Yalch, M. M., Vitale, E. M., & Kevin Ford, J. (2019). Benefits of peer review on students’ writing. Psychology Learning & Teaching, 18(3), 317-325.          https://doi.org/10.1177/1475725719835070