Flipping the Classroom Improves Performance in
Research Methods in Psychology Courses

 Ellen Furlong
Illinois Wesleyan University

Despite having taught it many times Research Methods in Psychology remains one of the most challenging courses I teach. The difficulty arises primarily because Methods has two major goals: (1) to teach students the required concepts (2) to be able to understand, evaluate, design, and conduct research. In short—we must teach both content (what is a hypothesis?) and skill (where is the hypothesis in this article? Is it strong? What is my hypothesis?), usually in just one semester.

The first few times I taught Methods I tackled this problem by covering content in class and relying on a semester-long APA-style research proposal for students to practice. On the surface this worked modestly—students typically wrote interesting papers with at least superficially solid ability to apply their knowledge.  

One semester I challenged my students to something new: I assigned a very short 2 page article (Kille, Forest & Wood, 2013) and asked questions about it (i.e., True or False. One of Kille and colleagues (2013) hypotheses was a rating of the likelihood that marriages of four well-known couples would break up in the next 5 years). This activity was a disaster. Although students readily defined a hypothesis or a dependent variable, almost none could correctly identify or differentiate them in the article. This revealed both a shallowness of understanding of the psychological concepts and a lack of practice applying and working with them.

I found this troubling not only for my students who would go on to graduate school or take upper level seminars, but perhaps most troubling for my students who would likely not receive more training in methods and might graduate without the ability to consume research critically. Successful consumers of research need to not only describe the concepts involved in research, but apply them readily to newspapers, blog posts, or Buzzfeed articles that they read. This is especially important in today’s age of ‘disinformation’ and false news.

In short, the problem with Research Methods is that to practice the skills involved in research, students first need to understand the concepts. And given the pressures of the semester we often don’t have enough time for them to do both.

This is hardly a new problem; others with similar difficulty have often turned to flipped classrooms (see, for example Peterson, 2016 and Wilson, 2013 who have used flipped courses for similar reasons in a statistics course). A typical flipped classroom involves presenting traditional lecture-based material (i.e., the foundational concepts) in an online video that students watch on their own before coming to class. During class students then work together under the guidance of the instructor to practice applying these concepts and honing skills (e.g., Lage, Platt & Treglia, 2000). This allows students to do the “easy” parts of learning—listening to a professor lecture, memorizing material, etc. —at home, while doing the hard parts—actually thinking about and applying the material—in the classroom with the professor’s help.

Flipped classrooms have many advantages. First, students can learn the content at their own pace because they can watch the lectures as often as they need to in order to understand the content. Second, through classroom activities, students can assess their own knowledge early, so they know what they don’t know before the exam, and target their practice accordingly. Third, because students practice their research skills in the classroom I can provide one-on-one time with them. I can offer instant feedback, can see where they struggle, and can scaffold them to success. I can correct their mistakes while they are making them, and adjust activities in the moment to ensure they fully meet my course goals.  When students practice their skills at home I may have no idea where or how they struggle.

In effect, flipping the classroom allows me to move from a “sage on the stage” to a “guide on the side”, emphasizing the skill involved in assessing and designing research rather than providing definitions and rote memorization of the jargon.

Implementing a flipped classroom is very time consuming and difficult—for every 10-20-minute video I made, I spent at least 3 hours writing a script (don’t think you can do this on the fly—you hem and haw and students feel like you’re wasting their time), creating slides, recording the video, editing it, and posting it to our course management system. Sometimes I found other people’s work that was far better than what I could have done (see Ben Goldacre’s Battling Bad Science TED Talk: https://www.youtube.com/watch?v=h4MhbkWJzKk) and that saved me hours, but for the most part I made my own lectures. I wrote online quizzes and discussion forums to ensure that students watched the lectures, and on top of all that I had to create an entirely new set of in-class activities to help my students practice their skills—the entire point of this exercise (The Society for the Teaching of Psychology  (http://topix.teachpsych.org/w/page/19980993/FrontPage), Teach Psych Science (http://www.teachpsychscience.org/),  and others have excellent resources for help on their websites). Each of these took at least another 2-3 hours to prepare, many of them much longer. In short, between making your own videos, exploring other people’s work, writing quizzes, and developing new in class exercises this is a daunting exercise, not to be assumed lightly.

However, despite the immense amount of time and effort it took to flip my course the outcomes were phenomenal and I hope that will be encouraging enough to motivate others to pursue it and, equally importantly, to motivate your students to give a flipped class a chance.

A brief word about what I will show you here—in the Fall of 2013 I taught Methods in a traditional lecture-based course and in the Fall of 2014 I taught the same course flipped with 16 video lectures spread throughout the semester. I chose to compare two fall semesters although my first time flipping the course occurred in the Spring of 2014. I did not examine this data as students in fall and spring typically differ in systematic ways (i.e., more first-semester juniors in the fall and more second-semester sophomores in the spring).

I assessed three measures over the course of both semesters: (1) applied exam questions, (2) a large APA style research paper, and (3) student evaluations of instruction scores. I chose exam questions that focused on particularly difficult foundational questions and for which there were least two questions per topic. For the APA style research paper, I randomly selected 5 student papers per class for in-depth assessment. These were scored on a scale of 1 (absent) to 6 (exceeds expectations). There was a good correlation between these scorings (r = .87) and the grading rubric I had initially used to score the papers. Student evaluation of instruction scores ranged from 1 (strongly disagree) to 5 (strongly agree) and included a number of questions that I will discuss below. Finally, because the sample size was low I accepted alpha values of .10.

T-tests revealed that students in the flipped course (F) and the traditional course (T) scored fairly similarly on most applied exam questions (Design: F 88%, T: 90%, p = .82; Hypotheses: F 81%, T 76%, p = .69; Sampling/ Assignment: F: 85%, T: 80%, p = .38; Reliability/Validity: F: 83%, T: 78%, p = .39) but for two of the hardest concepts, variables and causation, students in the flipped course greatly outperformed students in the traditional course (Variables: F: 90%, T: 79%, p = .06; Causation: F: 92%, T: 73%, p = .015).

Though this was impressive, the largest improvement showed in the APA style research papers. Interestingly students in the flipped course used evidence better (F: 5.2; T: 3.4, p = .02), had better argument organization (F: 4.8, T: 3.2, p = .05), stronger hypotheses (F: 6, T: 4.2, p = .03), better proposed methods (F: 5.13, T: 4.13, p = .03), were able to discuss their predicted findings in more profound ways (F: 5.6, T: 4.35, p < .01), and had overall better papers than students in the traditional course (F: 5.45, T: 4.5, p = .06). Students in the flipped course were also marginally better at synthesizing information across sources (F: 5, T: 3.8, p = .11). However, it wasn’t just that students in the flipped course were better writers (Writing style: F: , 4.54, T: 4.47, ns) or better at following directions (APA Style: F: 5.13, T: 4, ns) so their improvements in these areas seems targeted and important.

Student evaluation of instruction scores also told an interesting tale—students in the flipped course were more likely to recommend the course (T: 4.13, F: 4.70, p = .10) even though they found it provided a greater intellectual challenge (T: 4.40, F: 4.90, p = .06) and they found the difficulty level less appropriate (i.e., they reported that the course was too hard: T: 4.67, F: 4.10, p = .01). So even though students found the course harder they were more likely to recommend the flipped class to others compared to those in the traditional course.

While we’re talking about student evaluation scores, I will point out that my evaluation scores suffered a little the first semester I flipped the course (Spring 2014). While they dropped in some areas (i.e., students found me less available for help; thought my comments were not as useful) their overall evaluation scores stayed fairly similar (4.58 vs 4.59). Further, this ‘hit’ to my evaluations disappeared after one semester. My interpretation here is that I was frantically writing lectures and prepping in-class activities and didn’t have as much time to spend with the students and on comments. Now that all that work is done I have more time than ever to spend on my students. Since then my evaluation scores have stayed the same or risen (average 2014/2015: 4.58, 2015-2016: 4.60, 2016-2017: 4.82). Open ended student evaluations indicate that they very much valued the flipped experience and used it just as I would hope. For example, one representative comment said:

Teaching this particular material in a “flipped course” was effective. The nature of the material is generally easy to understand with previous experience in psychology but it was not always as simple to apply it; therefore, practicing application in class was helpful. Overall this fostered the ability to apply the knowledge across useful areas both in this course and other courses.

In summary, flipping the course in Research Methods is hard, but it benefits the students. While this benefit may not necessarily show up on every exam it shows where it counts—when students use their knowledge of methods to evaluate articles or design their own research. They are better able to think about important scientific controls, to design better experiments, and to keep their interpretation within reach of their data set. In short, this improves their training as scientists and consumers of research which we hope will persist throughout their lives. Though this work is hard (for both you and the students), it pays off.

I’ll leave you here with a few quick words of advice about flipping your own course: First, you don’t need to flip your entire course all at once. Consider flipping one day this semester and see how it goes. Next semester, add another. Second, borrow from people who have done this already. Raid listservs and teaching websites. Email me and I will happily send you my materials (scripts, videos, quizzes, activities, etc.) or give you a pep talk. Talk to your colleagues and share with them. Third, tell your students they will be in a flipped course and, importantly, why. Give them the data I’ve given you—reassure them that their papers will be stronger, their grades will be better, and they will be happier. They will get on board. Fourth, and perhaps the scariest for junior faculty like myself, accept that the first semester you flip, your teaching evaluations may take a hit. Know that you’re gambling, yes, but it’s on a good bet—they will likely rise higher down the road once you’ve sold your students, once they know what they’re getting by enrolling in your course, and once you have mastered the flip.

 

References

Kille, D.R., Forest, M.L. & Wood, J.V. (2013). Tall, dark, and stable: Embodiment motivates mate selection. Psychological Science, 24, 112-114.

Lage, M.J., Platt, G.J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31, 30-43.

Peterson, D.J. (2016). The flipped classroom improves student achievement and course satisfaction in a statistics course: A quasi-experimental design. Teaching of Psychology, 43(1), 10-15.

Wilson, S.G., (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology, 40(3), 193-199.

 

Biographical Sketch

Ellen Furlong is an Assistant Professor in Psychology and Director of the Comparative Cognition Lab at Illinois Wesleyan University. She received her B.A. in Mathematics from Transylvania University and her Ph.D. in Psychology from The Ohio State University. Before joining the faculty at Illinois Wesleyan University in 2013, she served as a postdoctoral fellow at Yale University. Ellen has taught several courses with "flipped" components including a survey level fully online course, a writing intensive research methods course with flipped lectures, and a team-taught, cross-institution (Illinois Wesleyan and Transylvania Universities) May Term travel course with flipped lectures and skyped class sessions.