Do These Things Even Work? A Call for Research on Study Guides
J. Hackathorn, A. W. Joyce, and M. J. Bordieri
Murray State University
If one had to predict the most common question asked by students each semester, it would be: “What will be on the test?” Moreover, this question is frequently and predictably followed by requests for a study guide. As good, well-meaning instructors, many of us sigh (maybe cry a little) but ultimately provide them. In fact, many of us even include them in course materials prior to the actual request, just to avoid the conversation. Given how common these requests are, it is surprising that there is little actual research regarding the effectiveness of study guides. A quick search, using key terms such as study guides and exam guides, on Google Scholar leads to only a handful of results, many of which are dated and focused on creating study guides (as opposed to assessing them). Thus, we suddenly found ourselves asking: How much do we really know about study guides? Do these things even work?
Arguably, any strategy or aid should help students to perform better on exams than nothing. However, some of the resources that students prefer may actually hinder their performance rather than help it. For example, in a recent analysis of learning aid use and exam performance, Gurung (2004) found that students rate textbooks’ bolded key terms as the most helpful study aid to them, but that their perceived helpfulness of this resource negatively relates to exam performance. Conversely, what they rate as least helpful (i.e., active review practices) has the strongest evidence of improving exam performance (e.g. Dickson, Miller, & Devoley, 2005). In another example, a comparison of exam review styles found that, although students do not prefer traditional (i.e., student directed question and answer format) style exam reviews, their exam performance is highest when they use this style, as compared to other styles (Hackathorn, Cornell, Garczynski, Solomon, Blankmeyer, & Tennial, 2012). Ultimately, this suggests there is a mismatch between what we (perhaps both the learner and the instructor) prefer and what actually improves knowledge, understanding, and exam performance.
To increase our understanding of study guides, the authors of this essay, as well as other faculty members, recently conducted two separate studies (Cushen, et al., currently under review for publication), using the General Psychology population at Murray State University (MSU). In the first study, we conducted a small experiment using all of the sections of General Psychology offered during a single semester at MSU. Using counterbalancing and random assignment of sections, we compared exam performance following an instructor-provided concept list study guide to performance following student generated study guides. Then, at the end of the semester we queried students’ preferences and gave another brief quiz over material from the first two exams. Our results indicate that despite benefiting the most from creating their own study guides, students strongly prefer the instructor-provided guides.
In a second study, after we realized that we were making assumptions by limiting study guides to only concept lists and student generated guides, we simply asked our students to identify the types of study guides they prefer. In replication of the past studies that showed students tend to prefer the least helpful study tools, we found that students prefer that the instructor provide study guides that include a list of concepts, followed by definitions and examples of application. In other words, students prefer that the instructor create what ostensibly could be referred to as “their notes.” They prefer excerpts from the textbooks and simple concept lists the least, but prefer an instructor provided concept list style more than nothing at all or creating their own study guide. In examining their preferences, we realize that it is probably not happenstance that the least preferred study guide styles are also the styles that require the most effort from the student to actively summarize, organize, or synthesize course concepts.
Obviously, the next question is: What do we do with this information? We do not believe that we should “throw the baby out with the bathwater.” In Fall of 2016, the primary author of this essay attempted to explain to one class why she would no longer provide study guides, and she was almost the victim of a lynch mob. Perhaps that is hyperbole. Still, the students did not appear to believe that the lack of instructor-provided study guides was in their best interest. In hindsight, the instructor may have been too quick to implement this change. There is much more information needed in this regard.
In our initial experiment, we tested the efficacy of a concept-list style study guide. Basically, we used the style of study guide that answers the ever-present question: “What is on the test? What should I study?” Correctly using this style means that students have to then find definitions, create mental models, links, and organization, and create their own application examples. However, it is unclear how many actually do that. It is possible that, instead, students simply look at the list, recognize the terms, and think that they have studied enough to be prepared for the exam. Future research is needed to see exactly what students do with those study guides.
In that same vein, beyond not knowing how to properly use a study guide, it is also possible that students do not know how to create a study guide. Although it is important for students to know how to facilitate their own learning, many students have defective study strategies (Bjork, Dunlosky, & Kornell, 2013). Our participants were students in a freshman-level course, with the vast majority being first-semester freshmen. Creating a study guide, especially an effective one, is hard work and takes a clear understanding of what type of information is important. Freshmen, specifically, may struggle with this skill. For example, in a recent General Psychology homework assignment, students were asked to create a mnemonic device related to neurotransmitters. The instructor was quite surprised when many of the students created an acronym depicting an arbitrary list of neurotransmitter names. Sadly, there were no exam questions that would ask them to provide a random list of neurotransmitters. Suffice it to say, freshmen may not have a strong understanding of what it takes to succeed on rigorous college-level exams.
Unfortunately, many new college students will find, perhaps too late, that their high school strategy of simply memorizing definitions will not be as successful in the college classroom. Thus, one of the first steps toward student success may involve taking time to teach them how to create good study aids. In our experiment, we do not report the types of study guides that students self-create. We can only imagine (and have discussed at great lengths) that they are probably terrible. A cursory review across a subsample of our students confirms that the vast majority of our students fail to consistently generate examples of course content and instead provide a simple list of terms and definitions or a chapter outline in their self-created guides. However, regardless of the quality of the study guides, their exam grades are still higher when they create their own study guides. As a result, even if the instructor gives students a foundation with the concept-list style, teaching them how to improve those study guides should prove fruitful. This assumes, of course, that we can convince students to try a new, potentially more intensive and effortful, study technique that they actually utilize rather than backtracking into old habits as the exam date looms closer (Dembo & Seli, 2004).
Unfortunately, it is still unclear which types of study guides are the most beneficial. Outside of the extensive work of Karen Wood (Wood, 1989; 1993), who outlines various types of study guides and their individualized purposes, there is a dearth of information regarding which types of study guides are the most effective and in which situations they are effective. The type of study guide one might use in an introductory course where students are being given a foundation for future classes is probably very different from the guide one might use in an applied research methods course in which students are practicing a skill. Thus, much more information is needed with regards to not only the general efficacy, but also the relevance and applicability of study guides across different courses and learners.
Finally, as tends to be the case in many of our classes, students sometimes appear to dislike assignments that really challenge or require effort of them. It is probably not a coincidence that students prefer the study aids that required less of their effort. And, before we all get migraines from rolling our eyes, it is important to consider that the students may not realize that this relationship exists. As an example, in a recent end-of-semester evaluation comment a student requested the following: “I do not want to be spoon-fed the information, but it would be nice if we could be provided with a list of concepts, in order from the most important to the least important, to help us study for exams.” Clearly, this student fails to see the connection here between spoon-feeding and the study guide that they requested. Moreover, we doubt this student is alone in this desire. As such, asking students to create their own study guides may result in backlash. Importantly some, if not all, of this backlash can be reduced with transparency, communication, and rapport. However, instructors will need to assess the risk/benefit ratio of implementing a change like this.
The most surprising aspect of our research is that very few of us question our own use of study guides, even though, frankly, we tire of creating them. Many of us create these study guides because the students ask for them, or to avoid potential mutiny. Yet, as study guides have been around for so long and are so ubiquitous in higher education, very few of us inquire as to whether they work. It is important to note that this does not make us (or you) bad instructors. Care and efforts for students in any form should never be disregarded. In fact, we suspect that there are myriad instructors who have found ways to improve the effectiveness of study guides, but have yet to publish them. Thus, this essay is a mere call to action. Help us, help them; help us, help ourselves.
References
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417-444.
Cushen, P., Vázquez Brown, M., Hackathorn, J., Rife, S. C., Joyce, A. W., Smith, E., …Daniels, J. (Under Review). "What's on the test?": The impact of giving students a concept-list study guide.
Dembo, M. H., & Seli, H. P. (2004). Students' resistance to change in learning strategies courses. Journal of Developmental Education, 27(3), 2 - 11.
Dickson, K. L., Miller, M. D., & Devoley, M. S. (2005). Effect of textbook study guides on student performance in introductory psychology. Teaching of Psychology, 32(1), 34-39.
Gurung, R. A. (2004). Pedagogical aids: Learning enhancers or dangerous detours?. Teaching of Psychology, 31(3), 164-166.
Hackathorn, J., Cornell, K., Garczynski, A., Solomon, E., Blankmeyer, K., & Tennial, R. (2012). Examining exam reviews: A comparison of exam scores and attitudes. Journal of the Scholarship of Teaching and Learning, 12(3), 78-87.
Wood, K. D. (1989). The Study Guide: A Strategy Review. Paper presented at the Annual Meeting of the College Reading Association. Philadelphia, PA.
Wood, K. D. (1992). Guiding readers through text: A review of study guides. Newark, DE, International Reading Association.
Biographical Sketch
Dr. Jana Hackathorn, Dr. Amanda W. Joyce, and Dr. Michael J. Bordieri are all junior faculty at Murray State University in Murray, KY. Between them, they study everything from close relationships to inhibition in children, from sex to mindfulness, and of course from pedagogy to teaching effectiveness. Last year, the entire junior faculty in Psychology at Murray State (there are a total of eight of them) pooled their efforts to conduct a study examining a topic for which they had all complained: student demands for study guides. As a result of the study, they bonded, resulting in much happier happy hours and a very functional, albeit odd, departmental atmosphere.
