When It Comes to Studying In Med School, Think ‘Retrieval,’ Not Review
When you’re struggling in med school, it can feel like you’re jumping from one study strategy to the next, hoping that something sticks. While there is no “one-size-fits-all” study method, retrieval practice could be the key to getting more out of your study sessions.
Skip Review and Focus on Retrieval Practice
In this podcast episode, host Ryan Orwig sits down with Dr. Jim Culhane, Assistant Dean for Student Academic Success Programs and Professor of Pharmaceutical Sciences at Notre Dame of Maryland University School of Pharmacy. In part one of the two-part miniseries, they analyze the benefits of retrieval versus other, more passive study methods.
Ryan and Jim examine why traditional study methods, such as rereading, can yield low results, and explore how retrieval practice has been proven as an effective study strategy.
Subscribe to the STATMed Podcast!
Announcer: Welcome to the STATMed Podcast, where we teach you how to study in med school and how to pass board style exams. Your hosts are Ryan Orwig and David LaSalle, learning specialists who have decades of experience working with med students and physicians. This episode is the first of two in which Ryan sits down with Jim Culhane from Notre Dame of Maryland University School of Pharmacy. They talk about one of the most important study skills for medical students and professionals, retrieval practice. What it is, how to do it, and why you absolutely should.
Ryan Orwig: Talk about retrieval practice, the importance of it, why it works, some of the history, some of the evidence behind it. And then you can also talk about some of the, where people get hung up on it and then we’ll go through a few of our favorite ways to have our students engage in retrieval practice.
Announcer: Here’s Ryan.
Ryan: Hey, welcome to the STATMed Learning podcast where we talk about learning in med school and in medical education. I’m Ryan Orwig, learning specialist and creator of the STATMed Learning program. With me is my friend and colleague, Dr. Jim Culhane, the Assistant Dean for Student Academic Success programs and a professor of Pharmaceutical Sciences at Notre Dame of Maryland University School of Pharmacy. So Jim, welcome to the podcast, thanks so much for coming on board with me. Why don’t you tell my listeners a little bit about your background.
Jim Culhane: Ryan, thanks so much for having me. It’s a real pleasure to be here today and I always enjoy having these conversations with you and that’s why it’s really great to have a chance to chat today about our topic. So I’ve been in pharmacy education for over 20 years now. My PhD is in pharmacology and toxicology and I’ve helped to launch two different schools of pharmacy over the first part of my career. I joined the administrative team here at Notre Dame of Maryland University as the founding chair of pharmaceutical sciences back in 2008 and served in that role up until last year when I assumed a new position as the Assistant Dean, Academic Success programs here. The, my move into that position was largely motivated by my relationship with you that we’ve developed over the last 10 years. And some of the really great things that you’ve shared with me and my students and the faculty that I work with, about evidence-based learning strategies and how and unique way that you actually apply those to help your students learn more effectively.
Ryan: Yeah, I mean, we met, it must’ve been about 10 years ago. I came up there, up to Baltimore and started working with your faculty, teaching them better methods for teaching and for guiding their students and for hopefully building better lectures. So then, we started teaching our version of our STATMed class with students there. And you know, every year I’m up there, where we’ve always just talked about what strategies are the best strategies? Where are the pitfalls? Where are things breaking down? How do we unlock better learning strategies for your students, for my students, for all these students, right? And that’s sort of where, how we’ve got to know each other, I guess, right?
Jim: Oh, absolutely. And I, you know I think you and I both share a real passion for helping students be successful academically. And I think that that’s really something that we share in common a lot.
Ryan: Yeah. Yeah and I think you and I are both interested in systems, methods and systems and how can we give really smart, really motivated students, these professional students, how can we give them tools so that they can be self-empowered and they can unlock their ability. So that we’re not just helping them plug holes, but actually building systems themselves so that they can take that into the next semester or take it into the next year, take it for the rest of their career. Because these people should be, our students should be lifelong learners and nobody wants to be inefficient and ineffective.
Jim: Right. Right.
Ryan: And, you know, it’s like the big catchphrase, like nobody teaches smart students how to study. And they get to these doctoral programs, these med school programs, these PharmD programs, these veterinarian programs. And, you know, there’s a huge percentage of these students that they don’t know the best way forward. So, you know today, we’re gonna talk about that central, one of the big bookend– For me it’s like a bookend piece, for others it might be more of a central piece, for any high-end learning activity, which should, it should be around about retrieval practice. So today, we’re going to just talk about retrieval practice, the importance of it, why it works, some of the history, some of the evidence behind it. And then you and I will also talk about some of the, where people get hang up on it and then we’ll go through a few of our favorite ways to have our students engage in retrieval practice.
Jim: That sounds great. Big fan of retrieval practice for sure.
Ryan: Yes. Yes you are. So, I don’t know, maybe give us a little bit of the background and your take on the background of this. What we know about it, give us a little like mini off the top of your head TED talk.
Jim: Sure. Yeah, I’ll do my best. So it’s really interesting. My first real exposure to retrieval practice as an evidence-based learning strategy was actually from you Ryan when you came to do some faculty development for our faculty and even working with some of our students. I’d never heard the term before. Although when you first explained it to me, it made a lot of sense and I recognized that many successful students are kinda doing this naturally, this self testing that many successful students do when they, as they’re learning. The, you know and as you and I began to work together, kind of in those early years, my interest peaked in some of these things. I began to do a deep dive into the literature, the cognitive psychology and educational psychology literature to try to learn a little bit more about some of the science and research that’s been done that supports the use of things like retrieval practice. And then to think about how I could operationalize this or systematize it like we talk about, so that I could teach my students how to do this in an ineffective way. So, you know what I’ve learned, is that retrieval practice is one of at least a half a dozen powerful evidence-based learning strategies that students can tap into and faculty can use in their teaching to help with learning and helping students to build what I refer to as a durable working knowledge base. So, retrieval practice is, my understanding about the history behind retrieval practice, is it’s built, the term really comes from a broader term called the testing effect. And this was something that was observed and essentially, quote unquote, discovered over 100 years ago. So it kind of blows my mind that this testing effect has been known about for so, so long but yet in higher education at least and in pharmacy education in general, it’s just really starting to permeate the programs across the country and faculty.
Ryan: Yeah, lemme yeah, I’ll interrupt you and then see if you can get back on your Ted talk, of course I’m throwing you on top of here. So, but yeah, I mean, this is the thing, right? It’s not just the PharmD programs, it’s med school, it’s middle school, it’s all the way down. Like I was talking to my daughter about that, she’s 13. And I’m talking to her about the difference between passively rereading stuff, as opposed to engaging in attempts at recall. And she’s like, well yeah I need to go, you know, this is before the lockdown and everything. She’s like, I need to get together with my friend to do this, to quiz myself. I’m like, no you don’t. So we sort of go through all this. And she was like, oh wow, that makes so much sense. And, you know, most kids are not that interested in listening to their parents, talk about learning theory. But, you know, it’s like, it’s not complex but it’s that blind feedback, I think that’s part of it. But we don’t teach students, smart students, we don’t teach anybody really the best practices. I think middle school would be a fine time to start introducing.
Jim: Oh, absolutely, I couldn’t agree more. I mean I think that this is one of the huge, you know, I wouldn’t say huge failing, but I think, I’d like to turn it and make it more positive and say that I think this is a real opportunity area for primary and secondary education to really pick up on this and teach students this stuff early on.
Ryan: Well, early and then woven throughout.
Jim: Woven throughout the curriculum, yeah.
Ryan: You’re right. And that’s really nice of you to be positive. I’ll take the other side of that. I’ll be super negative about it. I will say it’s a massive cultural failing. It is a massive cultural failing that we’re not teaching students how learning works, the mechanisms of learning. You know, we’re seeing this now, like when I talk to students, mostly med students, physicians contact me, now they have largely, I would say 50% of them have at least heard of retrieval practice. Oh, I need to do that more, I wanna learn more about it. Now look, 10 years ago, no like, no, when I was presenting to you guys. So we’re seeing a shift there in the last 10 years. I think there are a lot more people talking about it, I think books like “Make It Stick”, highly recommended. If you’re gonna read a book on it, like that book right there, that really helps make a fence post that we can all culturally build around. But no, I absolutely feel like such a massive cultural failing up and down the line kind of education. But think about this, so think about how far things have come with the science of like exercise and physiology in fitness.
Ryan: If I wanted to like put on X amount of muscle mass or if I wanted to lose X amount of weight, you can do that. I can find somebody here in my community in West Virginia that could take me through that, either at the gym or a local person through social media, right. Think about, let’s talk about movies for a second, cause we’ve talked to her for several minutes and we haven’t gotten into any pop culture yet. So, think about the idea of say they hire, you know, Chris Pratt to be in Guardians of the Galaxy. And he was sort of this shlubby, chubby, like laughable, like goofball from Parks and Recreation. Then all of a sudden he turns around and he’s absolutely ripped and jacked for this movie.
Ryan: Obviously I have to like give them to the obligatory like shirt off sweaty cut muscles. I worked for this, lemme put it on screen, right, like, and they can do this, they can say it’s gonna take X amount of time to do this. Now, think about where that was in say 1980. Okay? 40 years ago. But, so back then they were making Return of the Jedi and they tell Carrie Fisher, hey, heads up, you’re gonna be in a metal bikini for first quarter of the movie. So, and they, you know, kinda look at her and they’re, you know, obviously not being very politically correct or sensitive to her feelings. Like, you need to lose some weight. So, did they say here’s a dietician, here’s the exercise platened up? They literally said, eat a lot of celery and they handed her two leg weights, like the sand bag leg weights.
Ryan: And they were like, get ready for this role in a few months. Like, so think about how far we’ve come as a society as far as like aesthetically, in movies physically with our weight and conditioning, right? But we’re in the same place, well a little different like we said, a little more awareness. But up to 10 years ago, we were in the same place as far as like awareness of best practices for learning and studying. Which brings me all the way back to you talking about this whole testing effect, discovered, identified over 100 ago.
Ryan: And, the fact that it could be discovered this back then and here we are and it’s not a foundational thing that we’re teaching everybody. So, oh, sorry about that, tangent. But, so, I dunno, throwing it back to you, right? So we, it comes from this idea of the testing effect showing validity in this idea of retrieval practices, this idea that attempts at recall, improve the learning, right? So we’re seeing this as far back as then, then what can you tell us?
Jim: Yeah so, I think that, you know, as I take a look at the literature, I think that there were a lot of seminal studies that were done in the 1980s and ’90s that brought a lot of this, moved a lot of this theory forward. And so it was, I think it was investigated more deeply and from very different perspectives. And as you pointed out, you know, Make It Stick was one of those seminal books, like say if you haven’t read it, you should read it, it’s really great and it talks a lot about this and much better than I can. But, you know, I think it’s been a process over several decades of really doing more deep investigation into things like retrieval practice, how it can be applied in the classroom both from a teaching and learning standpoint and then really getting the word out. And that’s, I think, that’s been the challenge. And like you said, we’ve moved the needle quite a bit over the last 10 years, but I think we still have, you know, quite a bit of distance left to go. And I always consider myself, you know I’m not a researcher in this area, like say my background is in pharmacology and toxicology, I’m a scientist essentially. But, I think that, I like to think of myself as a cheerleader for evidence-based learning strategies in a lot of ways that just promoting it. The more people that we can get the message out to and get talking about this and thinking about it, the better off our students are gonna be.
Ryan: And that’s how and therefore our society.
Jim: Yes, absolutely.
Ryan: I mean and this is where I think our passions are. It’s like let’s empower our, these potential engines and give them fuel. And the fuel is the learning methodology. Cause if you give people the ability to learn anything, to optimize their ability, then we’re gonna have, the rising tide will lift all ships. So, whenever we think about teaching retrieval practice, I mean I think you can, you know, we have some videos on the web, on the STATMed Learning blog website, on the YouTube channel, you can go and learn about it’s all kinds of stuff on the internet about it now. But just watching a video here and there or maybe hearing me talk about it or hearing you talk about it, our students aren’t going to automatically take to it like the proverbial duck to water or whatever it is. It’s like, okay, they understand it but they’re going to often default back to their sort of preset, their original settings, right. So to me, I think the problem is, I look at it like a binary situation, it’s probably over simplifying, but that’s fine. Where I think, on the one hand you have review-based learning. And to me, I’ve turned in my lexicon review into a bad word. This idea of review is any time you have your eyeballs looking over something you’ve already studied. And it’s passive, it gives that sort of, that sense of familiarity, it entrenches familiarity, it feels productive, that trap of productivity, but you’re not really building anything at that point.
Ryan: Once you’ve already studied it, once you’ve already read it, whatever that might be. Once you revisit it and it’s review-based, right? So the opposite side of review to me is this idea of retrieval practice. That’s where it fits for me in that sort of simplistic paradigm. And I like teaching simplistic paradigms and then building outward from it from a framework. But yeah, the idea of retrieval practice is anytime you attempt to recall something that you’ve already studied and allowing for failure, right? You allow for failure and then you self-check it. And that’s where you see either, oh, I validate the stuff that I recalled correctly, or, oh, I’m actually fixing the stuff that I got wrong cause I’m actually correcting it. Now, to me, the biggest problem, I want you to think about maybe what you think the biggest problem is with teaching it. For me, the biggest problem is, it’s a blind improvement when you self-test. So to me, it’s like, oh, like I say I’m trying to recall 20 facts, nice round offer. And I remember eight, I didn’t get 12. Oh man, it’s terrible. It’s less than half, right. So then I self-check, okay those are the eight I recall and one of em was a little off, I fixed it, I’m fixing it, quote unquote, in my head by seeing it. And then I’m reaffirming the 12. Now what’s happening there is you’re solidifying the eight things you recalled and then you’re increasing. Maybe you’re not gonna remember all 12 but you’re increasing what you’re going to recall. The only way to prove it, is to do another self-test. So there’s not like a ticker tape print out, like that prints out of your head like, congratulations you have improved by 46% and you don’t get that, right. It’s a blind improvement. So to me, I think that very baseline primal behavioral-like gear that grinds is that you don’t get that immediate feedback. So, I’ll have students in our STATMed class use a very specific form of retrieval practice we set em up with. And you say, you have to do three self-tests on the same thing. And invariably they want, like my students they usually say you jumped five times and you usually jump 10. But for this, it’s just like, they, I didn’t get to that part. But every now and then though they will do it three times. And then they will come at you, like fanatic. Like they have like a zealot who has been on the verdict. Hey hey hey, I did that thing on the vaginal infections, I did it three times. I’m like, good job, that was, what I told you to do. They’re like, no, it works. I’m like yeah, yeah, like you don’t get it. I’m like, well, okay. Are you gonna, you know, like, I’m, I am excited, I like when my students validate all the stuff I’m teaching em. But, it’s funny because they’re so reluctant to do it because you take a beating, right. It’s like, I studied this stuff and I’m at my first real attempt to recall, cannot, it might not go well at all. It might go horrible. But then you self-check, which is a part of the circuit and then you get the blind improvement. You only find out if you do another self-test. So anyways, so that circuit I’m talking about is where I think it breaks down for most students. What is another problem or similar problem that you might see with us being the expert saying, please do this my students. And of course, for those of you listening, we’re teaching methodology, we’re teaching rationale. We’re not just like throwing it out there, kinda like we are right now. But, what do you think is one of the bigger problems of getting our students, our PharmDs, our med students to really engage in this.
Jim: Yeah, that’s a great question. And it’s really the central question I think for me in terms of what I do, what I mostly do on a day-to-day basis. I teach in the curriculum but I’m also an academic coach and work with our students that are in the curriculum that either want to take their learning into the next level or maybe are struggling in their courses and aren’t really sure why they are. And so part of what I do with them and this ties into the retrieval practice piece, is really first, sitting down with them and taking a look at different performance metrics, getting them to reflect on how they’re preparing for their assessments or their exams, and really getting into the diagnostic component of to try to figure out, okay, what kind of strategies are you using? You know, are you using those passive learning strategies like rereading your notes or recopying your notes or relistening to lectures over and over again? You know, we talk a little bit about, are you cramming your studying in three or four days before an exam? Or are you blocking your study? Meaning that you pick one or two days out of the week to study just one subject for many, many hours. And that can be problematic.
Ryan: As opposed to distributing, the distributive learning.
Jim: Yeah, yeah, distributed practice with the spacing effect which is another evidence-based learning strategy, right?
Jim: Which I’m a big fan of. But, you know, so, once I get through that process, it’s really about, I think the barrier for me as a coach is really getting them to understand that what they’re doing is not working and doing more of it is not the solution, right?
Ryan: But what is really required? And Jim, isn’t that, I think that’s a great one right there.
Ryan: It’s, you’re trying to get the kid that doesn’t know how to swim to let go of the side of the pool. Because, in their mind, they’re like, I made it this far doing this thing, rereading, recopying, rewatching. You’re like, I made it this far, I’m getting my doctorate, I’m in the doctoral program. Like, I’ve made it this far doing this thing and you’re telling me that not to do that anymore.
Jim: Yeah, it sounds crazy, it’s crazy– And when you think about it, I can certainly understand where they’re coming from, right. I mean, it’s a huge leap of faith, you know. Who are you, you’re sitting there with a student, they’re looking at you like, who are you? You know and you’re asking me to totally do a retool on how I prepare for these exams. And especially if they’re not doing well, they’re less likely to take that risk and make the jump. And I think that that’s what you and I are talking about. One thing I’ve learned over the year, a year and a half I’ve been coaching, doing academic coaching is that, it’s behavior change, you know. Understanding retrieval practice, like you talked about, it’s not rocket science, right. It’s easy to understand that the more times you bring information to the front of your brain, you try to remember it, the more deeply encoded that information becomes in your long-term memory. And we even know from a neurophysiologic standpoint that there are neuroadaptive changes that occur in the brain at the level of the neuron when you are encoding information. So we know an awful lot about this, not only from a theoretical or a practical standpoint, but also from a biological standpoint. So, I guess, you know, I really liked what you said, was about how you get your students to try it, right. Get them to try it so that they can see that it’s effective. And I do a very similar thing in my coaching sessions with students. And we’ll say, all right, here’s a couple of different ways we can go about this, let’s practice it and try it and see if you can see its effectiveness. And usually, it’s very, very quick that they pick up on it.
Ryan: The key I think, for us to keep in mind and of course anybody listening, don’t pick like an entire lecture to try it on.
Ryan: Pick like a half a column of something, pick a slide.
Jim: A slide, that’s exactly what I do, I start with a slide.
Ryan: Start small and iterate three times.
Ryan: Iterate three times, pick two slides and then leave them between each other, I don’t care. But, do it once, recall, get your butt kicked, self-check. Oh, okay, blah, blah, blah. You’re not gonna feel it, you’re not, you’re just gonna be like, oh wow, I just got punched in the face, that’s fine. You’re not gonna be like, aha, I just learned this. That’s not it. Do it again, self-check, do it again. Space out a little bit, put some stuff in between. That to me is the key for that proof of concept. So I think like what we’re talking about is the difficulty of getting them to actually try to be able to see it. So I’d say, two to three iterations with self-checking spacing and in between, small run, not big run. If it works with a small run, it’ll work in the big run as it unfolds. And then the other one, like I said, I think you, that I’m sort of underscoring, that you brought up, is we’re trying to get them to abandon their old way. Which did, I mean, in the back back of their mind, they’re like, yeah, that worked, yeah. That got me, I mean, even if they’re unhappy, they’re still like that got me this far. And then I think the most logical conclusion to draw and you said this, was, well, if I did that say, for, you know, three times per topic, in my old life, pre-med school, then I’m just gonna do it six times now. It’s just more in my PharmD or my med school program. Therefore, I’ll just do more of the same. That’s a very logical conclusion to draw, it’s just, it’s raw. And I don’t know, if somebody is doing it, these ways that we’re sort of disparaging and that works for you, more power to you. I don’t wanna talk about all students being the same.
Jim: Right, because they aren’t. And you’re absolutely right. There are students that are, you know, you could throw them, you know, I always laugh cause, you know, I have a teenage daughter just like you do and she’s very, very bright. And I always say, you know you can throw her out in the middle of the desert and a paper bag with a bunch of books and she would learn just as well as she does. It doesn’t matter the conditions, she’s just the natural learner.
Ryan: Yeah my wife is one of those people, my wife is a clinical pharmacy specialist infectious disease. Like, Cara is like, what are you doing, how are you, like, do they just like read it three times or they read the questions carefully? I’m like, this is not the conversation. You know she’s on the other end of it.
Jim: Yeah, but you know, and just, I don’t mean to interrupt but I think that what you just said there was so so important, is, you know, especially if we have any educators that are listening to the podcast and are thinking about, you know, trying to teach students about retrieval practice. You know, I always tell the faculty that I work with and I also have a number of post-docs and PhDs fellows that I work with to help prepare them for careers in academics and teach them how to teach. And I always tell them, remember you’re an outlier, just like your wife is, just like my daughter is, you’re an outlier. And, you approach students and you’re teaching them this technique or any technique to study and prepare for an assessment. Just remember that what worked for you, is probably not gonna work for that student especially if they’re struggling. So, I think that’s really important.
Ryan: Yeah and I do think that’s super important. And I think I would have been one of those students had I been–
Jim: Yeah me, oh yeah, I was one of those students.
Announcer: Thanks for tuning in to the STATMed podcast and be sure to look for part two of this conversation. If you liked this show, we hope you’ll subscribe. You can find more test-taking and studying strategies, specifically developer med students and physicians, over at our blog on statmedlearning.com. Thanks for listening.