Practical Ways to Include Retrieval Practice In Your Studying
When it comes to study strategies, many of the students we work with tend to double down on the techniques that helped them get this far. But, sometimes, those methods fall a little short in the onslaught of information you’re expected to know in medical school. If your go-to study strategy proves inefficient, retrieval practice is the med school study habit you need to try.
This is the second of a two-part miniseries discussing retrieval practice. Listen to part one here.
The Top Med School Study Habit You Need to Try: Retrieval Practice
In this podcast episode, host Ryan Orwig is back with Dr. Jim Culhane to dig into ways students can incorporate retrieval practice as their go-to med school study habit. They share three different retrieval practice strategies students can use to maximize their studying.
“There are a couple of components to retrieval practice that’s important if you’re going to do it correctly, You have to do the self-check. If you’re doing a self-test, it’s important to do a self-check for depth, breadth, and accuracy.”
– Dr. Jim Culhane
We teach this skill in greater depth in our STATMed Study Skills Class. You can also explore our blog and YouTube channel for more strategies and insights.
Subscribe to the STATMed Podcast!
Transcript
Announcer: Welcome to “The STATMed Podcast,” where we teach you how to study in med school and how to pass board-style exams. Your hosts are Ryan Orwig and David LaSalle, learning specialists who have decades of experience working with med students and physicians. This episode is part two of a two-part conversation with Ryan and Jim Culhane, from Notre Dame of Maryland University School of Pharmacy. They talk about one of the most important study skills for medical students and professionals, retrieval practice. If you haven’t checked out part one, we’d encourage you to listen to that first.
Jim Culhane: You know, failing is actually a really good thing. It’s the way that we learn. We don’t learn necessarily very well through our successes, but we learn so much through our failures, and the key thing is to fail in a controlled, low-stakes environment, right?
Announcer: Here’s Ryan.
Ryan Orwig: I’m saying it’s important that we understand that there are these inherent challenges to the art of teaching these highly important skills to students in this arena. Now, let me just, before we pivot, I think there’s a big difference between what you do, which is coaching students while they’re in the middle of the semester.
Jim: Right.
Ryan: This is what I do. I don’t work with students when they’re in the semester. We work in the spaces in between. We work when they’re on break in between semesters, in the summer, in the winter, on break. If they’re having to repeat, that’s when they come to us and lets us sort of pull the, so our, I see people like you, you’ve got the canoe going down the river and they’ve got leaks in it and you’re trying to help them plug those leaks while we’re still in stream. Whereas we’re like, “Hey, the leaks are so bad, let’s pull it out of the river, let’s fix this whole thing up and put it back in.” So it’s a very different, but it’s different but the same, there’s different avenues and different contexts in which we, we, we sort of intervene and I think both are really important and really, really viable.
Jim: You know, and I think that, you know Ryan I think that, you know, you work with a lot of our, our students every year with your, and in terms of RP 3616 course, you come in and teach your methodology. and I think, you know, students that are, are really struggling and they really need to retool everything that they’re doing, especially if they’re just, they’re not, they’re really not gaining traction with anything that they’re doing. I think that it’s really, really important to be able to pull them out of the water like you said, you’ve got to be able to, you know, pause and learn these new skills and practice them, and then, you know, and then get back in the water and start paddling downstream again.
Ryan: Yeah, I mean, I think both, like I said, like both of these sort of modalities of intervening for students they’re both extremely valuable. They can, they can work together. They can work in isolation. I just think that I try to be a voice to, number one, let people know if you’re in one of these programs? If you’re in med school, vet school, PharmD and you are struggling, it’s not, it makes sense a lot of times, it makes sense. It’s such a, an astronomical jump from whatever undergrad or master’s or biomed program you were in to going to these programs. And so many people don’t have the methods in place, and there are, there are a lot of things you can do. There are a lot of different strategies that can be combined, mixed and matched, interchange with each other to really elevate that overall performance. So when we, when we do talk about teaching retrieval practice, I thought now maybe you and I would just, I don’t know, share three different retrieval practice strategies. And look, if you’re listening into this and this is what we say might be more than enough to really ignite in you the ability to use these more. You know, you might need to do a deep dive into the website and look at some of the stuff that we have on there. Work of course always, you can always get a hold of me and talk to me about these things. But anyway, so I don’t know. I guess one of the first, I’ll go first, one of the simplest things that we do is we say we call it a basic self test. We use the word self test interchangeably believably with retrieval practice.
Jim: I use the term active recall too. That’s another, that’s another term that you hear. So these are all, I think three interchangeable terms.
Ryan: Is active recall the term that’s most commonly in the literature or is it retrievable at this?
Jim: No, you know, I’ve seen, I’ve seen, I’ve seen all three to tell you the truth. So, you know, I think it’s just a, that’s a good point to make sure that everybody understands that we’re talking about the same thing whether we’re talking about retrieval practice, self testing, or active, active recall.
Ryan: Yeah, and I, I sorta got away from using the word self-testing because I think people have such a negative association for test. Like I’ll have students say, like, “I don’t want to attend I don’t wanna do a self test ’cause I might not do well.”
Jim: Well, and that’s, you know, and you said before when we talked about barriers, I mean that is a huge barrier, the, the psychology of all of this and that fear of failing, and that’s something that I have to really have conversations with my students about and saying, “Look you know, failing is actually a really good thing. It’s the way that we learn.” We don’t learn necessarily very well through our successes but we learn so much through our failures, and the, the key thing is to fail in a controlled low stakes environment, right?
Ryan: Brilliant.
Jim: That’s the key. And so when you, and you do your failing there so that when you get to the exam, when you get to the assessment, or whatever it is that you’re doing, that’s high stakes, you’ve already, you’ve already failed. You’ve got, you’ve moved past that.
Ryan: Well, the attempts at failure are how we learn. Imagine like it a kid, imagine a kid eyeballing a bicycle. “Like I would love to ride that bicycle.” And they just like, sit there, and like, you’re like, “What are you doing kid?” They’re like, “I’m thinking about bike.” It’s like, get on the bike, get on the bike, try, fail, try, fail, try, fail. All of a sudden you’re riding a bike. Little kids are learning engines though, little kids are not as afraid of failure. When I think when you get these like really successful students who then get into these arenas and they’re, and they’re used to never messing up. And, and especially once they start to get rocked a little bit with some, some struggling in school, in med school and PharmD school, then they’re, they’re even less likely to be okay with quote unquote failing, right? Attempted recall failure.
Jim: Yeah, and you know, I love your example just kind of cued me in about the little kids and learning things because I think here’s an important point about failure. Failing without expert feedback is a, is not a good thing, right? Because you can end up just failing over and over and over again and not being able to fix the little things that are going on. A great example, you know, my teenage son he’s, he loves to skateboard and I’ve, you know, I’ve never even been on a skateboard in my life. So I know nothing about skateboarding other than I’d probably break my neck if I tried it, but he’s gotten into it. And he’s, you know, he’s trying to learn these tricks like calls an ollie and the kick-flip and, and all these things. And he’s, you know, he practices all the time and, you know, over and over again, and sometimes he gets frustrated. And so what I do is, you know, I’ll videotape him or not videotape, but I’ll, I’ll, I’ll, I’ll video him using my iPhone, put it on slow motion so that we can watch him and then we’ll go to YouTube and watch a pro do it and compare. And so that kind of feedback so that the next time he practices it, you know, he’s able to do, he’s able to do better, instead of repeating the same mistake over and over again.
Ryan: So how does that translate do you think to a student trying to deploy some retrieval practice and studying?
Jim: Right, so, I mean, I think, well, you mentioned it. I mean, I think that there’s a couple of components to retrieval practice that, you know, is really important if you’re gonna do it correctly. You know, it’s gotta be, you know, you’ve got to do that self check, right? So, you know, as you’re, as you’re doing the, you know, the self test or some of these techniques we’re gonna be talking about, it’s really important to do an immediate self-check of your answer for what I tell my students, depth, breadth, and accuracy, right?
Ryan: Beautiful, exactly. And that’s, that’s what I was, I was lobbing that up there for you.
Jim: Yeah, exactly. That’s, that’s, that’s really important, right?
Ryan: You don’t, you don’t, right, the expert is the source material you’re checking against. You don’t have to have some PhD explaining it to you. You just need to check the source, that…
Ryan: Unless you don’t understand it, that’s the other piece too. And I think that’s one other thing too that I think is really important for your listeners to understand, and I wanna reinforce because when I work with faculty, and I do faculty development in this area, sometimes you’ll get pushback about this because it sounds very similar to memorization, which in higher ed…
Jim: Rote memorization, bad word in higher ed and I, and I get it, but the example that I give them all the time is that okay, you know, to me rote memorization is being able to remember facts without con, contextual understanding. So for example, you could memorize six different Russian vocabulary words and not have a clue what they mean, and remember them and repeat them back later on, to me that’s rote memorization. You know retrieval practice is really bringing those words to the forefront of your mind, maybe using them in a sentence, understanding their meaning, how are they connected to one another. So that, when you do that in your retrieval practices it’s not just about rote memorization but it’s really about contextual learning, right? And, and really trying to think more deeply about what you’re remembering because the more connections that you make, right, between the information you’re trying to recall and prior knowledge that you have in your long-term memory, the stronger that information will be encoded into your long-term memory, and the longer you’re gonna be able to hold on to it and use it more effectively.
Ryan: Oh yeah, absolutely. I understand the fear of, of rote memory and I don’t, yeah, I don’t, I don’t think that that’s good.
Jim: Right, I know how you feel about that but I think some people, when they hear retrieval practice they’re like, “Oh, this is just a buzz, this is a buzz word for rote memorization.”
Ryan: Oh yeah, yeah. You and I are both making, you and I are both making like, stink faces, like, “No, that’s not what this is.
Jim: No, no, no, that’s not right.
Ryan: That’s a good point as well. So anyway, so I use the basic self test. Basic self test is like real simple, you pick something and you attempt to recall it, you know, and I like having like numeric prompts, like, “Oh there’s eight things I need to recall from this or there’s three sub categories with four things each under it.” Like some, some of those kinds of things and we talk about, maybe you can, as you’re reading, in lecture or reading, you can delineate those things, there’s missing categorical layers we call them, and then you can quiz yourself off that stuff. As you’re reading, instead of underlining and highlighting, you’re like, “Oh, okay, this little paragraph is really talking about like risk factors and there’s eight of them.” And it’s talking about, instead of like highlighting all eight, note that there’s just eight of them, maybe a number and then when I’m coming back to self tests, I’m like “What are those eight risk factors under this, this category? Boom, boom, boom, boom, boom, oh I got four, I forgot four. I got four, I, I picked two that aren’t part of it and I forgot the other two.” All, and then the self-check corrects it. And then you can toggle with that, you can toggle back. Like, you know, you can read for a little while and when you start to get tired, start to zone out, instead of like, let me read something else, say, “Let me just go back to the previous 30 slides I just did or 20 slides or 10 slides and let me find some stuff to self, some stuff to self-check, do basic self tests on and do that for maybe five, 10 minutes, and I just pick up and resume my reading.” That’s a small, simple way to use retrieval practice while in the middle of studying to change gears, to keep you in control of the learning, keep you on relatively on task but to make it more dynamic, more like cross training with, with the study. So that’s, that’s, you know, obviously a very simplified version of the basic self test. What is one that you might use?
Jim: Yeah, so, I mean, that’s really, that’s really the question that I asked myself when I, when I started, you know, when I first met you and I began to understand a lot more about what you do and the methodology that you teach is like, as a, as a teacher, I’m sitting here thinking to myself, “Okay this theoretically, this makes a ton of sense.” And I’m looking at the literature, I’m like, “Wow, there’s, there’s a ton of great empirical scientific evidence out there to support that this really works well.” And so the next question as an educator is how do I teach my students right, how to do this? And in a way that’s easy and accessible and maybe even familiar to them. So, you know, as I started to think about that a little bit it came to me that in some cases we were actually doing retrieval practice in our classrooms through active learning techniques, or pedagogy that we were using in our classrooms. Things like the minute paper is a technique that I, I talked to my students about. And we, we use that in a lot of our classes, mid lecture we might stop after we, we we’ve talked for a few minutes about a complex topic. We’ll tell our students to pull out a piece of paper or, you know, bring up a word blank worksheet. And for one minute, type down everything that you can remember about what we just talked about and write a paragraph about it. So it’s not just about remembering facts and information but it’s contextualizing it. It’s, it’s describing it in your own words. And this is really related to another evidence-based learning strategy that’s called elaboration and, you know, it’s just explaining. You know, you can do verbal elaboration you can do written elaboration. And so my students, because they were familiar with that technique that we use in the classroom, like, “Oh yeah, you know, Dr. So-and-so does this in their class.” I’m like, “Absolutely, guess what? You can do it while you’re studying too.”
Ryan: I love it as an instructional tool. Why is this not just a part of our culture, right? You know at some point the professor is gonna be like, “All right, hold on a second, write this out and then self check it.” I mean, that should be a part like three times a, a lecture, like, you know, it’s gonna happen. I feel like it would keep you more on your toes and it would demand that sort of synthesis recall, attempt, failure, you know, elaboration, that self-check and corrective and then you move on. I like the idea of like tying back into material through retrieval practice in various capacities as you’re moving forward, it just ties it all in better. But as you say, and look, we’re not, I, every day I talk to students and every day students are upset about their professors or their schools or their, the med school, they hate their curriculum, or this professor’s terrible, I’m like, “It doesn’t matter.” Like, okay, air it out, I’ll listen to it, I hear you and I feel for you, but it’s not changing. It’s not changing, so air out your grievances and then say “What can I do as the learner in the cockpit driving this thing, do to make it better for me.”
Jim: Absolutely.
Ryan: And, like drop out, say like during, especially if it’s like, on-demand lecture, you’re watching right now. Boom, pause, what do I remember? Boom, boom, boom, press start, carrying on, studying on your own. I’m in the middle of reading, let me just stop, jam out that white, the one minute paper. Boom, boom, boom, what do I remember? But you self check it and then you move on. It’s great and that ties in, so number three, our third one, my second one here, would be what we call the time self lecture. It’s very similar to the basic self test except we add a timer. I like three minutes, and people are like, I’ve seen people try to do stuff that’s like, it’s like watching a kid try to put a whole hamburger in their mouth in one time, like, why couldn’t I, why couldn’t I get through it in three minutes? ‘Cause you chose 50 slides. I mean, you’ve got to learn, like what’s like a, not, like Goldilocks, not too little, not too much just right. Let me pick like a sizeable amount that’s gonna make me really push to try to get through it in three minutes. So you got the ticking clock and you, and I want them to talk out loud and write it and it can be very abbreviated in some sort of fashion that like, I think of like on a whiteboard like you’re, like you’re organizing it on a blackboard or something. But like numbers, abbreviations. So it’s making you go through all these hoops. So the act of talking it out, like if you, if you can’t say it, you don’t know it.
Jim: Right.
Ryan: You’re gonna be evaluated eventually talking out loud when you get to like rotations and stuff like that.
Jim: Absolutely, exactly, right.
Ryan: So how often are you practicing that, you know, you ask people like, “How often do you practice that?” Like “Never”, like I’m not trying to make you feel ashamed, I’m just pointing that, pointing it out. Talking out loud and then that’s another sensory modality, it’s like, you know, coming out through the vocal cords, out to the ears, all good. But if you can’t say it, you don’t know it, having to write stuff down as another complication makes it easier to self-check, because if you didn’t put something there, then you’re not allowed to count it. All while operating under that timer ticking down because the joke I make, which I enjoy, you know, it’s not even a joke, I’m just like, “’cause, ’cause you’re never evaluated under time circumstances, are you?” Because you know, people are like, “Well, you know, I don’t have enough practice questions.” Like I don’t care about practice questions, this stuff is what really matters. If you are doing this kind of stuff, thinking under time pressure, it, it it’s, it’s, it’s the whole ball game. So I think the goal should be whatever your, the sequence is of your study, it should all be to get to retrieval practice. Ideally, some kind of like, like you said, like the one minute paper, the time self lecture, so that you’re actually burning and pushing to get through retrieval mechanism, retrieval circuits under some kind of time pressure. So that’s, that’s another one on my end. What’s another one that you might be able to throw on?
Jim: Yeah and that’s, you know what you mentioned there too is, is again another one of those evidence-based learning strategies is, it’s called dual coding, and exactly what the three-minute lecture utilizes and it’s really, really effective.
Ryan: Can you define dual coding?
Jim: Yeah, dual coding, it’s a, you, you did, you did a great job I think explaining the concept in, in your example there. It’s an evidence-based learning strategy that takes advantage of multiple modes of, of inputs, sensory input, right? Dual, dual, meaning two coding, so you’re, as you pointed out, you’ve got the audio from your, you know, talking out the concept out loud and then you’ve got the mechanics of writing or drawing. So the kinesthetic input. So together we think that that actually enhances encoding of information more than either one of those by themselves.
Ryan: So again, like why not bring that in? I mean, you know, our students, we have their, like they’re studying all day, they’re studying all the time or you’re not studying all the time and you wanna get the most bang for your buck when you are. I mean why not do things that, that emphasize best encoding practice? Encode, put it in, retrieve, access, pull it out. That’s, that’s the whole, the whole shebang really. So that’s what I think everybody wants to be, well, I mean, I think on the professor side is what we want our students to do, on the student side this is where you’re spending, your time is your currency. That’s like the, the, the scarce resource. So you wanna be able to get the bang for your buck while you’re there. Okay, what’s another one?
Jim: So another one that’s really, really interesting that has evolved for me is the empty outline technique. Again, this is another one of those classroom activities that professors can use with their students to help them to practice retrieval in the classroom. So basically all the empty outline is, is if your slides or your, or your handouts are in an outline format, you go in and you empty them out. You empty out the details and leave blank spaces and then the, the student’s job is to try to remember what goes into those blank spaces. And so, and, and I, this, this particular technique early on was really appealing to me because it, it forced the student to actually use the learning materials that I was giving them in class, or that their instructor was giving them in class. So, in other words, the PowerPoint presentations or the handouts, they weren’t recreating something like recopying their notes or creating these elaborate tables or whatever that takes a lot of time to do.
Ryan: It’s a lot of clerical time recopying, that feels good but you’re not really, it’s not high yield learning.
Jim: No, no, you don’t get, you don’t get very much return on your time investment with stuff like that. So, you know, my, my, my whole idea was how can we efficient, more efficiently do this by using materials that students have? And so I started teaching that technique to a number of the students that I, I was coaching at the time and as, as they were practicing this technique they were coming back and saying, “Dr. Culhane, you know, this is working, but it takes me so long to empty out these outlines or PowerPoint presentations, to go and delete everything and leave…
Ryan: That’s the clerical demand, that becomes the collective office work, clerical demand that we don’t, that we’re trying to avoid, right?
Jim: Right, and so what, but here’s the, here’s the really neat part about the story. And I know you and I have had this experience also is that, one of the students I was coaching and working with, he came back to me one day. He said, “You know what, I’ve got this idea.” He said, “Instead of the deleting everything,” he said, “What, I’ve, what I, what I did is, I just went into PowerPoint and drew boxes, colored boxes, over the text to,” you know, sort of like using a piece of paper to cover up a printed out note, right? Note page and so, and he said it was a lot faster in that way. And I thought that was a real kind of brilliant, you know, innovation with that. And, you know, as I started to think about ways we could improve on that, I added a lot of different elements to it and it evolved into a retrieval practice technique that you and I have talked about before is what I call the boxing and unboxing technique, which we might get into in another, maybe another podcast or discussion.
Ryan: Yeah, I think we’re gonna build something out on this. This idea of using boxes in PowerPoints is, is absolutely fascinating. It’s a great tool that, you know, you’ve sort of explained it to me on the fly and we looked at it a little bit. I was like, “Oh, this is something.” So, and you call it box, what are you calling it?
Jim: I call it, I call it boxing and unboxing. So basically…
Ryan: Boxing and unboxing?
Jim: Yeah, boxing and, you’re boxing the lecture, you’re boxing the lecture off essentially, you’re going through each slide, you’re, you’re drawing squares or rectangles on PowerPoint over key pieces of information that you need to retrieve and then inside the boxes, what you’re doing is you’re creating what I refer to as retrieval cues, okay? These are prompts or questions that will help you to retrieve or remember that information. So, you know, maybe you’re covering up the definition of a term and in the box you write, “Define this term.” Or you might even write, “How does this term relate to another concept, or why is this important?”
Ryan: It’s great. And some of that ties into what we call the missing categorical layer.
Jim: Yeah.
Ryan: I guess I feel like what happens with these experts, like the, the, the, the categories and sub categories get obscured. So what we want, like, okay, this is definitional. This is what are the ’causes, let’s say, we’re looking at sister cirrhosis or whatever. Like, what are the definition, the cause, transmission, results, epidemiological information, diagnostic stuff, you know, all that, like, because if you, I think we should teach students to read as they’re reading, ’cause you’ve got to read it once and learn how to seek and break and pull out those missing categorical layers and identify them. And we want to put those either on the side, in the margins, or you put it on top of the bar.
Jim: Or you’re, you’re, you’re making a judgment call about what to box out, how to box it out, what type of question or retrieval cue to put in the box. It’s, you know, and it’s really interesting you mentioned that because I think that the processes are very, very closely related. I didn’t think about that in that way, but, you know, I think that that’s really important. You’re unconsciously, like you, you’ve always said kind of building that blueprint or superstructure, right?
Ryan: Yeah, we call it a framework.
Jim: Framework, right. That’s what you guys call it, right.
Ryan: Yeah, my theory of like learning is about, like, I think a lot of our med students in the PharmDs and vets we, we work with, their brains don’t build those frameworks automatically.
Jim: No.
Ryan: Whereas most, most people in the field do, but I think a lot don’t. If you don’t then you need to do it overtly, right?
Jim: Yeah, an especially if your professor’s a mess too, and their lecture notes are a mess and totally disorganized and there’s no organization or structure to it, that only adds a layer, right?
Ryan: Yes.
Jim: Of difficulty.
Ryan: It’s a layer of like obfuscation and you can’t see it and you can’t find it, and then it’s just a mess. So this idea of seek, seeking and finding also makes the reading act much more engaged.
Jim: Yes.
Ryan: Because trying to read and memorize isn’t, it’s not really how it works. So if you’re like seeking and finding an extracting, it, it’s, it turns into a scavenger extension.
Jim: It’s analysis what you’re doing.
Ryan: You’re unlocking the critical, analytical side of your brain, and then engaging it by default, which is really powerful. Now again seek, go ahead.
Jim: Oh no, so I, I just realized I wanted to finish up the, the unboxing side of, of things, right? So even after you’ve gone through your PowerPoint presentation, your Word handout and you’ve covered things up in squares, and you’ve typed in your retrieval cues, you know, the next thing to do is to unbox the lecture, unpackage it. So that means going slide by slide and using those retrieval cues to try to remember what’s under the box or describe.
Ryan: That’s recall.
Jim: Right. It’s recall, right? And one of the neat things about it that I was so excited to kind of hit me is that the color coding system that we came up with for the boxes, right? Which allows you to track your learning and you mentioned this at the beginning of our discussion about that, that, that data strip that should come out of your brain telling you, “Okay you’ve encoded this and how do we do that?” Well, when you unbox the lecture in this technique if you successfully retrieve all the information underneath the box, I instruct my students to turn that box to the color green, indicating you’re good to go. If you have partial recall, you change it to yellow and if you can’t recall anything at all, you change it to red. And then you’ve got, obviously you’ve got your fourth color which is your default that you use when you first covered the box up that just indicates that you have not attempted retrieval on that material. And so what students can do now is they can track their learning on a day-to-day basis.
Ryan: Inherently without a huge time cost. Let me, okay, so we will, you and I’ll work together on this and we will, this’ll be built out and you’ll be able to find it on the, on the, on the, on the website and then we’ll probably do a deeper dive talk into it. ‘Cause I think it’s absolutely fascinating. So that’s, that’s boxing and unboxing is that what you’re saying? That’s what you’re calling.
Jim: Yep.
Ryan: Okay, beautiful. All right, so now, like I said, moving forward, last one for me I would say we use voice flash cues. So that’s using your phone, especially if you’re a board, a board prep level and they’re like, you’re reading and you’re seeing all this high yield information being laid out, like in the explanations. You’re like, “What do I do with all these fragments of information?” The idea is like you, and you can do this in any regard. You can do it while you’re studying in lecture, like after studying information after lecture however you like it. And you basically are turning factoids into question answer files, and you’re just reading them into your phone. And then you can listen to them on the go when you have a commute like, like, like physical labor, light workouts, go for a walk and you can be quizzing yourself, question, answer, question. So you listen to the question, pause it, attempt to recall the answer, then hear the answer. Boom, boom, boom, boom, boom, boom, boom, in a circuit. I don’t, I think of that as something as a boost, as a, as a bonus, not as a primary learning strategy. But it can be a really powerful resource for people especially if you’re like really in a crunch studying and you feel like I don’t have time to do anything other than study. Well, now you can make something on the fly and then go do something a little healthy, even if just to get out, walk around, and get on a treadmill, whatever might float your boat. Or if you’ve got you, you have a long commute and you wanna utilize that, like don’t just listen to a lecture, that’s very low yield.
Jim: Right, and students do that all the time. And you know, it’s, it’s, you know, and I’ve talked to you a little bit about my before, about my, my philosophy or idea about the process of learning, right? And how students move through different stages of learning to achieve content mastery. And, you know, I think one of the things that I find with my students that are maybe struggling a little bit, they spend way too much time in that understanding phase of learning where they’re, they, you know, “I don’t understand something, so I can’t start retrieval practice yet.” They go hand in hand, you know, you come out of class and you’ve got a certain level of understanding hopefully if you were conscious and awake and paying attention, but, as you start to get into retrieval practice, and if you use some of these elaborative strategies that we talked about, your understanding will grow as you’re encoding that information into long-term memory, right?
Ryan: Yes, absolutely and I think that’s really important is this idea of not waiting for dead mastery to move on. You’ve gotta be okay understand, like, it’s almost like your learning has to be like a crude hypothesis and the only way to really test it is by trying it, you know, building it, and that should be through attempting recall. Yeah, it’s like, I don’t know, I feel like it’s like somebody who goes to the weight room for three years straight and they never increase the weights.
Jim: Yeah.
Ryan: Like we had a buddy of mine growing up. We, we were out in our senior year and he’d been going to the gym, he’d been working out his entire high school career. And he was, so he talked about like increasing their bench press or something. And he sees like, “Wait, what?” And they’re like, “Yeah, like, you know, you increase it as you work out throughout the.” And he’s like, “Oh”, was real quiet, like “What, why, why do you say that?” He’s like, “He was like, I never increased the weight.” And we were like, “Over how long?” He was like “All of it.” So he was still benching and curling and squatting the same amount for three, three years. This guy by the way, this guy by the way, shows up at our, like there’s like, didn’t show up at our 10-year high school reunion, we didn’t know where he went, he was like working at a bar. Showed up at our 20-year high school reunion, and now look, at your 20-year high school reunion, there’s gonna be some people that show and they’re doctors, and it’s like obvious.
Jim: Yeah.
Ryan: And there’s gonna be one person shows up and they’re a doctor, and it’s equally hilarious and terrifying. That was him, that was him.
Jim: Oh really.
Ryan: Yeah.
Jim: It was bad, it was awesome, it was awesome. Super smart, we didn’t know.
Ryan: Oh wow.
Jim: I mean, great, it was awesome, you know, but he was, he, it’s his idea of, I just gotta keep doing the same thing without, you know, adding the resistance, adding the struggle, adding the failure, right? That’s where we grow, that’s where it grows. And so this idea of like “I’ve gotta keep it steady the whole way.” No, no, no, no. Overburden, fail, mess up, readjust and, and, and, and, and obviously it, it, it, it it’ll get better as you go, even though it won’t be pretty, learning should not be super pretty. It should be a little ugly, it should be a little messy.
Ryan: And, and that’s, I mean, that’s a really great point that you bring up about retrieval practice is that, you know, one of the conditions for successful retrieval practice that we know is introducing what Robert Bjork and his wife Elizabeth called desirable difficulties. Right?
Jim: Yeah. Right, and so this is important. So if retrieval practice and I love your weight room analogy, right. If retrieval practice is easy, right? If it’s, and you can, and you can do it without, without much effort, you’re not getting much benefit to it. It’s very passive.
Ryan: That was like the reason I brought in that awkward analogy about my buddy, like, because it, like working out had to be pretty easy for him for like at least two and 1/2 years. And he never like caught on.
Jim: And so he didn’t gain, he didn’t gain a lot from that. Right, exactly.
Ryan: Absolutely, yes. It’s like, man you’re spending an awful lot of time in the gym. You, you know, where’s the, where’s the yield. But it’s, it’s, right, it’s, it’s about understanding this desirable difficulty and that’s, that is, that is a great word to really add.
Jim: I love that, yeah, in fact, if you’re, you know, if you’ve got any, any sophisticated readers out there, people that like to dive into the literature, anything that, that Bob or Elizabeth Bjork has written is, is definitely worth looking into. They’re, they’re really at the top of the field with regards to things like retrieval practice, metacognition and, and learning, so.
Ryan: So, and then just talking about the, like those voice flash cues, I mean, I know it sounds a lot like Anki and Anki’s great ’cause Anki is like an automated thing. There’s already files out there, it’s detail-oriented, some of it might even go straight to audio. If that works for you, great. Let me just say this quickly as a caveat and I’ll, and I’ll elaborate on this elsewhere, Anki does not always work for everyone. I think Anki is really good for bottom-up learners, people who can learn from the details and then assimilate them into a, a framework subconsciously, but I meet a lot of people in there like, it’s like the emperor has no clothes on type of thing. “Like why is everybody swearing by this and it doesn’t work for me?” and usually those are top-down learners and they need an explicit framework. They need to be able to seek and find and build that framework themselves, and then the details will go in more fluidly and quickly, and it’s better encoding and retrieval. But I just wanna make sure that we understand that when I talk about something like, words flash cues like, this is very similar to Anki. both of those are detail oriented, and if you are someone in that detail first learning doesn’t work, it’s okay, it makes sense, but that’s, it’s often because of that, it’s like that, that, that, that framework building mechanism is just not as innate in that learner.
Jim: Yeah, absolutely, and, and, and I think another important piece that you bring up as well, too, you know, we have retrieval practice tools like Flashcards, Anki, Quizlet, those things are all available. There’s lots of apps out there now that help you with retrieval practice. And I think that one of the things that again, if you go to the scientific literature about retrieval practice and Robert Bjork talks about this as well too, is that you don’t, especially if there’s a long period of time in between assessments. If you’re a student, right, and you’ve got exams every month, every four or five, six weeks, okay?
Ryan: Yeah.
Jim: Maybe you have one at the midterm and one at the end of the semester, using the same retrieval practice strategy to study for that course over and over again, can be very, very ineffective because you develop something that the cognitive psychologists call automaticity. And so, you know, the, the retrieval becomes essentially automatic versus effort…
Ryan: And you’re losing The desirable difficulty, right?
Jim: Exactly, exactly. And so what, the way to beat that, and, you know, you have this built your system, and that’s why I, I really like what you, you and Dave teach is the, the different methods for retrieval practice. So you can mix it up, if you vary it. Now for my students that I work with, they’ve got exams every about every two weeks, major exams. I don’t necessarily think at least in my experience that for many of them that’s long enough for that automaticity to develop, because most of them aren’t hitting their material every single day, spacing it out on a daily basis. And there’s still probably some cramming and blocking going in there. But, you know, for any longer periods of that, I might suggest using multiple retrieval practice strategies.
Ryan: Yeah, that’s, that’s and that’s why you’re on here, that’s why you’re on here Jim for bringing those…
Jim: And like I said, I’m not a, you know, right, I’m absolutely no expert on this at all. This is not my area of expertise, I have no formal training and education other than, you know, what, I’ve, what I’ve learned over the years in reading the literature.
Ryan: And you’ve brought your PhD brain into this, and the way that you read and consume, you know, the literature is, is truly valuable. And yeah being an aggregator and a share of that information is really valid.
Jim: Well, absolutely. I just, all I’m trying to do is in the best way, and the most accurate way that I can is to.
Ryan: Buy you got your own little lab now, because you’ve got your students coming in to see you, and it’s like I told you, when you, when you took on this position, I was like, “You’re gonna love it.” Because you’re getting live real world experience and live real world feedback, working with those students, and that just levels you up, right?
Jim: Oh, absolutely. And that’s, and I think that’s really the difference too. You know, you and I have talked in the past too. I mean, there’s a lot of, there’s a lot of really good information on the, on YouTube and on the web about retrieval practice and then there’s some really poor information out there about it. And I think that, you know, it’s really, really important if you’re a student looking for this kind of, of, of information is to really evaluate your source material that you’re getting. And, you know, if you, and some students, you know, all they need to do is listen to a 15 or 20 minute YouTube video and they might be able to pick it up, right? It’s like the gifted athlete, you can teach them, you know, right, how to shoot a goal from midfield, right? Without, you know, 15 minutes and they’ve got it. But for most students, especially if you’re struggling, it’s really, I think important to engage with an expert, someone who can help guide you through the process, provide you with feedback and teach you different methodologies and strategies to do that.
Ryan: And unfortunately, I mean, the world we live in there’s, you know, not all experts are the same.
Jim: Right, right.
Ryan: You know, you do, you do have to be discerning and, and just because somebody might have like some credentials, it doesn’t mean like, if they’re telling you something and it’s not clicking, you know, it might not, it might be them and not you, you know?
Jim: It’s those, it’s those thousands of hours, you know, you, that you’ve accumulated working with students that struggle that, you know, you’ve built that mental database of, you know, how to diagnose, how to steer students in the right direction. And, you know, for you again, what I keep, what always amazes me as I think back on our, you know, kind of our journey together is that, you know, you were really promoting this stuff to us years before it was, it was really becoming mainstream. And so, you know, that’s that time, you know, that time and, and, and place, and really working with it is, is hugely beneficial.
Ryan: Yeah, I guess it does put me ahead of the curve, because I’ve been…
Jim: Yes, definitely, absolutely, without, without a doubt I think.
Ryan: Yeah, it’s weird, I don’t think about it. I, you know, it’s just like, it’s like, that’s just the way I see the world, so, but you’re right. Yeah, I mean, I’ve been, I guess I was swimming in that end of the pool by myself for quite some time, and I had so many students coming to me, they buy, we used the word outlier earlier, but I see, I think I’m actually an outlier by the, by Malcolm Gladwell’s definition because I was an early adopter of these skills, I had all kinds of time with my med students and my physicians who were struggling and my PharmD students. And then I, I, I, built, I started building solutions earlier than I think a lot of people did. So I think by…
Jim: And then refining them over and over again over years. I mean, that’s, that’s…
Ryan: That’s constantly and maddeningly, continuing to revise and fix and tear that…
Jim: And rebuild based upon that feedback that you get from your students, that’s really, that’s really, you, you told me that and, and, and, you know, it’s really paid off a lot because a lot of what I learned and a lot of the changes that I’ve made to methodologies that I recommend to students are based upon the feedback that I get from them. Hey, this is working because look, you can, you can come up with the greatest system in the world to teach students how to do retrieval practice, but if they can’t do it, if it’s not practical or they don’t wanna do it or they’re intimidated by it, it’s worthless, it’s worthless.
Ryan: Yeah, no, and, and that’s that, that is the magic piece is like, are you listening to your students? And that’s what I, I have relationships with students from 15 years ago. I still, I mean, they’re, they’re texting me today. You know, I, I got one of my guys I guess he’s a friend of mine now. Like he lives in, in, in Florida and he drives 45 minutes each way to go to his, to work in the emergency department. And he’ll just call me and we’ll talk, we’ll talk about other stuff too, but we also talk about what we’re doing, with STATMed and what he can do if he, if he, if he can contribute or, you know, he’s got a test he wants to talk about, but it’s, it’s fascinating if I don’t, if I don’t have that feedback, how am I gonna know? And that’s kind of how we built all this, was listening and continuing to listen to our students. So, all right, well, I think that’s, that, this has run long enough. I’ve taken up enough of your time.
Jim: No, it’s been great. I, you know, like you said, we, I love talking about this, this stuff. I kind of nerd out on it and that’s why I enjoyed talking to you about it.
Ryan: Yeah, this is not that different from conversations you and I have been having just one-on-one for the last, last 10 years, so.
Jim: Absolutely.
Ryan: Just recorded this time. So we’ll have you back on here for sure and we will dive into your boxing and unboxing and we’ll do something with that too. ‘Cause I think that’s just, that’s, that’s pure gold. So thank you for, you know, sharing, sharing your time and talking with us and we’ll have you back on here soon.
Jim: Thank you Ryan, take care.
Announcer: Thanks for tuning in to the STATMed podcast. If you liked this show, we hope you’ll subscribe. You can find more test-taking and studying strategies specifically developed for med students and physicians over at our blog on statmedlearning.com. Thanks for listening.