Learning in Med School 101 – Video Series Part 4
In this video series, “Learning in Med School 101,” we break down some key concepts about learning in medical school, including skills, strategies, and techniques we share with medical students (and those in related medical fields).
In this video, we help you determine the root of your bad test-taking. Are you missing most of your questions because you just don’t know the answers? Here’s how you can decide if something is a knowledge or test-taking miss.
Knowledge Misses
“I’ve never heard this before!”
“I’ve seen this somewhere before but don’t know what it is.”
“I’ve seen it, and maybe even studied it… but I can’t remember it.”
“I learned it wrong.”
Test-Taking Misses
“Oh… I knew that. I should have gotten it right! “
“I latched onto one clue and ignored the others.”
“I narrow down to two and always pick the wrong one.”
“I twisted the answer to make it fit.”
“I got blinded by my prediction of the answer.”
So, are you a bad test-taker? To find out, you have to be able to see your test-taking misses. You can only fix something if you know how it’s broken. We recommend you start by figuring out what your miss ratio is. Determine what percentage of your missed questions come from a lack of knowledge and which are from bad test-taking habits. For example, if you missed ten questions, read the answers and explanations and discover how many were knowledge misses. If your miss ratio was 3:7 (knowledge: test-taking), that’s pretty clearly bad test-taking. If most of your misses fall into the test-taking category, our Boards Workshop can help!
But, if you’re looking at more of an 8:2 (knowledge: test-taking) ratio, this can show that your knowledge may not be where it needs to be. This could require an upgrade in your knowledge. So either just more studying or going to a review program — or overhauling and upgrading your study skills, which we tackle in our STATMed Study Skills Class.
Subscribe to the STATMed Podcast!
Transcript
When it comes to test-taking at the medical boards level, I’m often asked how we determine knowledge misses from test-taking misses. So, in this episode of Learning in Med School 101, I’m going to illustrate how I define a knowledge miss versus a test-taking miss when prepping for medical board exams.
So, here are four ways we can tell if it’s a knowledge miss: Number one, we might say, “Okay, I’ve never heard of this before.” That’s a pretty obvious knowledge miss. Or we might have, number two here, a closely related response that goes something like this: “I’ve seen this somewhere before but I don’t know what it is.” So essentially, same issue. Now, number three, a different issue is when we say: “Oh, shoot, I’ve seen this and maybe even studied it before, but I just can’t recall it here now.” To be clear, regardless of how much you have studied this or how important you think this is to know, if you cannot recall it, this is a knowledge issue and not a test-taking issue. This gets into study-based issues and requires study-based solutions.
This is a breakdown somewhere along the line with the way you have organized, encoded, stored, and are retrieving the information. So this is a knowledge issue. Then I guess we could also say if you had it in your head wrong, that you learned it wrong, then that’s a knowledge issue as well, obviously. And it’s important to identify the nature of the miss so you can be more tactical in how you fix it. But then on the other side of the equation, we have what we would call test-taking misses. Not everyone taking pinnacle board exams makes a lot of these, but bad test-takers certainly do. Sometimes a miss jumps right out of you as soon as you read the explanation, and you’re like, “Oh, I knew that. I should have gotten that right.” Okay, that’s a legit response, but for me and my students, we want to dig deeper into this type of specific phenomenon and try to understand where and why it happened. Because usually with bad test-taking at the medical boards level, where the clinical vignettes are so dense, second and third-order questions, these types of mistakes are often usually pattern-based mistakes and a bad test-taker is probably doing the same handful of patterns over and over again.
We’ve identified about a dozen of these patterns, give or take, and we wanna help people find them for themselves. So one of those patterns, broadly speaking, would be that you realize you latched onto one single clue while ignoring other clues. This type of move is one way to validate a wrong answer choice in the heat of the moment and actually use your knowledge against yourself. It’s a really damaging maneuver.
A more broad and common statement is when people say, “I always narrow down to two and picked the wrong one.” Now, of course, this can be a big test-taking issue and it’s one of the most popular ones we hear. And like with other bad test-taking phenomena, there are patterns to be found lurking under the surface of this kind of issue. But let’s also keep in mind, not all tie-breaks are inherently test-taking in nature. Look, sometimes it’s a knowledge miss. A knowledge miss would look something like this: You might say, “My knowledge goes three levels deep but I need to know the fourth or fifth level, and is just not there.” This means it’s a knowledge miss, a fair miss, and needs to be fixed by improved studying. Again, getting into the organizing, encoding, storing, and retrieval circuits.
On the other hand, when we think about narrowing down to two, and quote-unquote, “always picking the wrong one”, this means the test-taker is doing something to tilt themselves away from the right answer. They might say, “I realize when I read the answer explanation after missing the question that, ‘hey, I knew this.'” And there are a few patterns that can come into play here. For example, they might say, “Since I didn’t know the right answer with a hundred percent certainty, I picked the other option.” This can be a quite insidious issue for certain types of bad test-takers. And then another type of test-taking miss is what we call twisting. And this is where you take a square peg and make it fit in a round hole, because you know what? If you hit that square peg hard enough, it will indeed fit in the round hole. And this is one of those bad test-taking moves where reading breaks down and you add a little extra, like a, “What if this” or “But just maybe that”, then by adding this one little or this one little clue that you’ve added on, you make a wrong answer appear to be right, which is another way a bad test-taker uses his or her knowledge to hurt themselves. Cartoonishly. It’s like looking at an equation that says 10 plus 10 plus 2 equals what? And you’re like, “But what if that second plus is actually a minus? Aha, it’s 18.” It’s quite hard to get it right if you change an action or an integer, you know? And that’s what twisting is. And certain bad test-takers do this all the time and they might not even know it.
And then one last one I’ll throw in, and there are many more, is what we call being blinded by a prediction. Good reading and thus good medical boards test-taking requires effective predicting and revising of those predictions. No doubt. But bad test-takers often force the prediction early on, locking onto it, then ignoring clues that go against this in an act of hidden cognitive dissonance. This can, again, be one of those silent killers for bad test-takers. Anyway, these all add up to a variety of test-taking misses and you have to then realize, “Oh, shoot, I shouldn’t have missed that,” after you read the answer explanation.
Some of these are obvious, some of them are a little below the surface, and some require some deeper analytical tools to uproot and identify. But to me, at the end of the day, if you could have gotten a question right using the fragmented parts of what you knew in the moment, regardless of the holes or deficits in your knowledge, then it’s a straight up test-taking miss. I mean, on test day, the test itself only cares if you clicked the right answer. It’s not an essay question where you get partial points. And our goal should be to reduce the number of test-taking misses a given person makes. And keep in mind, not all people make these mistakes.
So, are you a bad test-taker? To determine this, first of all, you have to have what I’ll broadly call serviceable knowledge. You have to know enough to be able to see your test-taking misses. You can’t fix the test-taking unless you have, quote-unquote, “good enough knowledge.” So, if you’re at a boards prep level, then this is likely something you can do. And we do this by figuring out what your miss ratio is, what percentage of your missed questions are knowledge versus test-taking.
So let’s say you do a bunch of practice questions, 20, 30, however many, and say you missed 10, ten’s a nice round number, and this is not a concrete 10, like I just did these 10, more of a representative 10s spaced out over multiple sets of 10 missed questions. So let’s imagine your representative 10 missed questions with these 10 game tokens. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. You take the 10 misses, you read the answer explanations, and you determine how many of the 10 are either in the knowledge side and how many go to the test-taking side. Again, think broadly, representatively, sort of fast, don’t overthink it, and tell me your ratio, bam, fast. So, for one person, they might say their miss ratio is 3:7, 3 knowledge to 7 test-taking.
This is obviously extremely bad test-taking and clearly needs to be fixed. This is like a straight shot to our boards workshop. Go fix this strategically. Address all the nuts and bolts, focusing on the training side and the behavior modification side, all that stuff. Someone else might be more on the extreme other side of the spectrum. Well, something like a 9:1 ratio, where 90% of their misses are knowledge based. If this is the assessment and their scores are not where they want them to be, then all the upgraded test-taking skills in the world won’t fix this. When I talk about test-taking, it’s never about beating the test. You can’t do that at the medical boards level. When we talk about test-taking, we’re talking about cleaning up your own user interface, fixing the way you plug into and navigate and use what you know to answer each and every question consistently. But a 9:1 miss ratio, 9 knowledge to 1 test-taking miss, that would require an upgrade in knowledge. So either just more studying or going to a review program, or in my realm, overhauling and upgrading the study skills. We do that in our STATMed class and our various study skills sources.
A third example for the miss ratio might be closer to an even split here, say, it’s a 5:5 miss ratio. This could call for a combo of study skills and test-taking skills, I suppose. But I mean, this is still a staggering occurrence of test-taking issues. 50% of the misses are unforced errors, test-taking misses. Even if the ratio was 6 knowledge to 4 test-taking, maybe even 7 to 3 test-taking, that’s a lot of points being left on the table and that can make all the difference on test day. And to me, that means the test-taking has to be fixed. Thinking in terms of the miss ratio is a great way to start assessing the nature of your test-taking if you think you might be struggling here. I hope this helps illustrate the differences between knowledge and test-taking misses. Thanks for watching.