Peer IX Vs Rosh Review Vs Board Vitals – Studying for the Boards

[Editor's Note: A couple of weeks ago I mentioned a new affiliate partner I have, ELFI, a student loan refinancing company known for low rates. The ELFI link and WCI deal ($325 cash back) are now live, so if you've been waiting for this one, now is the time! Also, if you are going to the ACEP Scientific Assembly at the end of the month, you can come meet me on Monday night (Oct 30th), get a free copy of my book, and some free food, but you do have to RSVP. ]

For about three months this summer I was pretty grouchy. If you didn't notice it in my comments or emails, I'm glad, but my family certainly did. Why was I grouchy? Because it was time for me to take the ABEM ConCert Exam in order to maintain my emergency medicine board certified status. This test is very similar to the original qualifying (written) EM board exam, but it has been 11 years since I last took a high-stakes multiple choice test. Since my hospital medical staff requires me to maintain board certification, I had to pass this test if I wanted to keep practicing. In reality, you can fail it once or twice, but the last thing I wanted to do was spend all my available time for months studying for it AGAIN next year. Plus, I didn't want to shell out another $1,850 to take the 5 hour computerized test at a local Pearson testing center. And pay hundreds of dollars for preparation materials. Which brings us to the subject of the post– preparation materials.

PEER IX Review

A well-known resource in Emergency Medicine is put out by the American College of Emergency Physicians and called PEER IX. PEER stands for Physician's Evaluation and Educational Review and this is the ninth version of it as it gets updated every couple of years. It is basically a 460 question bank with a free pre-test. ACEP charges its members $295 ($0.64 per question) for this resource but throws in the last version (PEER XIII, another 350 questions, lowering the price to $0.36 per question) for free. So the first thing I did was take the free pre-test. I scored something like 64%, which basically sent me into panic mode since a passing score on the ConCert exam is 75. However, I confess that I was immediately suspicious that the pre-test questions were harder than the actual questions on the exam in order to get me to buy PEER IX. More on that suspicion later. There was also a written companion to the question bank, for another $118. While I hate “doctor pricing”, that was better than the other review book I used for the original exam that currently costs $400. I passed on both of the books this time and shelled out my $295 for the question bank.

I did the entire PEER IX question bank, ending up with a score of 75% correct. I also did a few of the PEER VIII questions. Overall, I think the PEER IX questions were the most similar in style and length to the ones on the actual exam, although the writers of the questions say they have no association whatsoever with the writers of the actual test. I did find the questions to be significantly more difficult than the ones on the actual exam.

Now, when using a question bank, the quality of the explanations of why some answers were right and some answers were wrong is pretty important. The idea is that you want to avoid having to go to another resource to look something up as little as possible. In this aspect, I found the PEER IX explanations adequate most of the time. They were certainly better than the PEER VIII explanations, which I found so bad that I moved on to the next question bank instead of doing most of them. Here's an example of what you get with PEER IX:

That's pretty nice. You get a comprehensive explanation of the right answer, a brief explanation of the wrong ones, and usually a few extra tips below in the Peer Point and Peer Review sections, although those usually just include info that was in the comprehensive explanation above. But compare that to what you get with PEER VIII:

That's it. Not nearly as comprehensive as the explanations with PEER IX. There were also lots of references included with the PEER resources, but frankly, that was a waste. I mean, who's going to go look that stuff up? If I have a question about a subject, and the explanation provided wasn't adequate for me to refresh my understanding, I'm just going to look it up on UpToDate or even Wikipedia. This isn't cutting-edge research we're doing here, it's studying for the boards. I'm not going to the original literature.

Another annoyance I had with PEER was the CME system. While they do give you free CME with it (up to 150 hours) they only let you claim it after a practice test if you scored at least 75 on the practice test. Well, since I averaged 75, I clearly didn't get a 75 on every test. So I didn't get to claim anywhere near 150 hours, not that I'm running short anyway, but still. Very annoying. In the end, I claimed 15 hours for it but spent far more time than that with the database.

Rosh Review

After finishing PEER IX, I still had over a month until the exam so I looked around at what I was going to study next. I was intrigued by a private company called Rosh Review. Founded by Adam Rosh, an emergency physician in 2012, it enjoys a pretty good reputation among emergency docs studying for their written tests. But Rosh Review offers a little more than just EM questions. They also have FM, Peds, Peds EM, PA, and NP questions. So I emailed Adam and said, “Tell you what, if you let me use your database free for a couple of months, I'll include it in a review I publish on my blog.” He graciously gave me access. Which is good, because being the cheapskate that I am, I really didn't want to spend another $419 (30 days) or $449 (90 days) for it.

In reality, I probably would have spent it if I had to and been glad that I did. I'm not sure what motivated Adam to start this business, but he clearly took a look at PEER IX, saw the problems with it, and figured he could do a better job, offer a great resource to the EM community, and make a buck on the side. I think he succeeded admirably. Let me explain why:

With PEER you get 460 questions (or 910 if you count PEER VIII) with some “adequate” explanations. With Rosh Review, you get 2,141 questions ($0.21 cents per question) with perhaps the most beautiful explanations on the planet. I mean, I saw pictures of conditions I don't think I'd ever seen pictures of before. Wanna know what Koplik spots look like? Rosh will show you. These explanations were gorgeous. Let me show you what I mean:

Pretty impressive huh. I like it when they tell me how many of my peers are getting the question right (and which wrong answers they're choosing), but PEER did that as well. But what I really liked was that every question in the database had a little bonus question down below, the “One Step Further” question. So in reality, it wasn't just 2,141 questions you were getting, but rather 4,282! That's just 10 cents a question!

But as you can see, the explanations are way above and beyond what you're going to see anywhere else. It was very rare for me to need to go look up something in another resource while using the Rosh Review question bank. This is the real strength of Rosh.

Another great feature was the ongoing feedback about your progress.

It tracks your progress as you go along to see if you're getting smarter. I don't know how I aced the first two tests I took, but I became more consistent as I went along. Don't pay any attention to the last 8 tests or so. That was Whitney playing around with it the night before I took the test. This was my favorite screen:

Not only does it track your progress as you go along and give you your overall percentage correct, but it also projects your ConCert score and your probability of passing. I thought this feature was really cool and very reassuring. I ended up doing about 700 of the 2141 questions with an overall average of 82% and it was projecting my score to be 91 by the time I was done. But that gave me a lot of confidence going into the test that I was going to be just fine.

What didn't I like about Rosh Review? Well, it's the most expensive of the bunch to start with. I mean, you can cheap out and just get it for a month, but I can assure you that you aren't going to get through all 2,141 questions in that time period. Plus, it doesn't give you free CME. You have to spend another $100 to get 100 credits. Granted, that's still pretty cheap CME, but you'd think if you were dropping that kind of money on software that they could throw in the CME credits for free. I found the questions to be the easiest of the three databases I used, but still slightly harder than the actual exam. Obviously Rosh agrees given that even at the end most of my scores were in the 82-86% range and they were projecting me to get a 91. That was pretty darn accurate, given that my final test score was 93. I was pretty happy with that, since it was higher than my original written boards score. At least until my climbing partner told me he got a 98 despite studying less. Punk. Incidentally, Rosh was all he used, but at least he had to pay for it!

[Update 6/2018: Adam Rosh tells me they've increased the database to 5,000 questions, dropping the cost per question to $0.10.)

Board Vitals Review

Board Vitals is an affiliate partner that has been with me for over a year with a few scattered ads. In that time period, I haven't managed to get a single one of you to purchase their services. That's really too bad, not only because I haven't made a dime, but also because they offer a great database.

In fact, they offer dozens and dozens of databases for physicians, nurses, students, podiatrists, dentists, pharmacists, PAs, NPs, and even naturopaths. There is everything from the USMLE and COMLEX to pediatric neurology, radiology, and sleep medicine. I count 132 different tests you can prepare for using a Board Vitals question bank. Obviously, I was only interested in one of those, so I asked for a freebie in hopes that I could eventually get some of you to check out this sweet resource. They graciously granted me access to the database for a couple of months, although just like with Rosh Review, didn't get to give any input toward this review and in fact didn't see it any sooner than you did. I did the Board Vitals questions at the same time as Rosh Review, both after PEER.

The emergency medicine database contains 623 questions and goes for $249 for a month, or $429 for three months. That works out to $0.39 or $0.69 per question. That's not quite the value (price per question) you get with Rosh Review, but it is better than you get with PEER (assuming you get through the database in a single month). And who really has time to do 2100+ questions anyway? At $249, it is the cheapest of the three databases. Unless you want the CME, which will cost you another $200. Knock 10% off all those prices if you go through the links on this page.

I found the questions to be the hardest of the three databases and far harder than the test itself. In fact, they predict that if you can get at least 61% on their database that you'll pass the ConCert exam. But they had some questions that were real doozies. Like stuff I would call the Poison Control Center to ask. And that the toxicologist would have to look up in a book to give me an answer. They had quite a few questions on medication side effects that were particularly challenging as well. There were also a few questions that seemed like they had been stolen from their databases for other tests. For example, the vignette would read about how the patient came into the urgent care or into a clinic. Those questions left me wondering why I was taking care of them if they just walked into a clinic. But overall, good solid emergency medicine questions. Nice and hard and humbling. The explanations were not Rosh quality, but adequate and about on par with what you get from PEER. Take a look:

This is pretty typical of what you see with Board Vitals. A tough question, perhaps on the edge of what is really expected of an emergency physician to know. It lets you know how many people got it correct, although doesn't tell you what other test takers were guessing like PEER and Rosh do. In this case, 78% of exam takers got it right, which I found surprising since I missed it.

As I worked my way through the database, I was surprised by how many questions less than half of the takers got right. In some cases, only 25-30%. I felt good about myself, since I got many of those right, but I was worried that there were apparently many emergency physicians missing them. Which made me think that maybe those questions were being presented to other test-takers as part of other databases, thus explaining the low % correct.

They also give you a nice timeline (like Rosh but unlike PEER):

This report was nice to see as well:

Just like Rosh, it will break it down for you by category in case you only have time to do the categories you really suck at, but I was mostly interested in the top line. I think in the end I got 79% of the database correct. That put me at the 68th percentile as the median score was 69%. And again, since they predict you need 61% to pass the ConCert, that made me feel pretty good.

The Bottom Line

In the end, I liked different things about each database, so it is hard for me to give you one recommendation over another.

I think the PEER questions were most like the ones I actually saw on the test. But they don't do anything flashy, the explanations are merely adequate,  and they charge you quite a bit for the number of questions they're giving you ($0.64 per question). But you do get the CME for free, even if you might have to take the tests twice to actually claim it!

Rosh Review is the premium product in the space. Way more questions (more than I did between all three databases in three months) and gorgeous explanations. But they also charge like it, at $449 for three months plus another $100 for CME. It offers maximal value at $0.21 per question, assuming you can get to all those questions in three months. It is pretty clear that Adam Rosh is an emergency doc, as this is a pretty EM-focused business.

Board Vitals is the discount product on the market at the low, low price of $249 (minus 10% if you use the links on this page.) It offers good value at $0.39 a question ($0.35 if you use my links) and is definitely going to motivate you to study given the difficulty of its questions. The questions aren't quite as applicable to the exam as the other two databases, but there are still areas that showed up on the exam that the Board Vitals database prepared me for and the other two did not. Board Vitals also has the huge advantage that it isn't just an EM focused product. While Rosh is branching out, they've got a long way to go to catch up to Board Vitals as far as other specialties go.

In the end, which of these you buy should come down to what you're looking for. If you want it all, just buy all three databases. I was glad I used them all. They all helped me raise my score. Sure, it'll cost you a cool $1,000, but when the test costs $1,850 to retake and when you consider the value of your time, that really isn't all that much money.

If you're looking for a deal, well, you get over 600 questions with Board Vitals for just $224. If you're either not planning to study much or doing most of your studying using a course or a book and just want to take some practice tests, that should be adequate.

If you want to pick something in the middle, you won't run out of questions going with Rosh Review or you could combine Board Vitals with PEER to again get plenty of questions for not much more than you pay for Rosh.

If CME credit is really important to you, PEER is likely your best bet since the CME is included, but you can get it from any of the three databases for a little more money.

If convenience is your main concern, it's hard to go wrong with Rosh Review. You make one purchase where you get plenty of questions, a good predictor of how you'll do, CME credit at a reasonable price, and almost all the information you need to study without having to purchase another resource.)

Thanks again to Rosh Review and Board Vitals for giving me free access to your fantastic question banks and your service to the medical community.

What do you think? How did you study for the EM boards? If you've used more than one question bank, which one did you like best? If you've used Rosh or Board Vitals for another specialty, how did you like it? Comment below!