Menu

What SHOULD residency directors use to select residents?

Home The Lounge What SHOULD residency directors use to select residents?

  • Avatar wcinewbie 
    Participant
    Status: Physician
    Posts: 89
    Joined: 09/30/2017

    Im In a small competitive specialty. Doing good on an away / visiting rotation is the best. Second best is phone call with their letter writers.

    #150634 Reply
    Avatar Kamban 
    Participant
    Status: Physician
    Posts: 2487
    Joined: 08/01/2016
    definitely not board scores. board scores (IMO) have very little predictive power for who is a good physician, at least as I assess physicians. rote memorization of facts has less to do with doctoring than ever before. ymmv

    Click to expand…
    High board scores might not indicate great physicians, but low scores will probably indicate the dumber ones.

    Click to expand…

    I used to be of the opinion that board exams are money makers and there are good physicians who are poor test takers, as well as bad physicians who score highly in the tests.

    The latter is true but a well designed board exam does test you knowledge and critical thinking and reasoning powers as I found out in 2015. You might be a good day to day physician but unless you keep up with the journals, meetings, UpToDate and other material you will not be able to keep up with the times and might become a below average physician. The boards try to weed out such physicians. The physicians who I have seen fail in the IM boards have been average physicians who can take care of routine things but the moment they find anything puzzling or complicated they refer them out. The ones who are board certified do a better job of investigating and treating it and referring appropriately.

    Until something better comes along, board exams are better than anything else out there that measures quality ( and I am not in favor of MOC and other bogus money grabbing).

    #150649 Reply
    Liked by Craigy, q-school
    Avatar Monesthesiologist 
    Moderator
    Status: Resident
    Posts: 88
    Joined: 05/11/2018

    Don’t know how to improve on the current process other than to mandate visiting rotations. Much harder to fake it for a month than a day. But this idea is just not practical- interview season could take years.

    #150651 Reply
    SerrateAndDominate SerrateAndDominate 
    Participant
    Status: Physician
    Posts: 487
    Joined: 02/01/2018

    Don’t know how to improve on the current process other than to mandate visiting rotations. Much harder to fake it for a month than a day. But this idea is just not practical- interview season could take years.

    Click to expand…

    I think doing away rotations is a fantastic thing.  I think there are a good bit of backdoor deals involved with students so some get hosed.  In that situation, be realistic about what and where you might end up, then go dominate that rotation.

     

    Sadly, these can be tough for residency.  I had to ruffle a few feathers to get an away approved for fellowship stuff.

    Earn everything.

    #150663 Reply
    Avatar ratherBoutside 
    Participant
    Status: Student
    Posts: 22
    Joined: 09/13/2018

    I think the way that EM does it makes a lot of sense. Force everyone to do away rotations and have a standard letter of evaluation filled out about how this student performed compared to other people who rotated at your institution doing the job of an intern. Was he/ she top 1/3, middle 1/3, or bottom 1/3. Would you want them at your program?

    #150667 Reply
    Liked by MPMD
    q-school q-school 
    Participant
    Status: Physician
    Posts: 2629
    Joined: 05/07/2017
    definitely not board scores. board scores (IMO) have very little predictive power for who is a good physician, at least as I assess physicians. rote memorization of facts has less to do with doctoring than ever before. ymmv 

    Click to expand…
    High board scores might not indicate great physicians, but low scores will probably indicate the dumber ones. 

    Click to expand…

    I used to be of the opinion that board exams are money makers and there are good physicians who are poor test takers, as well as bad physicians who score highly in the tests.

    The latter is true but a well designed board exam does test you knowledge and critical thinking and reasoning powers as I found out in 2015. You might be a good day to day physician but unless you keep up with the journals, meetings, UpToDate and other material you will not be able to keep up with the times and might become a below average physician. The boards try to weed out such physicians. The physicians who I have seen fail in the IM boards have been average physicians who can take care of routine things but the moment they find anything puzzling or complicated they refer them out. The ones who are board certified do a better job of investigating and treating it and referring appropriately.

    Until something better comes along, board exams are better than anything else out there that measures quality ( and I am not in favor of MOC and other bogus money grabbing).

    Click to expand…

    I am talking about usmle step 1 mostly.

    But i continue to challenge you with regards to abms boards.  There are tons of people who have busy lives and work full time.  The opportunity to prepare for boards which may not cover much of the actual work they do is limited.  If you are also busy with kids and other commitments, then i think the field is uneven versus someone who takes the time to study extensively, attend classes etc.  if you are a thyroid only physician, spending a lot of time to study about rare genetic malformations is not useful to your patients and also does not assess your ability to be a good physician.   The score does not reflect your ability, and further i think is far less predictive as you move into years 3-10 after the examination.

    I also do not think that referring complicated things out indicates that someone isn’t a good physician.  As always, it is where the line is drawn.  🙂

    for me, it’s most predictive of who is a good test taker than anything.

    Jmo

    ymmv

     

    #150672 Reply
    Liked by Roentgen, Kamban
    Avatar loeffy 
    Participant
    Status: Physician
    Posts: 61
    Joined: 08/28/2016

    I agree that board scores is probably the best thing going right now.  LORs are very subjective and honestly all say similar things with very subtle “clue” words you have to look for.  Anecdotally, in my med school board scores were a pretty good indicator of the quality of the student overall.  I’m talking student not doctor, because med school is nothing like real world practice and I don’t think we really know what kind of doctor a student will be until they hit residency and actually do real work.  So we just have to do the best we can with what we got.  Having said that, I’ve worked with some high-scoring people who were terrible doctors and vice versa.  I think for fellowship PD’s have more material to work with.

    As for how PDs decide who to interview/rank, it depends on what you want out of your residents.  If they want residents that’ll publish and make a name for themselves and the dept, that can be a different type of applicant than one who can hustle.  Though I think I’d want someone who follows the 3 A’s: available (team player), able (physically and/or mentally) and affable (not weird).  Unfortunately, not all that info is easy to gather with today’s applications and interview process.

    #150797 Reply
    Avatar AR 
    Participant
    Status: Physician
    Posts: 850
    Joined: 03/10/2016

    I’m always a little confused by the idea that med school grades and board scores don’t correlate to how good a doctor someone is going to be.  I guess my experience is highly atypical but, in general, I have found that the students who have higher grades and step scores end up better docs than those who do not.

    Of course, the correlation is not perfect and there are plenty of exceptions both ways.  But I’d challenge anyone to find a metric with a better correlation to ultimate physician quality and is readily obtainable from a large applicant pool.  It just doesn’t exist.

    When I was in academia and involved in residency selection, I just ranked applicants based on their med school performance, step scores and other academic accomplishments.  The interview for me was just a binary thing.  If they were really weird or obnoxious then they came off the list (this happened rarely).  I wasn’t really concerned about someone being a “good fit” in some ill-defined way, because I think that almost all good docs will be a good fit and bad docs will be a bad fit.

    Unfortunately, I was alone in my straightforward approach as my rank list always differed tremendously from all of my colleagues.

     

    #150806 Reply
    Liked by Kamban, MPMD
    Avatar Roentgen 
    Participant
    Status: Physician
    Posts: 75
    Joined: 03/02/2018

    when I was a resident and students came through, my main thought was always would I want to spend 4 years with the person as my colleague, do I want this person on call with me evenings and weekends when we’re the only two people in radiology

    Click to expand…

    Agree with this.  My program had a lot of residency input in our ranking, which I thought was important since we dealt with people the most.  Being in path, I didn’t really care about Step scores, but I liked to see them in relation to their grades (assuming they actually had grades) and rec letters.  I was of the opinion that most people could work in my field, but they had to have the right mindset and work ethic.  We had some great applicants with great scores who we DNR’d because we didn’t want to be stuck in a room with them more than the interview day.

     

    When I got to interview applicants, I always asked them about jobs they held before med school. That was one good way to tell me a lot about them and see who wasn’t afraid to work.

    Click to expand…

    “Have to have the right mindset and work ethic.”  And I think you imply functional and preferably interesting personality as well, given that in my rads, like path, you have to sit next to them all day every day for five years.

    #150807 Reply
    Avatar Roentgen 
    Participant
    Status: Physician
    Posts: 75
    Joined: 03/02/2018

    The discussion on another thread suggests that AOA membership may not be the marker of quality it is assumed to be, but in response another forum participant said “it is better than having no markers at all to help differentiate between applicants.”

    That got me thinking – in an ideal world, what SHOULD residency program directors use to select their residents?

    In my field (Radiology), I propose we should have applicants:

    1. take a vision test (obvious reasons)

    2. Take a hand-eye coordination test (will they be capable in interventional work?)

    3. Take an IQ test (are they smart?)

    4. Take a test to assess them on the Big Five Personality Survey Inventory (I’d prefer applicants who score high on Conscientiousness and Emotional Stability)

    I’d much rather select people on the basis of that objective information than rinky-dink one-day interviews that anyone can fake their way through.

    What you you folks use to select the best residents in your residencies?

     

     

    Click to expand…

    I’d be reluctant to add the personality tests, especially after reading the WSJ article from Saturday about the Myers-Briggs test. The author of the article made strong claims about the lack of scientific validity of the MBTI.

    Sitting one ones butt, on the other hand, might be a valid indicator for the field, except IR (sorry! couldn’t resist!).

    In addition to the usual academic metrics, and extended 4th year rotation can allow an opportunity to assess some of the candidate’s soft skills and work ethic.

     

    Click to expand…

    Meyers Briggs is discredited.  I haven’t seen/read where the Big Five personality inventory has been, but maybe someone else has?

    #150809 Reply
    CordMcNally CordMcNally 
    Participant
    Status: Physician
    Posts: 2846
    Joined: 01/03/2017

    I still think board scores/grades along with an interview is about the best thing at this point. I think IQ tests and personality tests would be (are) pointless.

    “But investing isn’t about beating others at their game. It’s about controlling yourself at your own game.”
    ― Benjamin Graham, The Intelligent Investor

    #150810 Reply
    Liked by Craigy
    Avatar Roentgen 
    Participant
    Status: Physician
    Posts: 75
    Joined: 03/02/2018

    Ideal selection criteria should not be specified broadly and will be different for each specialty and even each program perhaps.  The earlier question of AOA revolves around what medical schools should do to provide quality transparency to residencies.  Unfortunately we don’t have better markers for things like diligence, good character, and ownership, qualities that trump step 1 scores in my opinion for safe and compassionate care.  This information, however, can be determined on rotations and constant exposure, which is why rotating students and home products (who are good) are preferred.  You’d easily take a Step 1 hit on a student who has shown diligent, hard work and takes ownership of patients.  Take more off if they’re a good person to just have around.  I think a “good hands” test at the interview is not unreasonable, but this can be practiced and gamed once the residency program’s method of assessment is known.  So it loses its validity.  Regarding IQ, I’d bet a lot of money that friends I had growing up who flunked out of college have higher IQs than the vast majority of physicians I’ve ever met.  Work ethic, grit, and determination were the great differentiators there, as they tend to be in life.  I’d rather have transparent markers for those qualities than the ones we have now (Step I, AOA), but on the average those two items do speak to work ethic IMO.

    As an aside, I’d love to see a Step 1 score regression analysis based on single, married, married with kids status in the first 2 years of med school.  Without knowing that data, someone who had 3 kids, a spouse, and who pulled off a 240 with publications makes me raise my eyebrow a lot more than the single person who got a 260.

    Click to expand…

    You wrote “Regarding IQ, I’d bet a lot of money that friends I had growing up who flunked out of college have higher IQs than the vast majority of physicians I’ve ever met.”

    Okay I totally don’t have research citations to back this up, but we were told during medical school by some “official” person at my school that I don’t specifically recall that the average IQ of a med student is around 120, and the average IQ of Ph.D. (medical and nonmedical) is about 140-145.  Basically, the message was docs are very bright, but “geniuses” find the vocational-technical aspect of medicine rapidly boring and tend not to go into medicine (or stay purely in practicing medicine).

    I’d love to know if that is true.  Or false.

    #150814 Reply
    CordMcNally CordMcNally 
    Participant
    Status: Physician
    Posts: 2846
    Joined: 01/03/2017
    Okay I totally don’t have research citations to back this up, but we were told during medical school by some “official” person at my school that I don’t specifically recall that the average IQ of a med student is around 120, and the average IQ of Ph.D. (medical and nonmedical) is about 14-145. Basically, the message was docs are very bright, but geniuses get find the vocational-technical aspect of medicine rapidly boring and tend not to go into medicine (or stay purely in practicing medicine).

    Click to expand…

    I’d find this incredibly hard to believe. Not necessarily that there are occupational groups smarter than physicians, but the average IQ of all Ph.Ds. is 140-145. I feel like I don’t need any resources or citations to completely debunk that.

    “But investing isn’t about beating others at their game. It’s about controlling yourself at your own game.”
    ― Benjamin Graham, The Intelligent Investor

    #150816 Reply
    Liked by Craigy, Roentgen
    ENT Doc ENT Doc 
    Participant
    Status: Physician
    Posts: 3517
    Joined: 01/14/2017

    Ideal selection criteria should not be specified broadly and will be different for each specialty and even each program perhaps.  The earlier question of AOA revolves around what medical schools should do to provide quality transparency to residencies.  Unfortunately we don’t have better markers for things like diligence, good character, and ownership, qualities that trump step 1 scores in my opinion for safe and compassionate care.  This information, however, can be determined on rotations and constant exposure, which is why rotating students and home products (who are good) are preferred.  You’d easily take a Step 1 hit on a student who has shown diligent, hard work and takes ownership of patients.  Take more off if they’re a good person to just have around.  I think a “good hands” test at the interview is not unreasonable, but this can be practiced and gamed once the residency program’s method of assessment is known.  So it loses its validity.  Regarding IQ, I’d bet a lot of money that friends I had growing up who flunked out of college have higher IQs than the vast majority of physicians I’ve ever met.  Work ethic, grit, and determination were the great differentiators there, as they tend to be in life.  I’d rather have transparent markers for those qualities than the ones we have now (Step I, AOA), but on the average those two items do speak to work ethic IMO.

    As an aside, I’d love to see a Step 1 score regression analysis based on single, married, married with kids status in the first 2 years of med school.  Without knowing that data, someone who had 3 kids, a spouse, and who pulled off a 240 with publications makes me raise my eyebrow a lot more than the single person who got a 260.

    Click to expand…

    You wrote “Regarding IQ, I’d bet a lot of money that friends I had growing up who flunked out of college have higher IQs than the vast majority of physicians I’ve ever met.”

    Okay I totally don’t have research citations to back this up, but we were told during medical school by some “official” person at my school that I don’t specifically recall that the average IQ of a med student is around 120, and the average IQ of Ph.D. (medical and nonmedical) is about 14-145.  Basically, the message was docs are very bright, but geniuses get find the vocational-technical aspect of medicine rapidly boring and tend not to go into medicine (or stay purely in practicing medicine).

    I’d love to know if that is true.

    Click to expand…

    Here’s an interesting article:

    http://www.iqcomparisonsite.com/Occupations.aspx

    #150821 Reply
    Liked by Craigy, Roentgen
    CordMcNally CordMcNally 
    Participant
    Status: Physician
    Posts: 2846
    Joined: 01/03/2017
    Here’s an interesting article: http://www.iqcomparisonsite.com/Occupations.aspx

    Click to expand…

    I had never heard the term sexton used to describe an occupation. It isn’t what I thought it might be.

    “But investing isn’t about beating others at their game. It’s about controlling yourself at your own game.”
    ― Benjamin Graham, The Intelligent Investor

    #150824 Reply
    Liked by Craigy

Reply To: What SHOULD residency directors use to select residents?

In case of a glitch or error, please save your text elsewhere, clear browser cache, close browser, open browser and refresh the page.

Notifications Mark all as read  |  Clear