How to Boost the Effectiveness of Your Program’s Ranking Process
Recruitment is the busiest, and some say the most impactful time of the Graduate Medical Education (GME) year. Overseeing the rank list submission and match process is The National Resident Matching Program® (NRMP®). This private non-profit was established in 1952 to optimize the rank-ordered choices of applicants and program directors for clinical training in the United States. The NRMP provides an impartial venue for matching applicants' and programs' preferences using an internationally recognized mathematical algorithm. In 2021 alone, the Match offered 38,106 total positions and filled 36,179. These were the largest numbers in Match history.
The Match service essentially provides a very intricate dating service used to find the best' partnership' between applicants and programs. Some program directors have even referred to the interview process as "speed dating"! Time is limited, so training programs need to ensure they get to know their potential future trainees as well as possible and allow those trainees to show who they are. Below are a few select issues that could arise before and during the ranking process, with recommendations on how to alleviate them.
- Find an effective and efficient way to organize info and scores of recruits, and do not wait until the end of recruitment season to begin tabulating this info! Recruitment can last months yet tends to fly by. So be sure to work on ranking lists and materials as the season unfolds, instead of leaving oodles to do when time is running out. Have the program manager or coordinator (otherwise known as PM) use an ongoing data collection method during the recruitment season, so not all the work is back-loaded at the end of the season. This should be in an easy-to-follow and understandable format for the whole recruitment team (faculty, chief residents, nurses, administrators, etc.). For example, using an electronic application system like ERAS or Thalamus and having the interviewers input their scores and/or comments directly into the database streamlines retrieving and correlating the information. These systems also usually allow an easy way to export the data later.
- Be sure that all interviewers truly understand the scoring system and know how to complete their assessments of each candidate. This will likely require a significant amount of time early in the season to train faculty. A recommendation would be to host a retreat or create an online module for all those who will be participating in the interview process. Most beneficial is a system for rating the candidates in specific categories of the program's choosing, i.e., test scores, volunteer activities, interpersonal communication skills, etc. The program needs to decide which categories are most important to them, placing certain weights on different categories. The training is worth the time investment as it will put all faculty members on the same page, allow for consistent scoring algorithms, and eliminate confusion.
- Work through intrinsic biases. It is safe to say that most people have some intrinsic biases. These biases may develop based on experiences or interactions with people or situations. Helping committee members understand their own intrinsic biases and teaching methods to see past these will lead to fair and equitable assessments of each applicant. For further reading on this topic, visit one of our recent blog posts: https://www.germane-solutions.com/Blog/371161/Tools-and-Practices-for-Diverse-and-Equitable-Residency-Program-Recruitment.
Use surveys or other tools.
- Send out a post-match survey to the candidates to look for trends. This should be delivered only to those above your lowest-ranked matched applicant (excluding those with whom you matched). Hopefully, you will receive some honest and valuable responses that will help make improvements to the following year.
- Send out a post-interview season survey to your recruitment committee to solicit feedback. Just as valuable as input from candidates, interviewing faculty, chief residents, PMs, etc., to determine what worked and what did not at various stages of recruitment. Then draft a plan to propose and implement the best amongst the suggestions.
- Keep track of important notes throughout the recruitment season. Have the PM keep a running list of all important notes/details/suggestions that come up during the interviews that can make the process better for the next year (or for the next interview day, for that matter). The survey processes above may miss one off-hand comment made during a meeting that could make a meaningful impact towards improvement.
- Find ways to cut costs. One example is eliminating hard-copy application binders for all the interviewers. Instead, perhaps just one master copy for the program director (or someone who may insist on having that hard copy), and ask the other interviewers to review their applications electronically beforehand. Furthermore, the approaches to making the recruitment and ranking process more efficient will streamline faculty's limited (and valuable) time.
- Have two separate rank columns on your master list. Have an 'initial rank' column (based on average numerical score) and a 'final rank' column (after input from all during the rank meeting). This will show how much movement may have occurred during the ranking meeting. Beware; this will require quick fingers and laser focus from the person updating the rank list during that final meeting! Microsoft Excel or other software programs are available to use for your ranking.
So much effort goes into the interview season that we want our ranking process to be as perfect as possible. One last thing to consider is that if the budget allows, offer a meal or some thank you to everyone for a job well done! After all, while recruitment can be very fun, it can also be quite exhausting, and a little appreciation goes a long way. Happy recruiting!
National Resident Matching Program, Results, and Data: 2021 Main Residency Match®. National Resident Matching Program, Washington, DC. 2021