Introduction

The COVID-19 pandemic disrupted medical student curricula with suspension or postponement of educational programs or clinical rotations. These disruptions have affected medical students’ career plans, future interests, readiness to take the next step in their training, and personal well-being [1, 2]. Residency training programs also suffered without traditional methods to connect with applicants. Orthopedics, for example, historically depended on face to face experiences in the form of away rotations with more than half of students matching at an orthopedic institution at which they had an in-person experience [3]. Previous clinical experience benchmarks used to assess competitiveness and readiness became no longer attainable or relevant. The crux of the student’s application shifted to other aspects, including STEP scores, letters of recommendation, and clinical clerkship performance. Meanwhile medical students reported concern over decreased opportunities to adequately develop a competitive application, citing low pertinent clinical experience and low confidence in receiving quality letters of recommendation [4, 5]. Added to this stress is the concern of exposure to the virus [6]. To cope with the increased uncertainty, some applicants are applying to more schools, and residency programs are receiving more applications [7]. However, our institution did not extend more interview offers than prior years and a survey of orthopedic surgery program directors revealed that they did not increase the number of interviews conducted, likely leaving applicants with a less than anticipated number of interview offers [7].

Beyond the dedicated interview time, the in-person visits facilitated program exploration. Applicants could vet current faculty and residents as potential coworkers and experience the hospital and city where they are looking to commit the next five to seven years of their life. Programs have attempted to replicate the interview process with virtual happy hours and recorded hospital tours. While the online format has been well received, applicants and program directors alike endorse a preference for the traditional in-person format overall [5, 7]. The benefits include saving money and time on travel and minimal interruption on training; however, adequately assessing program culture and networking with current residents have been widely cited shortfalls [5, 8].

We sought to understand the general surgery applicant perspectives during the 2021 Match cycle, so that we can make targeted improvements in the applicant and program experience to build confidence and satisfaction in the Match process. We were particularly interested in the applicant cohorts’ post-interview behaviors, understanding which program factors were most influential on rank list order, and applicant satisfaction and happiness with a virtual Match cycle.

Methods

Two investigator-generated surveys were created with oversight from a departmental surgical education research group and were distributed to all applicants who interviewed at our institution for a categorical general surgery residency position during the 2021 application cycle. The first survey considered the behaviors of applicants after interview day and asked what factors were most influential in generating their rank list order. The first survey link was distributed one week after applicant finalized rank list submissions were due (February 2021). The second survey was administered one week after Match Day (March 2021) and assessed applicant interactions and satisfaction with their Matched program. The surveys were hosted in REDCap (Research Electronic Data Capture), a secure, web-based software platform. Study data were collected and managed using REDCap electronic data capture tools.

Questions used a combination of yes/no, 5-point Likert scale style, and ranking values to capture the applicants’ experience and perspective. Example questions included ranking specific program factors from most to least influential and general behavior questions regarding the application process, such as “did you visit the city of the residency programs at the top of your rank list?” Factors deemed most important had the highest percentage response by applicants in the ‘most important’ ranking category. Factors deemed least important had the highest percentage response by applicants in the ‘least important’ ranking category.

Descriptive data were collated in tables and graphs. Variable analysis was determined based on the underlying distribution. For normally distributed variables, means with standard deviations were used. For non-normally distributed variables, medians and interquartile ranges were utilized. The Kruskal–Wallis test compared continuous covariates between both groups while the Chi-squared or the Fisher’s exact test compared categorical variables. Comparative analysis was conducted on the sample mean of the 5-point Likert scale. A p value of 0.05 was used for statistical significance. Data were analyzed using the R statistical software version 3.6.3 (R core Team 2015, Vienna, Austria).

Results

Two distinct anonymous surveys were distributed via email to one-hundred and eight categorical general surgery interviewees. The first survey was administered one week after the rank list deadline with a 40.7% (n = 43) response rate. The second survey was administered one week after Match Day with a 29% (n = 31) response rate. The median age was 26 and the cohort was 60.9% male and 39.1% female. The majority were non-Hispanic white (82%). Applicants completed a median of 17 interviews (Fig. 1). There was a trend towards male applicants having more interviews than females (18.5 vs 16, p = 0.16). 55.8% of applicants endorsed ranking all programs at which they interviewed. There was no statistically significant association between the number of interviews and the decision to rank all programs or not (p = 0.33), and there was no association between those who reached out to programs having a higher number of interviews (p = 0.83).

Fig. 1
figure 1

Box and whisker plot of the number of interviews applicants completed. Line represents median (17, range 10–47, n = 43)

Full size image

Applicant values

The most influential interview day factors when ranking residency programs were the faculty interview (40.5%), resident social (21.4%), and resident interview (16.3%). Least important on the interview day was the ability to assess the hospital facilities (Fig. 2). Among general program factors, applicants most highly valued camaraderie between program faculty, residents, and staff (31%), current resident happiness (18.6%), and program reputation (14.3%) when making their rank list. The least important program factor was diversity of faculty and residents (36.6%) (Table 1).

Fig. 2
figure 2

Applicants’ ranking of most to least influential factors on interview day. A Pictorial representation B Response percentages, highest percentage response highlighted

Full size image
Table 1 General program factors ranked most to least influential by applicants
Full size table

Interview behavior

Figure 3 shows that 44.2% of applicants did not rank all programs at which they interviewed. Most applicants had not visited the city of their top ranked program (58.1%). 55.8% of applicants contacted programs after their interview, and 69.8% endorsed feeling they had sufficient exposure to their top program.

Fig. 3
figure 3

Interview behaviors reported by applicants

Full size image

Program exposure

There was a trend of female applicants endorsing they had more sufficient exposure to a program (75%) when compared with male applicants (50%, p = 0.2). Of the applicants who reported adequate exposure, 60% were confident in their decision compared with just 40% of those who did not feel confident in their impressions (p = 0.005). Those who did not feel confident in their impression of programs attributed it to insufficient exposure (88%).

Match statistics

Applicants matched at programs that were in a city known to them (56%), compared to our pre-Match survey finding of only 41.9% stating they had been to the city of their top ranked program. 48% of applicants matched at a program they contacted, while 55% of applicants stated they contacted residency programs after the interview. Only 41% felt they had sufficient exposure to their matched program, compared with 69.8% endorsing sufficient exposure to their top ranked program on our pre-Match survey.

Satisfaction with the match

79% reported being happy or very happy with their results in the Match. More female applicants endorsed being happy with the results of the Match (85%) compared with 77% of the males (p = 0.24). There was no significant association between being happy with the results of the Match and the number of interviews the applicant had (p = 0.15). In fact, the group who endorsed post-Match happiness had a median of 19 interviews (range 16–22), while the group who were neutral or unhappy had a median of 24 (range 24–24.5).

Overall, a quarter of respondents felt they would have matched differently if interviews had been in person rather than virtual. Of the unhappy applicants, 43% believed they would have matched differently had interviews been in person, compared to 20% of happy applicants (p = 0.21). For those happy with the results, 83% were confident in ranking versus 79% were not (p = 0.09). 94% of the people who reached out to residency programs after the interview indicated that they were happy with the match, compared to 65% of those who did not reach out to programs (p = 0.05). Of those who ranked all programs at which they interviewed, only 42% indicated being happy versus 54% who did not rank all programs (p = 0.54).

Discussion

Applicants want to see happy residents

Applicants expressed satisfaction with the virtual match process, but some behaviors differed from prior in-person cycles. Applicants completed a median of 17 interviews, which is higher than historically reported median general surgery interviews (12.72) [9]. As programs did not endorse increasing interview offers, a stratified highly competitive layer of applicants may have took up a larger number of interviews without the traditional time and travel conflicts. Completing more interviews was linked to less happiness with the final match results in this study, which may be a result of these highly competitive applicants all ranking a small subset of programs and matching lower than anticipated. It may also be a consequence of being ‘spoilt for choice’; with greater exposure to over 24 programs, there may be a tendency to see what is lacking in their Matched program.

44% of applicants did not rank all programs at which they interviewed, which anecdotally was less common in prior years. One reason for this may be that virtual interviews are a less salient experience and applicants may be less likely to imagine themselves at that program. Applicants, despite no in-person contact, placed the highest weight on perceived camaraderie of current faculty and residents. Program camaraderie and current resident happiness were the top two most important factors when ranking programs for 84.9% of applicants. These qualities overtook some traditionally stressed program factors like early surgical exposure and program reputation.

This finding poses a true challenge for residency programs. In the virtualized Match cycle, applicants remotely conference with faculty one-on-one or two-on-one, which rarely affords the opportunity to observe how faculty interact with one another, and particularly limits observation of any faculty-resident interaction. Allowing prospective trainees to virtually attend grand rounds, morbidity and mortality conferences, or faculty-led educational sessions may better inform applicants to faculty-resident comradery. Recent trends indicate a heavy emphasis on resident interaction and resident participation in the interview process [10]. Overall applicant responses placed a higher weight on programs’ cultural factors, which are more difficult to demonstrate virtually, compared with objective data such as surgical volume.

Despite the current efforts to promote diversity within medicine, especially from institutions, applicants were overall less concerned with diversity within the faculty, as it ranked at the bottom of the list of factors influencing their rank lists. This result is quite surprising, and may not represent the overall population of applicants, as the survey respondents identified as non-Hispanic white at a rate disproportionate compared to the larger cohort.

Regional bias may affect programs more than applicants

Applicants were willing to explore diverse geographic regions for their surgical training, as 58.1% of them had never been to the city of their top ranked program. These bold choices appeared one-sided however, as only 44% matched in these cities. This could represent an element of familiarity or regional bias, stronger on the residency program side, and echoes a concern that applicants expressed with the loss of away rotations and inability to demonstrate interest in a different geographic region than their home program. With virtual interviews, applicants are no longer stratified by willingness to travel to a different region. Programs’ selection of trainees from their region may be due to applicants historically wishing to stay in the same region or that regional faculty and letter writers were better known, thus a safer choice when limited impressions could be made. Programs may have been surprised at the number of geographically diverse interviewees and not been able to evaluate the strength of intent to relocate.

Not only were programs challenged with determining interest, but applicants had to successfully sell their own commitment to programs. In a virtual setting, many nuanced nonverbal communication cues, like body language, can go unnoticed or be misinterpreted. Compared to in-person interviews, applicants find that virtual interviews are less successful in allowing applicants to accurately represent themselves [11]. Medical students and residents alike have indicated there are substantial difficulties in the ability to connect with ones interviewer virtually compared to in-person [12]. Without being immersed in the interview day experience, applicants may have had a more difficult time getting a sense of program culture, and articulating their interest.

Overcoming poor program exposure

Not only were applicants more comfortable than programs with expanding geographically, they also indicated sufficient exposure to their top ranked program more often than their resultant match. Only 41% felt they had sufficient exposure to their matched program, compared to 69.8% endorsing sufficient exposure to their top ranked program. This suggests that top ranked programs were more successful in providing a feeling of sufficient exposure than others. From our data, we can extrapolate that the top ranked programs must have been exemplary in their display of factors the applicants most cared about: resident happiness and program camaraderie. Further research will be necessary to identify how they achieved this and what more specifically led to these top ranked programs giving applicants a more satisfactory interview day.

Moving forward, our program has increased our web page usability, social media presence, and has showcased resident-resident interactions via virtual applicant happy hours. We are continuously evaluating and supporting additional methods of communication and interaction by which applicants can gain an impression of some of the intangible aspects of our program. Understanding the complexities of virtual interviews from the applicant perspective may lead programs and future trainees alike to a more successful Match.

Future directions and limitations

Limitations of our study include our overall small and homogeneous cohort, which did not allow meaningful analysis of any differences in race or ethnicity. Further, this population was not representative of the more diverse applicant pool that interviewed at our institution. Diversity as a program factor was ranked least influential by this primarily Caucasian cohort and may not reflect the value of this factor by non-Caucasian applicants. Focusing on only categorical general surgery applicants who interviewed at our institution limited the number of responses and perspectives received. Many respondents were happy with the result of the Match, which may have influenced their willingness to participate in the survey compared to those who were unhappy with the Match process.

Future research is needed to determine if sub-populations such as preliminary residents and fellowship or international applicants may see benefit from a permanent transition to a virtual option. Yet to be seen, however, is a comparison of in-person versus virtual interviewees in the same match cycle.

Conclusion

Even in the constraints of the virtual interviews, most general surgery applicants felt they had sufficient exposure to make their rank list and were happy with the Match. Surgery applicants were willing to highly rank cities they had never visited and to reach out to programs, but they were ultimately less successful matching at those programs.