INTRODUCTION

The Maintenance of Certification (MOC) exam requirement is a highly debated aspect of physician board certification.1,2,3,4 Little is known about what physicians actually do to prepare for these exams and how preparation could influence patient care.

Past research suggests that imposition of the American Board of Internal Medicine’s (ABIM’s) MOC requirement, including an exam, resulted in meaningful improvements in practice, including decreased costs overall and increased beneficial mammography screenings.5, 6 Other studies report that both maintaining certification and better performance on MOC exams are positively associated with meeting process of care standards7, 8 and negatively associated with disciplinary actions.9 However, the mechanisms underlying these associations remain largely unstudied.

Our study aimed to inform debates around MOC by exploring how MOC exam preparation may have affected physicians’ knowledge and patient care, through interviews with 80 primary care physicians who had recently completed an MOC exam.

METHODS

Data Collection

We conducted semi-structured phone interviews, each lasting 45–60 min, with 40 primary care physicians each from ABIM and the American Board of Family Medicine (ABFM) who had recently completed an MOC exam (in 2016). We selected this group because they represent the largest body of primary care physicians in the USA.10 At the time of our study, ABFM required physicians to complete an MOC exam every 7 to 10 years; ABIM required an exam every 10 years for physicians initially certified after 1989.

Two anthropologists with expertise in interviewing physicians (BC, AE) led the interviews, which covered the following: physicians’ practice characteristics; approach to studying; materials used; impacts on personal and professional life; and what, if anything, they learned in the process. Additionally, some interviewees volunteered their opinions regarding the value of MOC in general (see Online Appendix sections 1 and 2). The interview guide (see Online Appendix section 3) was designed to review these topics through a natural-feeling conversation that also allowed interviews to go off-script to pursue new themes introduced by interviewees. Draft guides were piloted with 5 recent MOC exam takers to refine the questions and to verify all questions could be completed within the allotted time. Interviewers emphasized throughout that we sought interviewees’ honest and complete accounts of their MOC exam preparation experience.

While interviews covered a variety of topics, this paper focuses on interviewees’ responses to questions related to what they learned and how this may have affected patient care.

Recruitment and Sampling

The recruitment sample included physicians who reported spending most of their practice time in outpatient primary care settings (see Online Appendix section 4 for specifics). We recruited a diverse sample with a variety of perspectives, including different opinions about MOC and MOC exams, by emailing weekly batches of invitations to 20–60 physicians from the pool of possible participants, followed by email reminders over the subsequent week. The first set of invitations was random; later, we adjusted the mix to develop a representative overall sample in terms of attitudes toward the MOC exam, gender, in academic or solo practice, and whether interviewees had passed the exam on their first attempt. We ceased invitations after completing 40 ABIM and 40 ABFM interviews. It is commonly held that 15 to 20 interviews are generally sufficient to ensure thematic saturation11; we interviewed more than this to gather a broad range of experiences.

Data Analysis

A team of six coded transcripts, using the MAXQDA 2018 software, starting with a priori codes drawn from the guide and creating new codes as these emerged during analysis. To establish inter-rater agreement, all team members coded a set of interviews, then compared and reconciled results; next, all interviews were coded by at least two researchers who further compared coding throughout. We analyzed the coded interviews using the constant comparative method,12 then further analyzed selected portions, including a review of clinical content by physicians from each board (LL, LP).

All participants gave informed consent. The American Academy of Family Physicians IRB approved the study.

RESULTS

Overview

Interviewees described how exam preparation could affect patient care through the following: the types of information studied, knowledge acquired, and resulting direct or indirect changes to practice (see Fig. 1 for overview).

Figure 1
figure 1

How MOC exam preparation can affect knowledge and patient care.

Full size image

Demographics

As shown in Table 1, the interviewed group was very similar to the sampling frame. For example, the percentage of interviewees who said the MOC exam was unfair compared to the larger population was the same for ABFM physicians (25% in our sample and in the larger population) and very similar for ABIM physicians (27% in sample, 28% overall) (see Online Appendix section 2 for the analysis of selected quotes describing interviewees’ reasons for participating). Table 2 shows the years since initial certification.

Table 1 Characteristics of Sample Interviewed and Test Takers That Met the Study Inclusion Criteria
Full size table
Table 2 Years Since Initial Certification
Full size table

Preparation

Seventy-eight of 80 physicians devoted extra time and resources to studying different contents and using different study approaches than when not preparing for an exam. Sixty-six of 80 reported at least 2 months of studying, which could be concentrated (as in live review courses) or spread over months or years.

Physicians described how exam preparation differed from normal engagement with journals or conferences: studying could include reviewing material studied before, perhaps during residency, and could encompass the scope of their specialty (for more detail, see Chesluk et al.13).

Knowledge Relevant to Practice

Most physicians reported acquiring knowledge relevant to their practice (67 of 80) and/or provided specific examples of how knowledge acquired from studying benefited their practice (63 of 80).

Types of Knowledge

As shown in Table 3, knowledge relevant to practice could include information new to the physician, but also took other forms: information that was familiar, but could now be better comprehended and applied; information that validated or reinforced what they had already known well; and/or information they had once learned but had since neglected (e.g., basic science or topic areas far outside what they see in practice). Acquired knowledge sometimes integrated all these at once, as when interviewees described digging deeper into the basic science behind new studies they had seen previously but not fully investigated, which gave them better understanding of how to apply those advances to their practice. Exam preparation could require study of clinical information spanning their entire specialty area, and also potentially require reevaluation of their underlying scientific model of how to organize and apply their knowledge. Some described learning about the limits of their own knowledge: “[M]y take-home message from [studying a particular subspecialty area] was… how quickly you become uneducated in things that you don’t do… [I]t’s important to know… that maybe you should pull the trigger for [sub]specialty consults sooner on some things” (32 ABIM). (Throughout this paper and in the Online Appendix, we identify interviewees by a unique number between 1 and 80, and also indicate whether they are ABIM or ABFM physicians.)

Table 3 Physicians’ Descriptions of Types of Knowledge Acquired from Studying for an MOC exam
Full size table

Physicians also described passing on information to trainees, fellow physicians, or other providers, as when one described interactions with a colleague exempt from the MOC exam requirement: “[M]y partner [is]… grandfathered in internal medicine… so he doesn’t have to take the exam… I actually find myself enjoying giving him what the board currently thinks about this and that, and he listens very attentively … because I have more refreshed information” (42 ABIM).

How Studying Affects Practice

As stated above, most (63) physicians gave concrete examples of how what they learned benefited their practice. They described many different types of beneficial changes to practice, grouped into several, overlap** categories (see Table 4; see also Online Appendix section 5 for an expanded set of examples, by physician and by theme). Some practice changes were focused on specific conditions, as one interviewee reported:

[Before studying] I had a patient who had had a [Transient Ischemic Attack (TIA)]… and she went on to have a stroke. I had a similar patient recently and, using the ABCD guidelines for TIA, I realized she was at high risk of progressing, so I put her in the hospital and was more aggressive about the management and [I] suspect her outcome was probably better than it might have been otherwise. (9 ABFM)

Table 4 Physicians’ Descriptions of Examples of Practice Change Resulting from Studying for an MOC exam
Full size table

Others described changes that applied across various conditions. These broader changes to care could constitute a change in the scope of the care the physician could provide. One interviewee who cares for an indigent patient population said “I could probably reduce, significantly, the number of visits to the consultant… Having more in-depth understanding of diseases and theoretical knowledge helps me to reduce [subspecialist] visits and reduce the financial burden on [indigent] patients” (57 ABIM).

Physicians also recounted ways in which preparation affected their practice other than direct changes in patient care. Some described improved communication with subspecialists, including via more thorough, focused workups to accompany referrals. Some felt they could now better understand other providers’ treatment decisions, and so could explain those decisions to patients or coordinate or co-manage patients’ care with subspecialists. Physicians also described teaching what they had studied to trainees or sharing with colleagues, potentially catalyzing changes to others’ practices.

Interviewees additionally recounted changes to patient communication, saying studying enhanced their ability to explain their own decisions to patients, or educate them about clinical matters:

[O]ne of the things I learned when I was studying is… I practice a lot of defensive medicine. [Y]ou do it because… the patient wants to, but it’s not really clinically appropriate or it's not according to the guidelines. So now, I learned to… talk to patients more… [B]efore, I used to just… do what the patient wants, but I now talk to them more about the risks, the complications…. (7 ABFM)

Some used the MOC exam itself to win patients’ trust: “It just gives you confidence you’re doing the right thing, and that you could tell the patient about it… I would say, ‘You know, I just read about this on my board exam’… And that was pretty powerful. Otherwise, if I weren’t [studying at] that time, I don’t think I would have that conversation” (47 ABIM).

Our interviews suggest several mechanisms by which MOC exam preparation can affect practice:

  • Direct—new information can lead directly to a change in practice.

  • Reinforcing—information can increase physicians’ confidence in current practice patterns, e.g., by providing validating evidence.

  • Communicating—information can lead to improved communication with providers or patients, e.g., by enhancing understanding of subspecialists’ perspectives or improving credibility with patients.

  • Indirect—regardless of how information might apply to one’s own practice, physicians can spread it, formally or informally, to other providers or trainees.

Examples of Knowledge Seemingly Not Relevant to Practice

Twenty-five interviewees volunteered that they had studied content that seemed irrelevant to their practice. Some gave examples related to conditions they would rarely be called on to treat, particularly obstetrics for family physicians: “[T]hings that maybe weren’t as practical for me would be fetal monitoring strips for OBs, since I don’t do OB and I am not going to do OB…” (18 ABFM). Others described studying very rare conditions: “[T]here was an information bit [in the study material] about an infection from a sand fly only found in the Middle East. I thought, ‘What the hell is this doing there?’” (41 ABIM). Physicians also gave examples related to issues they thought would be exclusively handled by specialists: “[T]here was quite a bit of stuff in the subspecialty section that I just had no idea even existed… [L]ike, they did four different drugs, and I hadn’t heard of any of them. And I never really need to know that, because obviously a person would go to the specialist with that” (32 ABIM).

DISCUSSION

As we have shown, studying for MOC exams requires significant time and effort, as physicians engage in activities very different from what they normally do to stay current. Most interviewees said they acquired knowledge from MOC exam study that was relevant to their practice and could improve their patient care.

Our study suggests several mechanisms by which exam preparation can directly and/or indirectly affect physicians’ practices—both the practice of those studying for the exam, and also, potentially, the practices of fellow physicians, other providers, and trainees (see Fig. 1). These are not mutually exclusive; many of the examples quoted illustrate several in action together. Some of the benefits of exam preparation we identified may stem from the high-stakes, summative nature MOC exam requirement itself, as research on the “testing effect” reports learners in contexts that feature testing retain information better, independent of extra study time.14, 15

These mechanisms may, in part, explain the associations with costs and quality of care described in previous studies. For example, our interviews yielded several possible explanations for Gray et al.’s finding of an association between mammography screening and initiation of the MOC requirement,6 including raised awareness of new guidelines or increased understanding of the underlying research, which could, in turn, result in improved ability to communicate credibly with patients about these procedures. Beyond changes to physicians’ own practices, this knowledge could have been shared with other providers, multiplying the potential positive impact. Physicians in our study described experiencing all of these, in various combinations and related to numerous conditions and patient populations. In sum, these interviews suggest MOC exams may catalyze learning relevant to patient care, even in “the era of widely available instantaneous digital access to information and colleagues.”1

The limitations are as follows: (1) reliance on interviewees’ recall and perceptions of what was beneficial and/or relevant; (2) physicians’ narratives could be affected by social desirability bias, despite our efforts (described above), especially considering researchers’ roles as ABIM and ABFM staff—this also may have colored interviewing and analysis; (3) while we worked to ensure a sample as representative and diverse as possible, hypotheses about how our findings could apply more generally are tentative, both because we interviewed only primary care physicians from two specialties, and because those invited who chose to participate may differ from the invitees who did not—many interviewees expressed that they participated out of a strong desire to personally contribute to the current debates around MOC (see Online Appendix section 2).

Despite these limitations, we hope this paper contributes to debates around MOC, by exploring physicians’ descriptions of how exam preparation can benefit patient care. Further research can explore these connections between MOC exams and patient care to test whether they are generalizable to the broader population of physicians, both primary care and subspecialists; to understand any relationship between learning and factors such as years in practice, opinion of MOC, and exam performance; to assess whether learning leads to measurable improvements in care delivery or health outcomes; and to explore the impact of new types of MOC assessments16 on practice.

CONCLUSION

Most physicians interviewed from two primary care specialties reported ways in which MOC exam preparation resulted in acquiring knowledge that was relevant and beneficial to their patient care.