Introduction

Tolerability in cancer clinical trials has traditionally focused on the degree to which overt adverse effects of treatment can be tolerated by patients and has typically been assessed by clinician report using the Common Terminology Criteria for Adverse Events (CTCAE) and other trial data (e.g., dose modifications, dose discontinuations, hospitalizations) [1]. However, patients often experience side effects that are not detected by clinicians but have a significant impact on their daily activities and quality of life [2, 3]. Patient reports of adverse events (e.g., pain, nausea, fatigue) are generally earlier and more frequent than clinician reports of the same adverse events [3]. Furthermore, patient reports of symptomatic adverse events are more strongly associated with overall health status, while clinician reports predict clinical events like emergency room visits and death. Therefore, there is increasing interest in incorporating patient-reported measures of tolerability in cancer clinical trials to understand the extent to which symptomatic adverse events affect patients’ ability or desire to adhere to cancer therapies [4,5,6]. In line with this shift, a recent, patient-focused definition of tolerability has been offered: “The tolerability of a medical product is the degree to which symptomatic and non-symptomatic adverse events associated with the product’s administration affect the ability or desire of the patient to adhere to the dose or intensity of therapy. A complete understanding of tolerability should include direct measurement from the patient on how they are feeling and functioning while on treatment.” [4].

Measures that reflect this new patient-focused definition of tolerability are needed. The patient-reported outcome (PRO) version of the CTCAE was developed to assess specific symptomatic adverse events directly from the patient [7,8,9]. However, in addition to assessing patient-reported symptomatic adverse events, it is important to assess the global impact or bother associated with treatment side effects to capture the aggregate impact of multiple side effects and help compare across treatments with different side effect profiles [4]. The United States Food and Drug Administration (FDA) recently published its “Core Patient-Reported Outcomes in Cancer Clinical Trials” draft guidance in which it identified a set of key PROs, including those aimed at capturing tolerability [10]. Most directly, these include symptomatic adverse events and an overall side effect impact summary measure [10].

One of the leading options for an overall side effect impact summary measure is the Functional Assessment of Cancer Therapy‐General (FACT) item GP5: “I am bothered by side effects of treatment” [10]. The GP5 item was introduced as part of the FACT – General (FACT-G) measure, wherein it is included in the Physical Well-Being subscale [11]. The GP5 was included in the FACT-G’s item generation and review process whereby its importance to health-related quality of life (HRQoL) in cancer was established through input from advanced cancer patients and expert clinicians [11]. Since then, the GP5 has been included in numerous cancer clinical trials, often as part of the FACT-G or other measures in the FACT system. In addition to its inclusion in multi-item FACT scales, it has also been analyzed independently (as a single item) in cancer trials [12,13,14].

The GP5’s measure properties have been examined, providing support for its use independently. Previous research demonstrates that greater bother from cancer therapy side effects is associated with increased clinician‐reported adverse events, worse health-related quality of life, and a greater likelihood of discontinuing treatment prior to completing protocol therapy [15, 16]. Greater bother from side effects is also associated with lower quality of life and less enjoyment of life [15, 17], which may further reduce tolerability among patients undergoing cancer therapy. Though initial development of this item was informed by patient input, there is opportunity to expand on the original qualitative evaluation by elucidating what specifically patients think of when coming to a response for GP5, how they interpret the term “bother” in particular, and how this item is perceived by patients prior to starting systemic treatment, which is a common aspect of cancer trial design.

The goal of the current study was to assess the patient-centered comprehensibility and relevance of the FACT GP5 item. Using data from three studies, we examined how patients with various cancer diagnoses, including those naïve to cancer therapy, interpret and respond to this item. We were particularly interested in learning more about what experiences and symptom characteristics (e.g., severity, frequency) influence how patients rate their global bother related to cancer therapy side effects.

Methods

Study design and participants

Data were drawn from 3 separate qualitative studies. Two of the studies aimed to establish the content validity of 2 symptom index measures within the FACIT system: the NCCN/FACT Ovarian Cancer Symptom Index – 18 Item Version (NFOSI-18) and the NCCN/FACT Lymphoma Cancer Symptom Index – 18 Item Version (NFLymSI-18). The FACT GP5 item (I am bothered by side effects of treatment) is included in each of these instruments. Therefore, analysis of data from these studies (referred to as Studies 1 and 2 below) is on a secondary basis. The third study was specifically focused on understanding responses to GP5 among patients naïve to systemic cancer therapy. Accordingly, the data from this study (Study 3 below) is analyzed on a primary basis. In each study, patients were recruited from the Robert H. Lurie Comprehensive Cancer Center of Northwestern University.

Study 1: content validation of NFOSI-18 in advanced ovarian cancer [18]

The first study was a qualitative, cognitive interview study that aimed to assess the content validity of the NFOSI-18 [18]. In addition, it explored patient interpretations of bother (and whether it was related to severity), when responding to items using the term bother, such as GP5. After establishing eligibility, a convenience sample of patients were recruited for the study. Eligible patients were female; aged ≥ 18 years; diagnosed with stage III or IV high-grade serous adenocarcinoma of epithelial ovarian, fallopian tube, or primary peritoneal cancer; and assessed as Eastern Cooperative Oncology Group performance status rating (ECOG PSR) 0 to 2. In addition, patients must have been able and willing to sign an informed consent document. Patients not fluent in English were excluded. Sample size was based on evidence that saturation (point at which no new concepts arise) often occurs as early as 12 interviews [19] as well as considerations of measure complexity, population diversity [20, 21], and prior experience conducting cognitive interviews. The study was reviewed by the Northwestern University Institutional Review Board, which determined it was non-human subjects research (STU00203656). Informed consent was obtained for all study participants.

Study 2: patient-reported outcome dossier in support of the NFLymSI-18 indolent B cell non-Hodgkin’s lymphoma (iNHL) patients [22]

The second study was a qualitative, concept elicitation and cognitive interview study that aimed to evaluate the content validity of FLymSI-18 in patients with iNHL [22]. After establishing eligibility, a convenience sample of patients were recruited for the study. Eligible patients had histologically confirmed diagnosis of iNHL and had received one or more prior lines of treatment. Patients aged ≥ 18 years, with ECOG PSR ≤ 2, and life expectancy ≥ 3 months were eligible. Patients were recruited after being identified in the medical record and were then approached in clinic. Each participating patient gave consent. Patients were recruited until saturation was reached. Saturation was evaluated after 12 interviews, and every 3 interviews thereafter, if necessary, until saturation was obtained. Saturation was defined as 3 interviews occurring without new, relevant content introduced. The study was approved by the Northwestern University IRB (STU00102531) and informed consent was obtained for all study participants.

Study 3: GP5 cognitive interview study among treatment-naïve patients

The third study was a qualitative, cognitive interview study that explored understanding of the GP5 item among newly diagnosed patients who had not received systemic chemotherapy or radiation therapy for cancer. After establishing eligibility, a convenience sample of patients were recruited for the study. Eligible patients were aged ≥ 18 years, had a diagnosis of any cancer, and had never received systemic cancer therapy. Patients who had surgery for their cancer were permitted to participate and still considered naïve to systemic cancer treatment. We excluded patients unwilling to provide informed consent and those who were not fluent in English. Each patient gave oral consent to participate. Since the study was low risk and conducted by teleconference, oral consent was deemed by ethics reviewers to be appropriate. We recruited and consented patients until saturation was reached. Saturation was evaluated with the same procedure as described for Study 2. The Northwestern University IRB that reviewed this study protocol made a determination of non-human subjects exemption (STU00210027).

Cognitive interview methods

For Studies 1 and 2, a trained interviewer approached eligible patients, explained the study, and obtained written informed consent. Interviews were conducted in person during the patient’s infusion appointment or in a private room following the patient’s clinic visit. Interviews were audio recorded. For Study 3, clinicians identified potentially eligible patients, and then trained study personnel contacted these patients by phone. If the patient was interested and willing to participate, an oral consent form was completed before conducting the interview.

The overall cognitive interview approach was similar in each study. Semi-structured cognitive interview guides were used to gather patient input. For each study, the cognitive interviewing protocol was based on the work of Willis [23] and ascertained comprehension of the question and response processes. Specifically, patients were asked to (1) restate each item in their own words; (2) describe how they arrived at their answer; (3) define words/phrases within particular items; (4) indicate whether the meaning of the item was clear to them; (5) indicate how confident they were about their answer when responding to the item; and (6) indicate whether the question was relevant to their experience with cancer. Each of the studies took this approach to investigate patients’ comprehension and response process for GP5. In addition, Study 1 featured additional targeted probes to elicit patient interpretations of the term bother, which features in GP5.

Cognitive interview questions

The cognitive interviews in each study featured some common questions, including open and structured questions, as well as questions with a hybrid format. Open ended questions had no pre-specified response options and were used to elicit a descriptive, qualitative response from the patient. Close-ended questions had a pre-specified set of response options. Hybrid questions had a pre-specified set of response options as well as the opportunity for patients to expand on the responses in their own words. Table 1 shows the core set of 9 cognitive interview questions administered in each study and whether the question was administered as an open-ended, close-ended, or hybrid format question. These questions addressed how patients came to their response to GP5, the understandability and relevance of GP5, and the period of time patients thought about when answering GP5. The cognitive interview in Study 1 featured 8 additional targeted questions and probes on the term bother, which are also shown in Table 1. These additional questions explored what causes bother and how bother tracks with side effect severity.

Table 1 Cognitive interview questions
Full size table

Data analysis

The data from Studies 1 and 2 were combined for analysis where possible since studies both included patients on treatment. Data were not combined for questions that were asked in only one of the studies. Since it included only treatment-naïve patients, Study 3 was analyzed separately. Participants’ demographic and clinical characteristics were summarized by study. These characteristics included age, gender, ethnicity, race, education level, ECOG PSR, cancer diagnosis, time since diagnosis, and whether the patient was currently on treatment or had previous cytoreductive surgery. Categorical variables were summarized with percentages and proportions, and continuous variables were summarized using means, standard deviations (SD), and ranges. Participants’ responses to structured questions were analyzed similarly, with summaries of responses shown for the overall sample and by study. De-identified responses to open-ended interview questions were first transcribed to text and then entered into a spreadsheet in Microsoft Excel for analysis. Responses to open-ended questions were analyzed via a constant comparative approach by one study team member with qualitative and measure development experience (J. D. P.) [24], wherein responses to open-ended questions from the first 5 interviews were reviewed to inform development of an initial codebook. The codebook was revised iteratively throughout analysis of subsequent participant responses to capture new codes, revise existing codes, and edit the codebook. After all participants’ data were coded in Excel, we reviewed all coded responses with the final codebook and made revisions to reflect updated codes, as needed. Next, we collapsed similar codes into broader themes. This approach is commonly applied to interpret qualitative data generated for the purpose of develo** patient-reported outcome measures [25, 26]. Finally, the frequencies of responses falling under each theme were totaled.

Results

Sample characteristics

In Study 1, between February and April 2017, 282 patients were screened for eligibility. Of these, 41 were eligible. Of these, 21 consented to participant and completed a cognitive interview. In Study 2, between May and December 2015, 133 patients were screened for eligibility. Of these, 22 were eligible, and 19 consented to participate, with 15 completing a cognitive interview and 14 completing questions relevant to GP5. In Study 3, 26 patients were approached to participate. Sixteen patients consented and completed a cognitive interview. For Study 1, saturation was not used to guide recruitment. Participating patients’ demographic and clinical characteristics are shown in Table 2. Across the studies, patients were generally similar in terms of age and ethnicity, with a somewhat greater proportion of Black/African American patients participating in Study 3. Since Study 1 focused on ovarian cancers, all participants were female. Larger proportions of patients in Studies 2 and 3 attained college or advanced degrees and had an ECOG PSR value of 0.

Table 2 Selected demographic and clinical characteristics of study participants
Full size table

GP5 understandability and relevance to patients on treatment

Responses to structured questions assessing GP5’s understandability and relevance to cancer patients provided by patients on treatment (Studies 1 and 2) are shown in Table 3. The overwhelming majority of patients (overall = 88%) reported that they would not rephrase GP5; the wording was acceptable. Similarly, the overwhelming majority reported that the meaning of GP5 was clear (overall = 97%). All but one patient reported being Very Confident or Confident in their ability to answer GP5. Similarly, all but one patient reported that GP5 was relevant to their experiences with cancer. In addition, all patients who answered GP5 reported that the question was relevant to their cancer treatment and treatment side effects.

Table 3 Understandability and relevance of GP5 in patients on treatment (Studies 1 and 2)
Full size table

How patients on treatment came to their response to GP5

Analysis of responses to the open-ended question referring to GP5, “What kinds of things did you think about when you answered the question? (i.e., How did you come to the answer you gave?)” from patients in both Studies 1 and 2 yielded the following themes: chemotherapy/treatment, nausea, fatigue, general side effects, neuropathy, constipation, pain, and psychological bother (Fig. 1). Selected quotes from the qualitative analysis of additional open-ended questions in Study 1 (patients on treatment) targeted toward patients’ interpretation of the term bother are shown in Table 4. For the question, “What is it about your side effects that bothers you?,” the following themes emerged: specific side effects (e.g., “It’s the nausea that really bothers me. Fatigue doesn’t bother me as much…”), life/routine disruption (e.g., “Anything that is changing my routine.”), and psychological impact (e.g., “It’s psychological concerns and wondering if they are going to go away.”) For the question, “How would it have to worsen to bother you?,” the following themes emerged: more frequent treatment side effects (e.g., “If I had nausea every day. If I had no energy every day.”), more severe treatment side effects (“It would have to be more severe.”), reduced function/ increased dependency (“If I couldn’t get out of bed.”), and specific side effect worsened (e.g., “If I had less energy or worse nausea. […]”). For the question, “How would it have to improve to bother you less?,” the following themes emerged: side effects go away completely (e.g., “If the neuropathy would go away.”) and side effect severity reduces (e.g., “If I had minimal rather major lack of energy. If I had only a little nausea.”).

Fig. 1
figure 1

Themes emerging from responses to open-ended question on how patients come to their GP5 response

Full size image
Table 4 Selected quoted responses to open-ended probes on the term “bother” (Study 1 only)
Full size table

Probes on the term “bother”

Structured questions from Study 1 about interpretation of the term bother demonstrated that, for a large majority of patients, the severity of side effects determines bother (Table 5). Moreover, our qualitative coding revealed that participants reported the expected relationship of bother to side effect severity wherein severe side effects would be more bothersome than moderate side effects, and moderate side effects would be more bothersome than mild side effects. An analysis of responses to the open-ended question, “What does ‘bothered’ mean to you? (i.e., how do you define…?),” among patients from Study 1 yielded the following themes: functional impact, emotional impact, specific side effects, overall quality of life, and physical impact (Fig. 2).

Table 5 Cognitive interview questions on the term “bother” (Study 1 only)
Full size table
Fig. 2
figure 2

Themes emerging from responses to open-ended question on patients’ perspectives on the term ‘bother’

Full size image

GP5 understandability and relevance to treatment-naïve patients

Responses to both structured and open-ended questions from Study 3 about GP5’s understandability and relevance to treatment-naïve patients, along with how they came to their GP5 response, are shown in Table 6. A majority (57%) said GP5 was ok as is (would not rephrase) and 81% were either very confident or confident in their answer. Similar proportions said that GP5 was relevant (n = 7, 47%) and irrelevant (n = 6, 40%) to their cancer treatment, while 13% (n = 2) said they were unsure. Of the 40% who said GP5 was not relevant, half (n = 3) reported being “Not at all” bothered, two patients reported being “A little bit” bothered, and one could not decide between “Not at all” or “A little bit.” When asked what they thought of when they answered GP5, the largest proportion (n = 7, 44%) said they had negative expectations of side effects. Of these patients, four of the 7 reported being “Not at all” bothered on GP5, two reported being “A little bit” bothered, and one could not decide between “Not at all” or “A little bit.” Approximately one third (n = 5, 31%) said they could not answer and/or had no side effects. The remainder said they thought of surgery (n = 3, 19%) or specific side effects (n = 1, 6%). When asked what time period they thought of when answering GP5, patients responded variously with time spans ranging from over 4 weeks (n = 3, 20%) to right now (n = 2, 13%).

Table 6 Understandability, relevance, and what patients thought of when answering GP5 in treatment-naïve patients (Study 3)
Full size table

Discussion

Given increasing interest in the FACT GP5 as a patient-reported measure of tolerability in cancer trials, gaining additional insight into how patients determine how to respond to GP5 is important. We drew upon 3 qualitative studies totaling 51 patients of diverse cancer types. For patients currently on treatment, the results suggest that GP5 was highly comprehensible and relevant, and it captures critical aspects of the side effects of cancer therapies. In addition, the term bother did not compromise patients’ ability to respond to GP5, and this term appears to capture multiple aspects of side effect impact, including symptom severity and frequency as well as the functional impact of side effects. Although patients considered a variety of experiences when determining their responses to GP5, all of these considerations were germane to treatment tolerability. For treatment-naïve patients, there was more variation in GP5’s meaning and usefulness. This study supports the use of GP5 in its current form as a valid, brief measure of tolerability in cancer clinical research, especially for patients on treatment.

There was a close correspondence between patients’ reports of global bother and side effect severity as evidenced by the large proportions of patients who reported that bother would proportionally increase with side effect worsening, and vice versa (bother would proportionally decrease with side effect improvement). Responses also revealed that bother taps into side effect frequency (e.g., daily or persistent side effects) as well, although likely to a lesser extent. In addition, bother also seemed to capture elements of side effect impact, with multiple comments pointing to interference with daily activities or increased dependency. This diversity in interpretation of the term bother underscores GP5’s global nature and its flexibility to capture varying elements of side effect experience that different patients may find most important.

Responses about the comprehensibility of GP5 for patients on treatment were similar to those from a previous report by Jensen and colleagues [27] in a previous content validity study of the NFOSI-18 among ovarian cancer patients where 94% (17/18) of patients stated that they had no questions about how to answer GP5, 94% (16/17) stated that GP5 was understandable, and 88% (15/17) were very confident or confident in answering GP5. In the current study, over 90% of patients on treatment said the meaning of GP5 was clear and that they were very confident or confident in answering GP5 and 88% said they would not rephrase GP5. Taken together with these previous results, it appears that patients find GP5 understandable in its current form. In addition, similar to our study, Jensen and colleagues [27] found that patients commonly thought of fatigue, nausea, neuropathy, and vomiting when responding to GP5. Gastrointestinal side effects like nausea, vomiting, and constipation [28], as well as neuropathy [29], are among the most common and impactful chemotherapy side effects. Reports that patients commonly think of these side effects when responding to GP5 support the item’s face validity.

One concern with GP5 has been about its usefulness and interpretability among treatment-naïve patients or at baseline in trials. Although patients are slightly more likely to skip GP5 at baseline compared to other FACT-G items (5–15% skip GP5 vs an average 2–3% on other items), the vast majority of patients do respond to the item at baseline when administered in a trial, and nearly 10% report high side effect bother before any treatment starts [30]. However, what determines patients’ responses to GP5 before starting treatment has previously been unclear. Our study found that treatment-naïve patients are influenced by negative expectations or fears about potential side effects. Negative expectations about chemotherapy side effects have been reported elsewhere [31, 32]. In addition, it is notable that some treatment-naïve patients in Study 3 reported it would be challenging to answer GP5 or reported that it was not relevant to them prior to starting treatment. On the other hand, we note that all but one patient could give a response to the item when asked.

The potential for varying meanings for GP5 at baseline or lack of perceived relevance of this item at baseline may have implications for how to analyze GP5 in clinical trials. Depending upon the factors that contribute to the item response, change in GP5 scores from baseline may be difficult to interpret. In treatment-naïve patients at baseline, GP5 item response has, in one study, provided information about the future (post-baseline) GP5 values and treatment discontinuation [16]. In addition, a prospective cohort study of hormone-receptor-positive breast cancer patients who had recent surgery found a significantly higher relative risk (1.83; 95% CI: 1.03–3.26) for experiencing side effects 2 years post-baseline among patients who had high negative side effect expectations at baseline in comparison to those who had low negative side effect expectations at baseline [33]. The possibility that bother with other recent treatments (in this case surgery), or bother with side effects from other prior treatments, or negative expectations regarding pending treatment, could forecast subsequent tolerability is an area that could benefit from further research.

This study had both advantages and limitations. Advantages include the opportunity to elicit rich input through qualitative, individual interviews from patients with several different types of cancer. As PROs play a larger role in drug development and evaluation, emphasis has been placed by the US FDA on qualitative research to help determine whether these PROs capture the intended concepts and in ways that reflect patients’ experiences [34]. The data from this study will help support the use of GP5 in future cancer trials. On the other hand, like all qualitative studies, our results may not be generalizable to broader cancer populations. Additional research is needed to confirm and expand upon our findings. In particular, further qualitative work should examine what drives patients to select specific responses to GP5 over others in order to further elucidate the cognitive process involved in answering this item. An additional, methodological limitation was the use of only one coder to generate themes in our qualitative analysis.

In conclusion, we report patients’ perspectives on GP5’s understandability, including qualitative responses to open-ended questions. Our results indicate that GP5 is well-understood by patients, relevant to their experience with side effects, and able to flexibly capture several important elements of tolerability. This study suggests that GP5 is a useful, succinct measure that captures how cancer patients tolerate their treatment.