FormalPara Key Points

Comments provided by key opinion leaders indicated that BRA should inform decision making with a clear plan and structure in place throughout the product lifecycle with better planning to address gaps, minimize uncertainty, and facilitate regulatory interaction at an early stage.

Key opinion leaders emphasized the need to incorporate patient’s input and their perspectives on benefit–risk trade-offs into various aspects of BRA, via the selection and development of appropriate methodologies and to enhance the communication/visualization of BR information in a way that can be understood by all stakeholders.

The interview findings provide insights into the current practice of BRA and how to further advance the field through implementation and optimization of existing processes, development of new methods, and ways to address gaps and limitations.

1 Introduction

Benefit–risk assessment (BRA) is a critical component of the decision-making process during medical product development and in obtaining market approval. In the past, there was a lack of clarity and transparency of the key considerations informing the assessment of whether the benefits of a product outweigh its risks [1]. Research has been conducted by various organizations over the last decade defining structured benefit–risk assessment (sBRA) as a transparent, standardized, and organized process to guide discussions and provide better documented benefit–risk (BR) decisions on medical products throughout the product development lifecycle. Several processes, frameworks, visual displays and tools for sBRA have been developed, such as the Benefit–risk Action Team (BRAT) framework [2], Problems, Objectives, Alternatives, Consequences, Trade-offs, Uncertainty, Risk attitudes, and Linked decisions (PrOACT-URL) [3], the European Medicines Agency (EMA) effects table [4] and the FDA benefit–risk framework [5, 6]. The sBRA frameworks provide a means to assess and communicate the evidence, uncertainties, and trade-offs in a standardized and transparent manner [7]. Furthermore, research initiatives have examined various quantitative methods such as Multi-Criteria Decision Analysis (MCDA) and weighted net clinical benefit, using algorithms incorporating utilities or weights based on physician and especially patient preferences [8,9,10,11,12]. Pharmaceutical and medical device companies are employing such quantitative methods for complex BR decisions but several challenges in adopting a quantitative approach have been identified [13]. With respect to patient preferences, greater patient involvement in benefit–risk assessment is a focus area for both regulators and drug and medical device companies, and methods to seek patient input have been developed [14,15,16,17,18,19]. Given these advances, it is important for BRA to follow a structured approach underpinned by a clear process with appropriate planning and documentation in place.

Practical advice to companies on implementation of a sBRA framework and process for specific activities such as marketing authorization applications as well as more broad application across the product lifecycle has been published previously [20,21,22,23]. Implementation may have lagged or be inconsistent and there is scope for further research into the usage of sBRA across companies, and provision of additional advice on good practice for BR assessment and planning.

The American Statistical Association (ASA)—Safety Evaluation Working Group, Benefit–Risk Assessment Planning (BRAP) Taskforce was formed in 2019 and comprises safety scientists, physicians, epidemiologists, and statisticians from various pharmaceutical and medical device companies, consultancies, and regulatory agencies. The Taskforce has undertaken a series of research projects to help shape the development of a BRAP planning document, which may be considered for use during the medical product development lifecycle and to facilitate interactions with regulatory agencies. Two of the projects, a review of the global/regional regulatory BRA landscape [24] and a survey assessing the usage and application of sBRA within the pharmaceutical and medical device industry, have been completed. The BRA landscape review highlighted three key regulatory trends of increased transparency, more consistent incorporation of quantitative elements, and broader inclusion of patient experience within the BR framework. Across all stakeholders, an increasing emphasis on a more rigorous, holistic, and scientific approach to BRA was observed in the literature review. More effective utilization of the patient perspective in the BR decision-making process was identified as an area for further development. The industry survey confirmed a wide usage of sBRA by many companies involved in research and development. Future opportunities like the implementation of visualization tools were identified along with some challenges such as the cross-functional comprehension of the added value of sBRA. This manuscript summarizes further research conducted by the Taskforce to engage with the wider scientific community and seek expert opinion from Key Opinion Leaders (KOLs) within BR on the current status of the field and how this may develop in the future, with regard to regulatory requirements, methodological research, the value of planning, and uptake by industry globally.

A range of perspectives on BRA were gathered by interviewing pharmaceutical and medical device professionals experienced in BRA representing industry, regulatory agencies, academia, and patient group from three regions (US, EU and Asia [Japan]). The KOL interviews were conducted during the period from December 2020 to April 2021.

The results from the interviews provide insights into the current status of BRA practice and suggest ways the discipline may evolve to further implement and optimize existing approaches, develop new methods, and address gaps and limitations.

2 Methods

The KOL interviews built on the results obtained from a survey which was previously conducted by the BRAP Task Force to explore the status and current implementation of sBRA in the global drug and medical device industry. Most of the questions formulated for the KOL interviews emerged from this early study and were intended to identify any potential gaps regarding BRAP and the implementation of processes. The industry survey results highlighted challenges in implementing sBRA activities and identified areas where companies should focus their effort, e.g., demonstrating the value of sBRA; further understanding where it could be applied and implemented into company processes and decision making; enhancing regulatory agency and industry discussions regarding BR; develo** visualization tools to facilitate the assessment and communication of BR; understanding the use of quantitative sBRA approaches to assess benefit–risk; aligning the planning and conduct of safety and benefit-risk assessment; and, integrating patient preference information into the  benefit-risk assessment.

Based on the industry survey findings, 5 questions were formulated for the KOL interviews to further understand areas for improvements. The 5 questions were in regard to the future uses of BRAP in the process of medical product development. The questions covered the following five key areas:

  • Current and future application of BRA

  • Value of planning in the execution of BRA

  • Integration and alignment of safety and BRA

  • Planning, assessing, and communicating BRA activities throughout the medical product lifecycle

  • Patient engagement in BRA planning

A full description of the questions can be found in the electronic supplementary material (Section 1). Further details on the conduct of the interviews and analysis are provided in the following sections.

2.1 Subject Characteristics and Study Design

Participants were selected by the BRAP Taskforce members based on their expertise as pharmaceutical and medical device professionals representing industry, academia, and regulatory agencies in the planning and conduct of BRA. The sole inclusion criterion was expertise in BRA as determined by evaluation of the individual’s level of relevant experience and scientific contributions to the field (e.g., quality and number of relevant publications on the topic, conference presentations and participation in BRA working groups and advisory meetings). Every effort was undertaken to obtain as good a representation as possible from a range of different types of organizations and regions.

A list of potential participants and contact information was collected and stored on a password-protected IT system used by the lead interviewers. Participants were contacted by e-mail or phone and asked whether they would be interested in taking part in the one-to-one interviews. Participation was voluntary and no renumeration was offered. Interviews were conducted by phone or web audio in English. The invitation to participate, provided in the electronic supplementary material (Section 1), served as the process for informed consent.

Participants who agreed to be interviewed were asked 5 questions about BR planning and processes. The questions contained multiple probes and were piloted during the interview process to ensure clarity. As part of obtaining informed consent, the questions were sent to the interviewees at least 4 weeks in advance of the interview. Each interview involved a total of three interviewers, one interviewer led the questions, and two interviewers listened and took notes to ensure all responses were captured. Interviewers were selected from the members of the BRAP Task force. The duration of each interview was about an hour. Answers were recorded in a written format and a summary was produced based on the notes taken during the interview. There was no recording or retention of audio. No identifiable information of either the interviewee or organization was collected, hence responses were anonymized for further analysis, and opinions could not be attributed to any one individual or organization. No proprietary information related to an organization’s products or assets was requested during the interview. By design, the number of participants was determined by the number who were contacted, recruited, and who agreed to have an interview completed within a 6-month period.

2.2 Creation of a Coding System and a Marking Sheet Template

Once the KOL interviews were completed, a Coding System to describe responses captured in the interview summaries was developed through labelling and identifying themes and associated sub-themes. An exercise was undertaken to identify potential themes and sub-themes and the interview summaries were reviewed to highlight relevant content map** to these pre-specified themes. A marking sheet template (Master Template) based on an Excel spreadsheet was generated to ensure that information was collected in a consistent and consolidated manner and to facilitate the analysis. The Master Template included dedicated cells for the pre-specified themes and sub-themes so that relevant content from the interview summaries could be recorded, as well as information on region (US, EU, or Asia), sector (regulatory, industry, academia, or patient group), and specialty (clinician, statistician or other) for each interviewee. The Master Template also allowed evaluators to add new themes/sub-themes, in addition to those already pre-specified. To check the adequacy of the Coding System and Master Template, a pilot assessment was conducted on a small set of randomly selected interview transcripts. The check did not suggest the need of major revisions or the addition of new themes or sub-themes. An overview of the Coding System used for the evaluation and analysis of the KOL interviews can be found in Table 1.

Table 1 Coding System for themes and sub-themes
Full size table

2.3 Evaluation and Statistical Analysis

A qualitative assessment of the responses from the interviews was performed by 10 selected members of the BRAP Task force with two evaluators assigned to assess each interview and arrive at a consensus view. The evaluators performed a thorough assessment of each interview summary, identified relevant information corresponding to the themes and sub-themes in the Coding System and summarized the findings into the Master Template. An independent quality assessment of the information recorded by the evaluators was conducted to ensure the content from each interview summary was objectively reflected, and confirmation bias was minimized.

A qualitative content analysis was performed on the aggregated dataset to extract key findings and common opinions. A quantitative analysis was also performed by assessing the number of times a given theme/sub-theme was recorded across all the interviewees. Percentages were derived representing the contribution of a specific theme/sub-theme out of all of the interviewees considered. Descriptive statistics of the aggregated results from the KOL interviews are presented in the following section.

3 Results

The KOL interviews consisted of a total of 27 interviewees from which, 11 (40.7%) individuals were from regulatory bodies, 11 (40.7%) were from industry, 4 (14.8%) from academia and 1 (3.7%) from a patient advocacy group. The interviewees largely comprised clinicians, scientists and statisticians, see Table S1 (electronic supplementary material) for more details.

Each interviewee was administered 5 questions (Q1–Q5). A full description of the 5 questions can be found in the electronic supplementary material (Section 1). Key findings from the qualitative assessment of the responses for the 5 questions, including the themes and sub-themes identified for each specific question, are summarized below and presented in Table 2.

Table 2 Summary of key findings for KOL interview questions Q1–Q5
Full size table

For each question, the most prevalent themes based on the quantitative analysis are presented in Figs. 1, 2, 3, 4 and 5 and discussed below along with key findings from the qualitative content analysis.

In question 1 (Q1), interviewees were asked about the future application of BRA. They were asked about what should be achieved in BRA, and how that should be achieved over the next 5 years.

The bar graph (Fig. 1) uses a color-coded legend to illustrate different themes and corresponding color bars to represent sub-themes under each of these themes. For example, the blue color represents the theme Current status of BRA and the blue bars represent the corresponding sub-themes, Minimal to no usage currently, Emerging only, Some measures in place but more required and Already well embedded with dedicated groups, etc. A similar legend was used to identify themes/sub-themes for the remaining questions Q2–Q5.

Fig. 1
figure 1

Bar graph summarizing the percentage of interviewees who contributed to a given sub-theme for Question 1. We know there is growing interest in not only what should be achieved in benefit–risk assessment (see M4E R2 and E2C R2), but also in how we achieve that. How do you think this will proceed over the next 5 years for your organization and for development of medical products within the wider medical product sector? BR benefit–risk, BRA benefit–risk assessment

Full size image

Regarding the theme of Current status BRA, the most frequent sub-theme Some measures in place but more required was highlighted in the comments provided by 70% of the interviewees, while Minimal to no usage currently was highlighted in the comments provided by only 4% of the interviewees. For the theme Step to the future—value demonstration, the most frequent sub-theme was Regulatory authority expectations, potential for further guidance based on comments provided by 78% of the interviewees suggesting the need for regulators to set expectations and provide further guidance. For the theme Step to the future—development of process (e.g., working practices and procedures), the sub-theme Process guidance by expert groups was the most frequent and was highlighted in the comments provided by 63% of the interviewees, while comments by 48% of the interviewees contributed to the sub-theme Incorporation of BRA into other (e.g., existing) processes. Here interviewees highlighted the value of dedicated BRA expert groups in guiding best practice within industry, and the need to have a clear, structured, and well-integrated BRA process, with BRA hel** to inform decision-making throughout the product lifecycle. For the theme Step to the future—development of expertise, the sub-theme Establishment of BRA methodologies and refinements was highlighted in the comments provided by 67% of the interviewees, which was the only identified sub-theme within this theme. Interviewees suggested a need to improve methodologies and tools over the coming years with qualitative assessment of BR as the foundation supplemented by quantitative methods as appropriate (Fig. 1).

In Q2, interviewees were asked about the value of planning in the execution of BRA and to indicate how and when planning should occur in the medical product life cycle based on their own experience.

Regarding the theme Value of planning, the sub-theme Identification of best approach/fit for purpose was the most frequent and was highlighted in the comments provided by 56% of the interviewees. Interviewees indicated the importance of identifying the best approach for BRA along with early planning to ensure appropriate trial design to facilitate the BRA (for the theme Level of planning activities, comments provided by 63% of the interviewees contributed to the sub-theme Trial design level). The need for interactions between pharmaceutical and medical device companies and regulators starting during early development and continuing throughout the product lifecycle was also highlighted (for the theme When to start planning comments provided by 44% of the interviewees contributed to the sub-theme Phase 2, and the Importance of BRA throughout the product lifecycle was highlighted in the comments provided by 33% of the interviewees). For the theme Plan review and update, the sub-themes Milestone driven and Ad hoc based on significant new data becoming available were the most frequent sub-themes and were both highlighted in comments provided by 44% of the interviewees, suggesting the BR planning needs to be milestone-driven and updated “ad hoc” as new data becomes available (Fig. 2).

Fig. 2
figure 2

Bar graph summarizing the percentage of interviewees who contributed to a given sub-theme for Question 2. Planning and execution are two key steps of any clinical development endeavor. How do you see the value of planning in the benefit–risk assessment activities and what have you learnt from your own experience? Please articulate whether, for example, you see this planning as solely a compound/indication level activity or also a trial design activity. When during the lifecycle should planning occur? How should plans be maintained and updated over time? BRA benefit–risk assessment

Full size image

In Q3, interviewees were asked about the integration and alignment of safety and BRA. They were also asked about the criteria used to determine when plans should be made for additional assessment of benefit or risk, including the information sources that should be considered and the role of observational data.

Regarding the theme of Integration of safety and BRA, the most frequent sub-theme was Alignment between processes (e.g., working practices and procedures), which was highlighted in the comments provided by 59% of the interviewees. Interviewees indicated that safety should not be viewed in isolation and there is a need for more interaction between the benefit and risk functions within companies. They considered that BR should be viewed as the overarching concept with relevant processes aligned under this umbrella. Where feasible, the processes for BRA and Health Technology Assessment (HTA) should be aligned. For the Criteria for additional assessment theme, Upon identification of significant safety concerns and Based on availability of new data and its relative value were the most frequently reported sub-themes and were highlighted in the comments provided by 41% and 37% of the interviewees, respectively. Post-marketing, observational and clinical data sources were the most frequently identified sub-themes under the theme Sources of data for BRA, with Observational data considered Of some value based on the comments provided by 48% of the interviewees and Very valuable by 30% of the interviewees. Observational studies were thought to be of particular value in the post-marketing setting. Interviewees mentioned the frequent use of observational studies to address safety questions and also identified an opportunity to utilize observational data to evaluate benefit/effectiveness post-approval. Potential drawbacks of observational data such as bias, database limitations and a need to develop the right analytical tools were stated (Fig. 3).

Fig. 3
figure 3

Bar graph summarizing the percentage of interviewees who contributed to a given sub-theme for Question 3. Risk and Risk Management are key components of benefit–risk assessment. How do you see safety assessment and benefit–risk assessment being better integrated or aligned? All products require ongoing monitoring for safety, even if just through passive event reporting. What are the criteria used to determine when plans should be made for additional assessment of benefit, or risk (beyond routine pharmacovigilance), to demonstrate that the product has, or continues to have, a favorable benefit–risk? What are the sources of information that should be considered? What role does observational data have? RWD real-world data

Full size image

In Q4, interviewees were asked about planning of activities for assessing, expressing and communicating BRA activities throughout the medical product lifecycle. Interviewees were also asked what kind of activities these could be. Finally, interviewees were asked to comment on the important priorities concerning BRA in the future.

With regard to the Planning and assessing of BR activities theme, the sub-theme of Clarity on the task and objective of the BRA was the most frequently reported sub-theme and was highlighted in the comments by 44% of the interviewees. Interviewees communicated the importance of investing time upfront to fully understand and define the aims and objectives of the BRA before proceeding further.

For the Priorities regarding BR (in the future) theme, Understanding BR value at different stages of the lifecycle was the most frequently reported sub-theme and was highlighted in the comments provided by 44% of the interviewees. With respect to this sub-theme, interviewees discussed the importance of starting BRA early in development to ensure appropriate selection of the target population/indication, characterization of risks, trial design, and to address gaps and minimize uncertainty. It is too late to act if gaps are only identified at the time of submission. Interviewees stated that the type of analysis performed may differ depending on the lifecycle stage, amount of data available, therapeutic area (e.g., BR thinking may be different for severe conditions with a high unmet medical need, such as oncology indications, compared to other therapeutic areas) and whether the BR profile is straightforward or more challenging. A relatively rudimentary assessment may be performed at the early stages of the product lifecycle, which then evolves over time as more information becomes available. Having a dedicated safety management/BR team to support the BRA through the product lifecycle was considered helpful.

With respect to the theme of Expressing and communicating BR, Understanding the audience (e.g., clinicians, patients, or regulators) and Appropriate contextualization of benefits and risks were the most frequently reported sub-themes and were highlighted in the comments of 44% and 41% of the interviewees, respectively. Interviewees indicated a need for better communication of BR information in a way that can be easily understood by all stakeholders including patients and considered that consultation with communication experts and more education/training on effective methods of communication may be helpful in this regard. Experience gained with assessing and communicating the benefit-risk of COVID-19 vaccines could also be leveraged. Interviewees recommended using the latest technologies and platforms such as smart phones and social media as a way to disseminate information. It was also considered important to share information on BRA methodologies and decision-making tools through publications. In terms of appropriate contextualization of benefits and risks, interviewees mentioned the use of tools such as value trees [25], the appropriate structuring of document templates, and adoption of an integrated approach across different company functions in order to develop a unified perspective on BR rather than the benefits and risks being evaluated and presented separately. The potential to compare BR profiles across products was also mentioned along with the possibility of develo** measures/metrics that are more readily understood by clinicians and patients (Fig. 4).

Fig. 4
figure 4

Bar graph summarizing the percentage of interviewees who contributed to a given sub-theme for Question 4. Using the provided Figure (see Fig. 1 in Appendix) as a reference model for reflection, how will you be planning activities to assess, express, or communicate benefit/risk at different times of the product lifecycle such as when: develo** a target product profile?, seeking a clinical trial application?, discussing pricing and reimbursement with payers?, preparing communication materials for health care professionals and patients?, submitting a dossier for a new indication for an existing approved product? What kind of activities could these be? How do you see these activities changing? What do you think will be important priorities concerning benefit/risk assessment in the future? BR benefit–risk, BRA benefit–risk assessment, HTA health technology assessment, RWD real-world data

Full size image

In Q5, interviewees were asked about patient engagement in BRA planning. Regarding the theme of Value of patient input, the most frequent sub-theme was Utilize patient experience/preference to guide new product development and BRA, which was highlighted in the comments by almost all (96%) of the interviewees. This was followed by the sub-theme of Understand patients’ tolerance for risk and trade-offs, which was highlighted in the comments provided by 74% of the interviewees. Interviewees indicated that patients should have a voice and be involved early in the product development process. They can provide valuable input into various aspects of development such as drug candidate selection, clinical trial design, benefit-risk assessment, and product labelling. Factors important to patients such as convenience of the dosing regimen, should be identified. Interviewees indicated that the focus on patient engagement will continue to grow in coming years and develo** a common approach across industry would be beneficial. Companies may work together to understand what is important to patients within a given therapeutic area.

Regarding the theme of Data collection/input, How to gather information (from patients) was highlighted in the comments provided by 78% of the interviewees and was the most frequent sub-theme. The sub-themes Selection and development of appropriate methodologies (to solicit patient input) and When to start involving patients were highlighted in the comments provided by 59% and 41% of the interviewees, respectively.

Interviewees indicated that patient input can be sought via patient organizations and advocacy groups and patient representatives can be included on advisory and steering committees. The importance of learning about individual patient perspectives was highlighted. Potential methods to solicit patient input such as qualitative interviews, surveys, diaries, patient journey map** activities, direct ethnographic observation, risk perception studies, BR preference studies/elicitation methods, Patient Reported Outcomes (PROs), Patient Preference Information (PPI), and Clinical Outcome Assessment (COA) were described. Interviewees highlighted the need to carefully consider the methodology and analysis to ensure the data collected from patients add value rather than creating noise. The time required to develop innovative endpoints/methods and seek regulatory agency buy-in should also be considered. With respect to communicating BR information, interviewees considered that visual presentations and interactive tools that are tangible to patients should be used (Fig. 5).

Fig. 5
figure 5

Bar graph summarizing the percentage of interviewees who contributed to a given sub-theme for Question 5. How can patients be involved in benefit-risk assessment planning? How should the collection of patient experience and patient preference data be incorporated within benefit–risk planning? BRA benefit–risk assessment, PPI patient preference information, PROs patient reported outcomes

Full size image

4 Discussion

Several important findings emerged from the interviews. The interviewees indicated that there are some measures in place to conduct BRA in regulatory agencies and in most pharmaceutical and medical device companies, but more still needs to be done. The interviewees emphasized the importance of starting BRA in the early phases of medical product development and using BRA to inform decision making throughout the product lifecycle. Scenarios where BRA may support decision making include dose selection and targeting of a medication to the most appropriate patient population. The BRA may be relatively simple in the initial stages of development and evolve in complexity as more knowledge and data accrue. This concept of starting BRA early in development with use of a relatively simple approach followed by more in-depth analyses as the product proceeds to the later stages of development is consistent with a published BRA operating model [21]. In this model there is a particular focus on BRA at key development milestones with use of a modified version of the FDA BR framework at the early first-in-human milestone, and introduction of value trees and effects tables at later milestones.

More rigorous and proactive planning of BRA was identified by the interviewees as an area for development. The interviewees stressed that the components of a sBRA should be considered during planning to ensure that the appropriate data (relating to key benefits and risks) are collected during the development program, and gaps and uncertainties are identified early and addressed to the greatest extent possible. Interviewees communicated a need for companies to establish working practices and team structures to support BRA and better integrate BRA into the entire medical product development process. Enhanced cross-functional collaboration between the safety, clinical, epidemiology and statistical functions would be beneficial in this regard and enable a more unified approach. The importance of companies seeking early regulatory authority input on BRA was also highlighted and this was considered of particular importance for the design of pivotal trials. In drug development, the end of Phase 2 was proposed as a good timepoint to share the BRA plans with regulators. It is critical to include appropriate endpoints in pivotal trials in order to facilitate the eventual BRA conducted to support regulatory submissions. Interviewees viewed qualitative assessment as the foundation of BRA, and this can be supplemented by quantitative approaches as applicable, e.g., when the BR is more challenging and there is a need for more complex analysis. Sullivan et al. [21] defined three categories of BR analysis consisting of descriptive, semi-quantitative, and fully quantitative analyses with fully quantitative assessments reserved for more complex and challenging scenarios. The Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (PROTECT) Benefit–Risk group also published a flowchart showing three types of BR assessment (qualitative, semi-quantitative, and quantitative) and when they may be applied [26].

The interviewees felt that more guidance from regulators and international expert groups on recommended practice for BRA would be valuable, particularly with respect to establishing scientifically sound quantitative BRA methodologies and their appropriate applications. Multi-Criteria Decision Analysis is emerging as a key methodology for quantitative BRA (qBRA) [27,28,29], but the need for further guidance on when and how to use qBRA methodologies was highlighted previously by Kürzinger et al [30], and a study of the state of qBRA practice within industry [13] noted a lack of consensus on which qBRA methods to use in which circumstances and the absence of regulatory guidance in this area. Once finalized, the CIOMS Working group XII report on BR balance for medicinal products will serve as an important reference source for guidance on good BRA practice [31]. A taskforce from ISPOR, the leading professional society for health economics and outcomes research, has also published recommendations [32] and a case study [33] regarding good practice for qBRA.

Communication of BR information in a clear and logical manner was identified as an area for further development. This is relevant for all stakeholder interactions but is particularly important when communicating outside of industry and regulatory agencies. Interviewees considered that information should be communicated in a way that can be readily understood by both healthcare professionals and patients, and tools to facilitate the communication, such as visual representations of BR, should be explored. The field can learn from the way BR information for COVID-19 vaccines, including presentation of BR by age group, was communicated to the public. With respect to this topic, the PROTECT Benefit–risk group previously defined four criteria to determine audience visual compatibility and proposed several types of visualizations that could be considered for the communication of BR information [34]. Colopy et al [35] have also presented some examples of visualizations designed to facilitate discussion and planning of BR.

The importance of seeking patient input was emphasized in the comments provided by almost all the interviewees and was considered as a key area of focus for BRA and medical product development in the future. Patients can make a significant contribution to BRA by providing their perspective on meaningful benefits and acceptable risks. It was also recognized that patients can provide input to the overall medical product development process, e.g., by evaluating the convenience of a trial assessment schedule to reduce patient burden in clinical trials. Interviewees described several options for soliciting input from patients and expressed a need for further methodological development and refinement in this area. The interviewees’ perspective on the value of including patients in decision-making is consistent with various initiatives in the medical product development field. The FDA stresses the importance of patient-focused drug development [16, 36] with incorporation of patient input into decision-making, and a methodological guidance series [15, 16, 37, 38] related to the collection and use of robust and meaningful patient input has been released. The Patient Preferences in Benefit–Risk Assessments during the Drug Life Cycle (PREFER) consortium has also released recommendations [39] on why, when and how to assess and use patient preferences in medical product decision-making. The recommendations were developed following consultation with patients, patient organizations, industry, regulators, health technology assessment bodies, payers, and clinicians. A section of methodological considerations to gain patients’ insights are also addressed in the CIOMS Working group XII draft report [31].

Overall, the opinions expressed by the KOL interviewees are broadly aligned with the results of the industry survey conducted by the BRAP Task force on the current status of structured BRA in the global pharmaceutical and medical device industry and the areas in need of improvement. In accordance with the KOL interviews, the survey found that many pharmaceutical and medical device companies are currently conducting sBRA. The industry survey found several areas for further development, such as the design of cross-functional processes that interface with sBRA, and better understanding of the application of quantitative approaches to sBRA, which were also captured as key findings in this study. This concordance is reassuring but not entirely unexpected as the KOLs interviewed included experts from industry.

One of the key limitations of this research is the use of a deductive approach to develop the Coding System (pre-specified themes and sub-themes), which may be prone to bias [40]. Efforts were made to reduce such limitation to the extent possible by not restricting evaluators to the pre-specified coding but allowing them to propose new themes/sub-themes that emerged from their review of the interview summaries. However, no new themes or sub-themes were identified by the evaluators. Ten evaluators were involved in the review process, which may have led to some inconsistencies in the review and interpretation of data. To reduce inconsistency, two lead evaluators oversaw the coding process including training of the other evaluators and review of the datasheet entries for quality control. Whilst the interviewees were selected from experts in the BR field from industry and regulators, the sample size was relatively small, and some stakeholders such as patient groups are underrepresented. Understandably, the thoughts and opinions summarized in this paper may not be fully representative of the broader BR community. In addition, not all interviewees had direct experience in develo** and implementing a BRA process. Whilst this was not a fundamental requirement, further experience in the practicalities of translating thoughts and ideas into a workable process and awareness of some potential difficulties and pitfalls when implementing and introducing new activities would have been beneficial. We also recognize that the example activities included in Question 4 were most relevant to industry. Interviewees from outside of industry were all experts in BR and they considered the question in its entirety. They did provide valuable responses, but we acknowledge that the question could have been better framed in order to optimize its applicability across sectors.

The BRAP Task force has conducted several activities to assess the current status of BR across the medical product lifecycle and identify the areas for further development and prioritization in the future. These activities include a review of the global regulatory landscape relating to BR, a survey to understand the current state of BRA practice within industry and the KOL interviews presented in this paper. The Task force will synthesize and build upon the insight gained from these activities and other published literature in the field, to inform the development of a BR planning document. This document will include information on the key points to consider when planning a benefit-risk assessment and facilitate benefit-risk planning, implementation, and communication throughout the product lifecycle in a standardized and harmonized manner.

5 Conclusion

In this study, KOLs in BRA were interviewed to gain their perspective on the current status and future application of BRA planning throughout the medical product lifecycle. A number of key findings emerged from analysis of the interview transcripts. Key opinion leaders considered that BRA should commence early in product development and inform decision-making throughout the product lifecycle. The importance of including the patient voice in BRA and medical product development was emphasized. Analysis also identified improved communication of BR information, establishment of methodologies for performing BRA and soliciting patient input, and better planning and integration of BRA into existing working practices/procedures as areas for further development. The information collected from this study will inform the development of a BR planning document.