Introduction

All modern intensive care units (ICUs) are technology-rich environments, with sophisticated monitoring equipment, life support technology, and a growing assortment of diagnostic and therapeutic tools. Because of the obvious benefits of ICUs and the ability to more efficiently treat less critically ill patients (thus either preventing hospitalization or allowing earlier discharge), ICUs (including coronary care units) account for a substantial and growing proportion of in-patient days and hospital costs [1, 2]. Health technology assessment (HTA) refers to the process of evaluating the effectiveness, costs, and broader health system and societal impacts of a health technology [3]. HTAs extend evaluation beyond efficacy (or diagnostic accuracy in the case of diagnostic tests) to include effectiveness, cost-effectiveness, affordability, the technology's implications for health care delivery (e. g. the personnel, training, and services needed to deliver primary angioplasty for acute myocardial infarction), and ethical issues. HTAs can provide guidance to clinicians, administrators, and policy makers about the benefits and costs of ICU technologies. Despite numerous editorials, commentaries, and consensus conferences during the past decades advocating more frequent HTAs of critical care technologies [4–7] there have been remarkably few conducted, and their influence may be limited [8, 9]. In this article we discuss the factors that may be responsible for the slow-adoption of HTAs in critical care and suggest strategies for change.

Characteristics of technologies in critical care

In the HTA literature “health technology” is a non-specific term that has been used to describe a wide range of interventions, diagnostic tests, and policies including preventative measures, vaccines, pharmaceuticals, monitoring and therapeutic devices, and the systems within which health is protected and maintained (“Global networking for effective healthcare”, International Network of Agencies for Health Technology Assessment, Stockholm, Sweden, www.inahta.org, accessed 17 September 2007). This contribution focuses exclusively on non-pharmaceutical devices in the ICU since they are important technologies in critical care, and the process of evaluating pharmaceuticals is relatively well established.

The fundamental role of medical technology in ICUs dates to 1952 when patients dying of respiratory paralysis were first placed on mechanical ventilators during the Copenhagen polio epidemic [10]. This epidemic also marked the first widespread use of another technology, tracheostomy, as a conduit for providing positive pressure ventilation. During the next decade the use of these technologies for other clinical conditions led to the organization of respiratory ICUs in many hospitals in Europe, the United States, and Canada (“History of critical care”, Society of Critical Care Medicine, Des Plaines, Ill., USA, www.sccm.org, accessed 17 September 2007). There is no disputing the huge impact these therapeutic devices have had on patient outcomes and the health system.

The success of ICUs has had important resource implications. ICU patients account for approx. 10–20% of in-patient acute care beds in the United States [1, 11, 12], and the associated costs have been estimated at 0.5–1.0% of that country's gross domestic product [13]. The demand for ICU resources is expected to further increase as our population ages [14, 15]. Although the direct costs of medical technologies account for only a proportion of total ICU costs (the capital costs may represent less than 5% of total costs [16]), the associated indirect costs can be substantial [17, 18]. For example, technologies such as mechanical ventilation or continuous renal replacement therapy are expensive because of the human resources and training needed to operate them and the high nurse-to-patient staffing ratios that they require [19, 20]. Although some ICU technologies clearly save lives, surprisingly few have been shown to improve outcomes such as mortality or health-related quality of life [21, 22].

Health technology assessment in general

Some researchers have described HTA as a ‘bridge between science and policy’ [3]. HTA is related to the evidence-based medicine movement [23], but its target audience is as often health policy makers, decision makers, and managers as individual clinicians. While the randomized trial is typically viewed as the best method for evaluating efficacy (the benefits of a technology when used optimally in carefully selected and compliant patients, cared for by highly competent clinicians) [24], HTA usually requires additional methods such as pragmatic randomized trials [25], economic evaluations [18, 26], analyses of administrative databases [27], studies of small area variation [28], outcome assessments [29, 30], policy analyses [31], systematic reviews, and others. An important component of every HTA is to ensure that all relevant literature is identified and incorporated into the analysis. The HTA policy brief from the World Health Organization is an excellent general summary of the topic and provides a current list of ongoing activities across the European Union [3].

Technologies that are clearly beneficial, such as mechanical ventilation for acute respiratory failure, do not require an HTA. HTAs are typically undertaken when the evidence of benefit from an intervention is uncertain (for example, bronchoscopy for diagnosing ventilator-associated pneumonia [32]), when the evidence is good, but the amount of health benefit from the technology appears small relative to the cost (for example, antibiotic impregnated catheters [33, 34]), or when the economic impact of acquiring the technology may be significant (for example, implementing remote telemedicine monitoring for ICUs lacking full-time intensivists) [35].

HTAs in the ICUs

Remarkably few ICU technologies have been evaluated despite the fact that many agencies for HTA have been established internationally (Table 1 presents a list of agencies). Some examples of ICU technologies for which HTAs have been performed include non-invasive positive pressure ventilation [8], left-ventricular assist devices [36], and cerebral microdialysis [37], and several technologies have been or are being evaluated using pragmatic clinical trials (“Tracman trial: tracheostomy management in critical care. Summary protocol”, National Health Service, Oxford, UK, www.tracman.org.uk, accessed 17 September 2007) [38] or cost-effectiveness analyses [39]. However, many ICU technologies are widespread despite equivocal or poor evidence of effectiveness, including bronchoscopic alveolar lavage for the diagnosis of ventilator-associated pneumonia [40, 41], oesophageal Doppler probes for the measurement of cardiac output [42], cooling catheters intended to augment traditional approaches to inducing hypothermia [43, 44], self-propelling nasogastric feeding tubes [45], bispectral index monitoring [46], and high-frequency oscillation for acute respiratory failure [47].

Table 1 Resources for those interested in conducting/learning more about HTA
Full size table

Why are ICUs relatively HTA-free zones?

Why have ICU technologies been adopted without rigorous HTA? Table 2 presents several explanations, which can be broadly categorized into clinical/methodological and social/political reasons, along with comments on their validity and possible solutions. Many of these challenges are not unique to critical care but reflect barriers to conducting clinical research (and thus also HTA) in general.

Table 2 Challenges for increasing health technology in the ICU
Full size table

Clinical/methodological obstacles to HTA

Inexact definitions of syndromes

If it is impossible to agree upon a definition of a disease, it will be difficult to assess the benefits and harms of potential therapies for that disease. A major stumbling block to HTA (and indeed to clinical trials) in general is the limited understanding (and differing definitions) of disease syndromes, their underlying pathophysiology, and patient populations [48]. For example, acute respiratory distress syndrome (ARDS) and sepsis actually represent a spectrum of underlying disease mechanisms and variable disease severity [49–51].

The solution to this problem is conceptually easy but practically difficult. Intensivists have agreed upon common definitions of disease syndromes for clinical trials (which typically include detailed criteria for various physiological measures) that represent a substantial proportion of the patients with a particular syndrome. Most research in sepsis and ARDS has used this approach [52, 53]. The assumption when using such ‘syndromic’ definitions is that the underlying pathophysiology is homogeneous, and therefore the response to any given technology or treatment will be consistent. However, this is problematic when varied disease processes lead to the same clinical syndrome.

Poor standardization of technologies

The indications, applications, and output of many new technologies have been poorly standardized making study design a challenge [54–57]. Before any technology can be assessed, there should be a clear understanding of the intended target population and associated clinical problem, and standardized operating procedures for the administration of the technology. Diagnostic devices pose unique challenges to HTA. The output of the device must be shown to be easily interpreted, reliable, and responsive. There should be convincing evidence that this output is accurate when compared to a criterion or gold standard, and thresholds for sensitivity, specificity, and likelihood ratios should be available. Most importantly, the output must provide information that will change the management of the patient by modifying the pre-test prognosis or by establishing a diagnosis that requires different and cost-effective treatment approaches (which will presumably lead to a change in patient outcome). Furthermore, there should be general agreement about the therapeutic actions that should be taken in response to the output provided by the new diagnostic device.

Methodological challenges

Blinding of patients and providers to some ICU interventions studied in randomized trials is not possible (for example, a novel type of mechanical ventilation). However, unblinded assessments of many interventions have yielded important results and are not a reason to abandon clinical trials [58]. The distress that critical illness places upon patients and their loved ones can make it difficult to obtain informed consent [59]. The sample sizes needed to detect plausible impacts of a new therapy or diagnostic test are often large, if one compares them to current best practice. The market for devices is substantially smaller than that for pharmaceuticals, and therefore less money is available for research. These are all legitimate and important issues but in our view do not justify the generally small number and small sample sizes of clinical trials of ICU technologies.

Two particularly challenging aspects of designing clinical trials and conducting HTAs in the ICU relate to how much investigators wish to ‘un-bundle the black box’, and how to define the appropriate control group. Concerning the former, most new technologies are often evaluated within the confines of standardized protocols. For example, a clinical trial of a multi-component protocol incorporating a novel central venous catheter capable of continuously measuring venous oxygen saturation for treatment of sepsis has been shown to improve survival, but whether the novel catheter alone or all components of the protocol must be provided is not clear [60, 61]. Another example is deciding whether trials of the pulmonary artery catheter have failed to show benefit because the catheter itself is not helpful, or whether the protocols directing the use and interpretation of the catheter lead to no improvements in the delivery of therapy [38, 62]. These examples highlight a fundamental challenge to HTA in general; most technologies will be evaluated in the context of a protocol, and one must decide how much this protocol should be unpacked to judge their success or failure [63].

Regarding the control group, concerns have been raised when studies examining protocolized approaches to ICU care require control groups that do not reflect standard care [64]. Furthermore, randomizing patients to fixed treatment protocols may disrupt pre-existing relationships between illness severity and level of therapy, leading to findings about a technology that are difficult to interpret [65].

Determining which outcomes to evaluate

Defining therapeutic benefit using meaningful clinical outcomes and quantifying harm has been difficult in critical care [48], and consequently ICU research often involves underpowered studies which detect only very large mortality differences. Furthermore, designing and funding large multi-centre trials that are adequately powered to detect mortality may be impractical, especially for smaller companies. Conversely, studying non-mortality endpoints poses other challenges. Surrogate clinical endpoints, such as ventilator-free days, may be more susceptible to bias and may not be associated with mortality differences [66, 67]. Similarly, physiological endpoints may be misleading and do not always translate into improved clinical outcomes [58]. Patient-centred outcomes, such as health related quality of life, have been infrequently employed in critical care and are expensive to measure [48]. Finally, the choice of outcome to be evaluated is often related to the cost and risk of the intervention; in general, devices associated with higher risk should have clear evidence of therapeutic benefit in terms of clinical outcomes. Such challenges with defining meaningful endpoints for clinical trials also make it more difficult to conduct HTA in ICU.

Social and political obstacles

Culture of intensivists to be early adaptors, and physiologically oriented

Intensivists are accustomed to working with complex diagnostic and therapeutic technologies and may be more willing to incorporate new devices into their practice despite limited assessments [68]. The familiarity of ICU clinicians with technological devices may have created an ‘early adopter’ culture and caused many technologies of questionable benefit to be implemented [5]. Intensive care has traditionally been concerned with correcting and treating physiological derangements, and this likely has caused intensivists to enthusiastically adopt new devices for measurement.

Relatively high status of ICU within the health care system

Critical care is a politically powerful sector in health care and historically may not have been required to justify expenditures with the same level of scrutiny as other parts of the health care system. Furthermore, a new technology still comprises only a relatively small proportion of the overall costs of the ICU and therefore may not be scrutinized to the same degree as a similar technology implemented in an out-patient department (where it would comprise a much larger proportion of that department's costs).

Lack of stringent regulatory requirements for randomized trials of non-drug devices

There has been a lack of strong political will from regulatory bodies to mandate HTA of new devices prior to their implementation (unlike pharmaceutical agents which require, at a minimum, randomized trial evidence of efficacy and safety) [69, 70]. Identifying new technologies for HTA before they are adopted for widespread use can also be a challenge [71]. The role of funding agencies and government regulating bodies is crucial, and historically there may have been less attention paid to the relatively new specialty of critical care. For example, the National Institutes of Health has no critical care section, and thus ICU research has typically been funded by diverse branches, for example, the National Heart, Lung, and Blood Institute for research on mechanical ventilation for ARDS and the National Institute for Neurological Disorders and Stroke for research on the use of thrombolytic agents for stroke. In contrast, the United Kingdom has developed a national body for HTA in ICU; the early success of this group at evaluating widely used devices such as the pulmonary artery catheter and tracheostomy is encouraging (“Tracman trial: tracheostomy management in critical care. Summary protocol”, www.tracman.org.uk) [72].

The politics of HTA

There are several other challenges to conducting HTA that are not unique to ICU. Historically, many of the generators of HTAs have been non-clinicians and consequently may have lacked credibility with the payers, clinicians, and patients. Some HTAs appear to demand unrealistic methodological perfection, which can also lead to a lack of credibility in clinical circles. Because HTAs focus on effectiveness rather than efficacy, their conclusions are more likely to be negative than individual clinical trials. Also, when the evidence for the benefits of a technology seems clear, HTAs are unlikely to be solicited. Because HTAs are not usually supported by industry, and there are few politically powerful advocates for HTA in the ICU clinical world, their results tend not to be aggressively disseminated. Finally, there may be concerns that mandating HTAs for new technologies will slow the progress of ICU research and development, which is already resource and time intensive.

Increasing HTA in ICU

Given the limited resources available for health care and increasing demands, funds spent inefficiently or unnecessarily on technologies in the ICU will mean that effective therapies are not available for other sectors within the hospital, and potentially effective ICU devices may not funded because of lack of resources. Therefore we believe that it is important that HTAs be undertaken more frequently, and their results used to influence clinical practice and hospital and regional-level policy decisions. We believe that demand from stakeholders (patients and taxpayers), clinicians working in areas of health care outside of the ICU, and funding agencies or payers will soon lead to more calls for HTA of ICU technologies (for example, the recent HTAs that have been sponsored by the United Kingdom's National Health Service (“Tracman trial: tracheostomy management in critical care. Summary protocol”, www.tracman.org.uk) [72]. This will require a culture change related to the implementation of many devices. New devices may appear promising, but they must be adequately scrutinized before we employ them on our fragile critically ill patients.

It will be very difficult to remove currently entrenched technologies from our routine care, and studying such interventions would require equipoise on the part of clinicians. For this reason we instead suggest mandating HTA for all new or proposed therapeutic devices to prevent their widespread dissemination before effectiveness and cost-effectiveness have been demonstrated. Regulatory bodies should adopt a more standardized approach that includes rigorous HTA prior to approving or licensing such new technologies, similar to the process already in place for licensing pharmaceuticals in most jurisdictions. Access to new therapeutic devices that have not yet been evaluated by HTA should be restricted to use within the confines of ongoing clinical trials or prospective studies with suitable controls. Such trials must consider patient-centred outcomes, such as health related quality of life or mortality, and measure not only physiological endpoints. Furthermore, the onus should be on industry to establish standard operating procedures for the use and interpretation of new technologies. Consideration could still be given to providing access to new yet unproven technologies in special circumstances on a compassionate basis, similar to the current practice for many chemotherapeutic agents for treating end-stage cancers. Table 2 provides further suggestions for increasing the use of HTA in ICU.

It should still be possible to evaluate many of the ICU technologies that are already in widespread use. However, the funding for such evaluations is unlikely to come from industry unless demanded by regulating bodies or government. For existing technologies frontline clinicians could be engaged in the HTA process using pragmatic clinical trials or cluster randomized trials for technologies in which there is equipoise [73]. Observational studies have also proven to be useful for evaluating technologies in situations where a lack of clinician equipoise or other considerations make clinical trials impractical [74].

Deciding whether to adopt a new technology in the ICU

Even if an HTA has been performed for an ICU technology, a decision must be made regarding its appropriateness for adoption. Economic evaluations of ICU technology must be rigorously critically appraised and evaluated, and the incremental cost-effectiveness of the technology compared with the current standard of care. Of course, many issues other than cost effectiveness, such as ethical and political considerations, will affect the implementation of a new ICU device.

The ‘rule of rescue’

When deciding which technologies to adopt at the level of a healthcare system, using cost-effectiveness analyses alone may fail to consider society's proclivity to save endangered life, or the “rule of rescue” [75]. A compelling example of this pitfall occurred when the state of Oregon sought to establish health care priorities using solely cost-effectiveness analyses; this process lacked face validity because it favoured minor treatments over lifesaving ones. Similarly, there will always be some situations for which a new technology may be considered as a rescue therapy prior to having undergone an HTA. The framework for using an unproven therapy for compassionate reasons has become well established for pharmaceuticals. A similar approach could be adopted for new technologies, for example, using high-frequency oscillatory ventilation for patients with refractory hypoxaemia in ARDS. In these situations careful and systematic documentation of the indications, patient characteristics, response to treatment, and adverse events should be required to create a data repository for informing decisions about the future use of the technology.

Conclusion

In the context of rising healthcare costs there is an obligation from clinicians, administrators, and policy makers to scrutinize all new healthcare expenditure to ensure that cost-benefit is established. Policy makers and healthcare funding bodies may soon expect more accountability from ICU clinicians and administrators for the use of ICU technologies. In this context we expect that there will be more frequent calls for HTAs. To ensure that costly and unproven technologies are not adopted prior to proven efficacy there will also need to be increased monitoring and oversight at the system-level from policy makers and healthcare funding bodies. Until these changes occur, we can contribute as ICU clinicians by remaining sceptical of new technologies that have not been adequately evaluated by HTA.