Heterogeneity in the definition of delirium in ICUs and association with the intervention effect in randomized controlled trials: a meta-epidemiological study

Delirium is a neuropsychiatric syndrome reflecting an acute brain dysfunction that occurs frequently in intensive care units (ICUs) [1]. It may be induced by a physiological stress related to a systemic pathology or to the critical care interventions and other specific factors (e.g., sleep disturbances, light pollution at night). Characteristics such as age, sex, disease severity, mechanical ventilation are risk factors [1, 2]. The incidence is about 30%, with significant variations (from 10 to 80%) depending on the admission cause and the definition used to characterize it [3]. This disorder is challenging to assess reliably because of varied symptomatology, including fluctuating mental status, disturbance in consciousness, attention and judgment disorders, disorientation, circadian disturbances, and psychomotor slowing and/or agitation, which could lead to under-recognition [4, 5]. The reference diagnosis is based on the diagnostic and statistical manual of mental disorders (DSM) criteria after a psychiatric evaluation. To facilitate the diagnosis and recognition of this trouble by ICU physicians in their everyday practice, scales were developed in the 2000s and included the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) or the Intensive Care Delirium Screening Checklist (ICDSC). Both scales were validated after comparison with the DSM criteria [6].

Delirium is associated with increased mortality, prolonged hospital stay, prolonged mechanical ventilation, and increased risk of long-term cognitive impairment [7‐10]; therefore, it is a major therapeutic issue. Current research on delirium in ICUs focuses on the evaluation of prevention and treatment strategies including various pharmacological or non-pharmacological interventions that have been evaluated in many randomized controlled trials (RCTs) [11]. However, few studies have raised the issue of the definition of delirium, which seems to be heterogeneous in the published literature despite the existing tools. The definition of an outcome such as delirium may be an important source of heterogeneity and variation in the intervention effect making comparison between trials on this topic difficult [12].

In this study, we aimed to evaluate the heterogeneity in the definition of delirium used in RCTs included in meta-analyses evaluating the prevention or treatment of delirium in ICUs and to explore whether the intervention effect varies depending on the definition used.

In this systematic review of meta-analyses including RCTs assessing prevention or treatment strategies of delirium in ICUs, the definition of delirium was heterogeneous across trials, and one-fifth did not report how they defined delirium. We attempted to assess the impact of this heterogeneity on intervention effect estimates by using a meta-epidemiological approach and found no significant difference between trials using a validated definition and those using a non-validated one in our primary analysis. However, this analysis included few studies and may lack power. The secondary analysis, based on a multilevel model including more studies, found significantly larger intervention effects in trials using a non-validated definition than those using a validated one, which suggests an association between the definition used and intervention effect.

This is the first study evaluating the heterogeneity in the definition of delirium and its association with the intervention effect by a meta-epidemiological approach, the reference method to identify bias [13]. Our sample included mostly recent meta-analyses. The definition of delirium was extracted directly from the RCTs because this information was seldom reported in meta-analyses despite its importance. We classified delirium definitions into different categories specified a priori based on a literature review and considered as a validated definition the DSM criteria because it is the gold standard and CAM-ICU, ICDSC, NEECHAM and DRS-R98 because they have been validated in numerous countries and publications [6, 18‐22]. Our classification of validated and non-validated definitions is consistent with the NIDUS list assessment tools for delirium screening [17]. We used two different approaches for the meta-epidemiological analysis and performed sensitivity analyses accounting for important confounding factors.

However, our study has limitations. First, the search strategy might have missed some meta-analyses, but this should not have introduced bias. Second, because meta-analyses were covering the same research area, many RCTs were included in several meta-analyses and had to be removed, which left fewer RCTs available for analysis. Third, it was not reported in included RCTs whether delirium was systematically assessed by research personnel or as part of routine care. Assessment in the context of routine care may lead to under-recognition even when using a validated tool. However, because RCTs are experimental studies with delirium reported as a primary or secondary outcome, we can reasonably assume that it was systematically assessed. Fourth, concerning the meta-epidemiological analysis, only a small number of meta-analyses were included, particularly in the primary analysis, so this analysis may lack power. However, we could not exclude that a difference might exist, especially because the secondary analysis revealed a significant difference. Finally, the development and validation of new tools such as CAM-ICU and ICDSC were concomitant and contributed to an evolution in practices in ICUs with more screening and treatment of delirium. In 2017, the bundle ABCDEF guidelines appeared, representing evidence-based guidelines for physicians to optimize ICU patient care. Management of delirium is a large part of this bundle, which has been increasingly used in the day-to-day care. We tried to collect information on this bundle in trials, but it was seldom reported.

A previous systematic review analyzed the outcomes in RCTs evaluating prevention or treatment strategies of delirium in ICUs [25]. The authors found heterogeneity and multiplicity in outcomes, the most frequent tools used being the CAM-ICU and the ICDSC, which is consistent with our results. They suggested that delirium should be screened regularly with a reliable tool and that a core outcome set be developed to inform delirium research. Another recent systematic review evaluated the heterogeneity in design and analysis of ICU delirium outcome [26]. The authors included RCTs with delirium as primary outcome, evaluated by a validated tool, based on the NIDUS assessment tool list and the DSM criteria, in agreement with our classification. The most frequent tool was the CAM-ICU, which is also consistent with our findings. The authors suggested developing specific methods for statistical analyses and reporting in RCTs of delirium that, if used by most researchers, could improve the quality of clinical trials and the comparison between them. However, they did not raise the issue of the heterogeneity among the definitions of delirium used and included only RCTs using a validated definition. We believe that harmonization in the definition of delirium is the first step to improve the quality of the research on this topic, allowing the research community and physicians to talk about the same thing when considering delirium.

Our study showed heterogeneity in the definition of delirium, with nine different definitions reported, even if CAM-ICU was the most frequently used. The gold standard to define delirium is the DSM criteria, but this evaluation should be made by a psychiatrist. Before the development of tools such as CAM-ICU, ICDSC or NEECHAM, physicians and researchers used various symptoms such as agitation or confusion to define this disorder, thus increasing the heterogeneity in definitions used. One-fifth of trials of delirium did not define delirium. Although bad reporting does not mean bad methods [27], the lack of definition limits the interpretation of results including the comparison with other trials. This is why we chose to consider a not-reported definition as non-validated.

Concerning the impact of the heterogeneity in delirium definitions on the intervention effect, the primary meta-epidemiological analysis did not show any significant difference, although the difference was in the direction we expected, which suggests that trials using a non-validated definition may overestimate the intervention effect. However, this analysis lacks power given the small number of studies included. The two-step approach is the most used and the reference method for meta-epidemiological analyses [28], but it is restrictive because it requires including at least one RCT using a validated definition and one using a non-validated definition within each meta-analysis, thus reducing the number of contributing meta-analyses (seven in our study). This constraint does not exist with the multilevel approach, which allows for the inclusion of more meta-analyses. Use of this multilevel model revealed a significant difference in the same direction, which supports the possible existence of larger estimates of intervention effects in trials using a non-validated definition. The secondary analysis with different categories suggests that the difference in intervention effects between trials using a validated and a non-validated definition may be driven by the trials not reporting the definition used. Not reporting the definition of an outcome may reflect a lack of rigor in these trials, which could partly explain the results.

Sensitivity analysis for the primary analysis did not reveal significant differences, but after adjustment for the items of the RoB1, RORs were closer to one. Hence, the non-significantly larger estimates in trials using a non-validated definition may reflect a weaker methodology. In previous meta-epidemiological studies, an inadequate sequence generation, the absence of allocation concealment or lack of blinding was associated with an overestimation of intervention effect [29‐32].

Improving how delirium is defined is an important way to limit waste of research [33‐35] because it would facilitate the comparison between RCTs, leading to better-quality systematic reviews and meta-analyses on this topic [36, 37]. It may result in a better understanding of this disorder and a better evaluation of the efficacy of therapeutic or preventive interventions. For physicians, it would also improve the diagnosis of delirium, thus resulting in more efficient patient care in ICUs.

The NIDUS proposal listing tools for defining delirium was an important step. However, it includes 34 assessment tools for delirium screening, diagnosis or severity and 5 brief screening tools. Although this catalogue helps clarify the definition of delirium and facilitates the comparisons between trials, there is still a large panel of tools used, which results in heterogeneity with a possible association with the intervention effect. Physicians and researchers should agree on which validated tool should be used to define delirium. The development of a core outcome set may be useful, as suggested by Rose et al. [25].

This systematic review highlights the heterogeneity in definitions of delirium used in meta-analyses and RCTs assessing its prevention and treatment in ICUs, with one-fifth of trials even not reporting how they evaluated delirium. The primary analysis found no significant difference in intervention effects for RCTs using a validated definition and those using a non-validated one but lacked power given the small number of studies included. The secondary analysis, based on a multilevel model including more RCTs, revealed significantly larger intervention effects for trials using a non-validated than a validated definition. A single and consensual definition of delirium is important for a better evaluation of the interventions to prevent or treat delirium and to improve the quality of research. It may also lead to a better care of patients in everyday practice. The development of a core outcome set on this topic is urgently needed.