Outcome assessment for out-of-hospital cardiac arrest patients in Singapore and Japan with initial shockable rhythm

This was a secondary analysis of data from two prospective observational registries. The first was the Comprehensive Registry of Intensive Care for OHCA Survival study in Osaka (Osaka-CRITICAL), and the second was the Singapore Pan-Asian Resuscitation Outcomes Study (SG-PAROS). Details of these databases were previously published [17‐20] and are described in the S-Method 1, Additional file 1. The Osaka-CRITICAL registry was a multi-institutional prospective observational study of OHCA patients whose data were obtained from 15 tertiary critical care medical centers (CCMCs) and one non-CCMC community hospital in Osaka Prefecture [21]. The data are available from 2012 to 2019. The SG-PAROS database is also a prospective, population-based registry designed to provide data on the epidemiology, management, and outcomes of OHCA in Singapore from 2010 to 2020 [19, 20]. The ethics committee of Kyoto University approved the retrospective analysis of the Osaka-CRITICAL study database (R-1045), and the Centralized Institutional Review Board and Domain Specific Review Board in Singapore granted approval for the SG-PAROS database (ref no: 2013/604/C, 2013/00929 and 2018/2937) and Domain Specific Review Board (ref no: C/10/545 and 2013/00929). Informed consent was waived due to the nature of the observational study.

This study was performed using two established databases. We first developed and validated a prediction model for the outcomes mentioned below using the data from Osaka-CRITICAL, and then, we evaluated the outcomes by comparing the actual observed outcomes in Singapore and the expected outcomes derived from Osaka using the prediction model mentioned above.

Regarding ECPR availability, tertiary critical care centers in Osaka are basically able to provide the ECPR 24/7 basis. Several papers have previously described how ECPR is currently implemented in Japan [15, 22‐24], but we have included some explanation of ECPR in Additional file 1: S-Method 1. In Singapore, on the other hand, ECMO can only be implemented in two academic centers and ECPR was rarely performed during the study period.

This study included adult (18–74 years old) OHCA patients with initial shockable rhythm (ventricular fibrillation [VF] or pulseless ventricular tachycardia [VT]) confirmed at the scene. The inclusion criteria were similar to what is used for the selection of ECPR candidates in Osaka [12, 13]. This study excluded patients with the following: those who opted out of the study, had no prehospital record, had no resuscitation attempted, age younger than 18 years old or older than 74 years, had cardiac arrest due to external causes such as trauma, those who were not cardiac arrest at the contact with paramedics, and those whom the resuscitation was terminated in prehospital settings. In the derivation and validation of the prediction model, we included patients who met the criteria mentioned above from the Osaka-CRITICAL database between 1 July 2012 and 31 December 2019. In the main comparative analysis, we included adult OHCA patients with initial shockable rhythm as mentioned above from the SG-PAROS database between 1 July 2010 and 31 Dec 2016 and divided them into subgroups based on the cardiac rhythm at emergency department (ED) arrival: those with ROSC, those with persistently shockable rhythm, and those with nonshockable rhythm although they initially had a shockable rhythm. We excluded cases from 2017 to 2020 for the main analysis because the cardiac rhythm on ED arrival was not collected in the SG-PAROS database from 2017 to 2020, and we could not investigate the outcome by cardiac rhythm on hospital arrival for these cases. Regarding termination in Japan, the termination of resuscitation by paramedics is strictly limited. Paramedics can terminate the resuscitation only when the cardiac arrest cases are clearly considered dead such as presenting rigor mortis (the details are described in Additional file 1). Further, in the SG-PAROS database, the number of cases in which the resuscitation was terminated at the scene was only three cases during the study period (as shown in the study flowchart).

The primary outcome of this study was one-month survival with favorable neurological outcomes defined as Cerebral Performance Category (CPC) 1 or 2. CPC is commonly used to evaluate the neurological status after OHCA as follows (category 1, good cerebral performance; category 2, moderate cerebral disability; category 3, severe cerebral disability; category 4, coma or vegetative state; category 5, death/brain death) [25]. The secondary outcome of this study was survival admission to hospital (admission) and one-month survival. The neurological outcome and survival were evaluated by the physicians in charge or the researcher associate from clinical records, telephone, and/or face-to-face interviews [21, 26].

From both databases, we obtained the following clinical information: sex, age, witnessed, bystander CPR, automated external defibrillator (AED) applied by a bystander, prehospital initial cardiac rhythm, prehospital advanced airway management, adrenaline administration in the prehospital settings, cardiac rhythm on ED arrival, resuscitation time course, the disposition in the ED, application of ECMO, percutaneous coronary intervention (PCI), targeted temperature management (TTM), and outcomes. The disposition at the ED was defined as death in ED without admission, or admission to the hospital. Transfer to another hospital was categorized as admission to the hospital. The variable details are explained in Additional file 1: S-Method 2. We treated extreme outliers or contradictory data as missing. Missing data were imputed by a random forest-based imputation procedure using the “missForest” package for the study participants with eligibility [27]. We also added the detail of this imputation method and the missingness of the variables in Additional file 1: S-Method 3, S-Result 1.

We described the patient characteristics, in-hospital information, and outcomes. Data were shown as the median and interquartile range (IQR) for continuous variables, and as numbers and percentages for categorical variables.

Generally, to compare outcomes between different hospitals or regions, it is not appropriate to compare crude rates because the distribution of patient characteristics and baseline risk (case mix) is different between the populations, especially when case mix is associated with outcomes [28‐30]. To deal with this, first, we developed and validated a prediction model for outcomes using the Osaka-CRITICAL data, and second, we estimated the ratio of observed to expected outcomes (OE ratio) in SG-PAROS data adjusted for patient characteristics. The OE ratio is commonly used to compare outcomes with their expected results based on the average treatment in the reference population [28, 29].

In the derivation and validation of the prediction model using data from the Osaka-CRITICAL database, we divided the included patients into two cohorts: the derivation cohort from 2012 to 2017 and the validation cohort from 2018 to 2019. The derivation cohort was the reference population for comparing outcomes with Singapore. Traditionally, for risk adjustment, a logistic regression model is commonly used; however, we used a random forest (machine learning) model which has some advantages. For example, a random forest can handle nonlinear relationships in the data, reduce the risk of overfitting, and predict outcomes with high accuracy [30‐33]. We derived the prediction models using random forest to incorporate the following covariates: sex, age, witnessed, bystander CPR, bystander AED, prehospital advanced airway management, prehospital adrenaline administration, cardiac rhythm on hospital arrival, and time from call to hospital. The covariates above were selected because of the availability in both datasets, clinical importance, and intention to adjust the characteristics of the case mix on hospital arrival to evaluate the potential impact of treatment strategy. Evaluating the model performance in the validation cohort, the receiver operating curves (ROCs) and the C-statistics (the value of area under the curve [AUC]) with 95% confidence interval) were calculated. Further, calibration plots using the quantile of the predicted probability were drawn to indicate the relationship between the predicted and observed probability of outcomes in the validation cohort. The other details of the derivation and validation of the models are described in the S-Method 4 in Additional file 1.

For evaluation of outcomes in Singapore, we applied the model to the SG-PAROS database to calculate the risk-adjusted predicted probability and the OE ratio with 95% confidence intervals (CI) by the subgroups based on the cardiac rhythm at ED arrival: those with ROSC, those with persistently shockable rhythm, and those with nonshockable rhythm although they initially had a shockable rhythm.

To evaluate the robustness of the results, we performed the following sensitivity analysis. In sensitivity analysis 1, we also developed and validated different prediction models using the least absolute shrinkage and selection operator (LASSO) and conventional logistic regression models (Additional file 1: S-Method 5). In sensitivity analysis 2, we developed models excluding prehospital airway management and administration of drugs as predictors, considering the criticism that these variables are not appropriate as predictors as the practices regarding these procedures might differ considerably based on the region or location and they are more EMS system factors. In sensitivity analysis 3, to consider the possibility of changing results over time, we evaluated outcomes using the data from SG-PAROS 2017-2020 which were not included in the main analysis. As mentioned above, the cardiac rhythm on ED arrival was not available in the SG-PAROS 2017-2020. Thus, we developed and validated a model excluding the variable of the cardiac rhythm on ED arrival, and applied it to the SG-PAROS 2017-2020 data, and calculated the OE ratio by the prehospital ROSC (yes/no). All estimates were calculated with 95% confidence intervals (CIs). All statistical analyses were performed with R software, version 4.1.2 (R Foundation for Statistical Computing).

Of 18,379 patients registered in the Osaka-CRITICAL database, 1,255 OHCA patients with initial shockable rhythm were included in the analysis and divided into two cohorts (Derivation cohort, n = 885, Validation cohort, n = 370) (Fig. 1). Of 12,546 patients registered in the SG-PAROS database from 2010 to 2016, 1,789 patients with initial shockable rhythm were included in the analysis (Fig. 1). The median [IQR] age was 61 [50–68] in the derivation cohort, 60 [48–68] in the validation cohort, and 57 [50–65] in the SG-PAROS. The other basic characteristics of databases were similar except for the proportion of prehospital advanced airway management, and administration of adrenaline (Table 1). The proportion of ECMO was 35% (307/885) in the derivation cohort, 41% (150/370) validation cohort, and 0.6% (10/1,789) in the SG-PAROS cohort (Table 1). Good neurological outcomes were 31% (278/885), 34% (125/370), and 14% (244/1,789), respectively (Fig. 2). The characteristics of the cardiac rhythm on hospital arrival of the derivation cohort and the SG-PAROS were described in Table 2. Those of the validation cohort and missing values in each cohort are shown in Additional file 1: S-Results 1 and 2. Regarding PCI, TTM, and ECMO, among the patients who were the admitted to hospital, PCI was performed in 41% (257/629) of the derivation cohort, 43% (124/291) in the validation cohort, and 51% (309/609) of the SG-PAROS, TTM was applied in 58% (365/629) of the derivation cohort, 52% (150/291) in the validation cohort, and 33% (199/609) of the SG-PAROS database, and ECMO was applied in 43% (272/629) of the derivation cohort, 49% (143/291) in the validation cohort, and 1.1% (7/609) of the SG-PAROS database (Table 3). We also showed the frequencies by subgroups of cardiac rhythm on ED arrival (Table 3).

The random forest models were built using the derivation cohort from the Osaka-CRITICAL database. The details of model features are shown in Additional file 1: S-Results 4. In the validation cohort, for the good neurological outcome, the AUC of ROC of the random forest models was 0.896 [95%CI: 0.861–0.93] and the calibration plots showed good calibration performance (Additional file 1: S-Results 4). The other models' performances are also described in the (Additional file 1: S-Results 4). The OE ratios in the validation cohort implied that the observed outcomes by each cardiac rhythm on ED arrival were as expected from the derivation cohort (Additional file 1: S-Results-4).

The observed and expected probabilities by ED arrival are shown in Fig. 3. Overall, observed outcomes in the patients with ROSC were similar to expected; however, those without ROSC were lower than expected. The OE ratio for good neurological outcome in SG-PAROS was 0.905 [95%CI: 0.784–1.036] in the patients with ROSC on ED arrival, 0.369 [95%CI: 0.258–0.499] in the patients with shockable rhythm without ROSC, and 0.137 [95%CI: 0.065–0.235] in the nonshockable without ROSC (Fig. 4). For survival, the OE ratio was 0.903 [95%CI: 0.793–1.021] in the patients with ROSC on ED arrival, 0.369 [95%CI: 0.279–0.470] in the patients with shockable rhythm, and 0.235 [95%CI: 0.166–0.316] in the nonshockable (Fig. 4). OE ratio for admission was 1.006 [95%CI: 0.897–1.122] in patients with ROSC on ED arrival, 0.334 [95%CI: 0.276–0.397] in the patients with shockable rhythm, and 0.385 [95%CI: 0.332–0.443] in the patients with nonshockable rhythm (Fig. 4). For these primary and secondary outcomes, the observation in the patients with ROSC on ED arrival was as expected; however, observations in the patients without ROSC were lower than expected.

In sensitivity analysis 1, the logistic model and Lasso for the outcomes were also derived and validated (Additional file 1: S-Results 3–6). The results were consistent with that of the main analysis (Additional file 1: S-Results 7–9). In sensitivity analysis 2, the random forest models did not include prehospital airway management and drug administration as predictors. The OE ratios in this model were also similar to those of the main analysis (Additional file 1: S-Results 10 and 11). In sensitivity analysis 3, we evaluated the OE ratios in SG-PAROS 2017-2020. The results were consistent with the main analysis (Additional file 1: S-Results 12–14). The ECMO was also rarely performed (11/1,526, 0.7%) in SG-PAROS 2017-2020.

In this observational study, we evaluated the outcomes of OHCA patients with initial shockable rhythm in Singapore with an extremely low proportion of ECPR usage compared to those in Osaka with a high proportion of ECPR usage. The outcomes were similar between Singapore and Osaka, among the initial shockable rhythm OHCA patients with ROSC. On the other hand, the outcomes of the initial shockable rhythm OHCA patients without ROSC in Singapore were lower than expected derived from those in Osaka. This result was supported by several sensitivity analyses.

The results of this study support the idea that aggressive ECPR implementation may help to improve the outcomes of refractory OHCA patients with initial shockable rhythm. For example, the survival and neurological outcomes were almost the same between Singapore and Osaka among patients with ROSC, and these results suggest that the differences in outcomes may not be caused by differences in treatment after ROSC, such as the quality of temperature management or other intensive care. Furthermore, although hospital admission in Singapore was almost the same as expected derived from Osaka in the patients with ROSC, it was lower in patients without ROSC. This suggests that issues that make a difference in outcome may be prior to ROSC. In addition, the characteristics and important factors available at ED arrival were included in the model for risk adjustment; therefore, outcome differences were likely due to factors after hospital arrival. We did observe in this study that post-resuscitation care such as PCI and TTM were different between them; however, the overall frequency of these post-resuscitation treatments were not hugely different among all the OHCA patients who were admitted to hospitals in both datasets (Table 3). On the other hand, ECMO was rarely performed in Singapore but it was aggressively implemented in Osaka. Although other differences in post-resuscitation care such as how fast PCI or TTM was initiated, or the implementation of post-resuscitation care might have influenced the results, it may be reasonable to consider that an aggressive ECPR implementation policy contributed to improving the outcomes of refractory OHCA patients with initial shockable rhythm. Furthermore, some previous studies also suggested results consistent with this study. A nationwide OHCA registry in Japan reported that in patients with initial shockable rhythm who continued to have a shockable rhythm on hospital arrival (age: median [IQR], 66 [54–75] years), good neurological outcome and survival were 17.4% and 26.6%, respectively, and the proportion of ECMO implementation for them was 54.0% [34]. These results were almost the same in the Osaka-CRITICAL database.

One of the strengths of this study is that it was like a “natural experiment,” and the results are worth considering. Previously, one observational study attempted to investigate the outcome of OHCA patients between facilities with and without ECPR capability in the same area [35]. However, these studies could not eliminate the risk of selection bias based on different indications for ECPR in the areas. On the other hand, this study investigated the outcomes in Singapore and Osaka, which are large urban cities in Asia, and the patients in this study had similar characteristics such as age, the proportion of bystander CPR, and implementation of AED, and the percentage of PCI and TTM performed among the admitted patients were not so different; however, the proportion of ECPR implementation was very different. Although there might be some unmeasured confounding factors, this was like an ideal “natural experiment.”

One other strength was that this study investigated the outcome differences at the systems level. Previously, one RCT (the ARREST trial) showed a beneficial effect of ECPR, but the latest RCT did not [36, 37]. However, critics of this latest RCT suggest that the healthcare system was not adequately set up to provide high-quality ECPR, as the time taken to start ECPR was rather long (the median [IQR] duration from the arrest to the start of ECMO was 74 [63, 87] minutes) [37]. On the other hand, in the ARREST trial, the mean (standard deviation) time to start ECMO was 59 (28) minutes [36]. The outcomes of OHCA patients treated with ECPR are time sensitive, and a longer duration between the arrest and the start of ECPR has worse outcomes [38]. Successful ECPR essentially requires a chain of appropriate actions including immediate high-quality bystander CPR and AED, early recognition of the indications for ECPR, prompt transfer to a high-level facility with experienced staff, and well-trained teamwork for cannulation of ECMO, resulting in a shorter time to initiation of ECPR. Accordingly, an ECPR strategy should be evaluated at the system level. In the Osaka-CRITICAL database, the median [IQR] duration from ambulance call to start ECPR was 53 [45, 64] minutes [15], and the quality of systems providing ECPR in Osaka was relatively high. In this regard, although this study did not directly evaluate the association between the ECPR and outcomes, it may provide evidence that a good quality system of early ECPR implementation may lead to favorable outcomes in refractory OHCA patients with initial shockable rhythm.

Although the results of this study suggest the potential of ECPR to improve outcomes, we believe that there should still be caution before adopting an aggressive ECPR policy in all situations because of several concerns. One of the concerns is that the population which is suitable for ECPR is still unclear. The indications for ECPR in the Osaka-CRITICAL study were decided by each hospital protocol or physician in charge, but there were no strictly defined inclusion criteria. Previously, some studies were investigating which population is suitable for ECPR [15, 16, 39‐41], without a clear consensus.

The other important concern is that it is challenging to establish a well-functioning, sustainable system. As mentioned, successful ECPR requires a well-designed healthcare and education system [42‐44]. However, building such a system may require time, cost, education, human resources, a skilled team, and the accumulation of experience [42‐44]. Furthermore, since ECPR is a time-critical treatment [38], it is challenging to build a system providing ECPR equally to people living in different parts of a country, where the time and distance to hospitals vary [45]. Also, the implementation of ECMO itself requires a lot of costs [46, 47], which can be a burden for patients and health insurance systems. Previously, some studies of the cost-effectiveness of ECPR compared with conventional CPR indicated that the ECPR was acceptable based on the cost-effectiveness in Japan and some other countries [46, 47]. However, these studies only considered the cost of the ECPR itself, not how much funding is necessary to build the system and sustain it. In Japan, medical cost is a huge burden and there are some concerns about how to sustain the current healthcare system [48]. Furthermore, there are other priorities for improving the outcomes of the patients with OHCA such as promoting basic life support education or improving the accessibility of AED in public places [49]. Accordingly, further research is warranted to evaluate the cost-effectiveness of any strategy.

There were several limitations. First, we used the Osaka-CRITICAL database as a reference, which was not a population-based registry, as the patients are from tertiary care hospitals, whereas the SG-PAROS was a population-based registry. Although 99% of the patients in both databases were patients transferred to tertiary care hospitals, there might be the risk of selection bias in the reference population. Second, there might be the risk of bias due to unmeasured factors prior to hospital arrival that were not included in the model such as past medical history, frailty, socioeconomic status, or differences in prehospital emergency systems between Singapore and Osaka. We could not account for the clinical impact of these factors. However, we suggest that if unmeasured factors influenced the result, the magnitude was not so huge because the outcomes of patients obtaining ROSC in Singapore were almost the same as in Osaka. If the prehospital emergency systems in Singapore had a huge negative impact on the outcomes, the outcomes of patients with ROSC would also have been worsened. Third, there was some risk of overfitting of the prediction model. However, we performed some sensitivity analysis, and it supported that the main results were robust, even if the model was changed slightly or the statistical procedure was changed. Fourth, this study could not investigate the direct association between ECPR policy and outcome. This study only suggested that something was different between Osaka and Singapore after hospital arrival which led to worse outcomes among the patients with OHCA who had initial shockable rhythm but did not achieve ROSC. For example, some other resuscitation procedures in the ED, the timing of the decision to terminate resuscitation, or other differences in post-resuscitation care such as how fast PCI or TTM was initiated, or the implementation of post-resuscitation care bundle might cause differences. Unfortunately, we could not compare the details of resuscitation due to lack of data granularity. Further studies are necessary to confirm the generalizability of these results.