Comparison of the patient-derived modified Japanese Orthopaedic Association scale and the European myelopathy score

Degenerative cervical myelopathy (DCM), previously referred to as cervical spondylotic myelopathy, comprises a range of age-related pathologies that affect the cervical spine and lead to spinal cord compression and neurological dysfunction [1, 2]. With an aging population, DCM contributes substantially to global disability, reduced quality of life, and healthcare costs [3]. The Japanese Orthopaedic Association (JOA) scale [4], the modified Japanese Orthopaedic Association (mJOA) scale [5], and the European myelopathy Score (EMS) [6] classify the severity of myelopathy [7‐9]. Following adaptation for Western cultures, the mJOA is currently one of the most widely accepted assessment tools for myelopathy and functional status in patients with DCM [10, 11]. The mJOA is, however, physician-administered and not patient-reported, a drawback that can introduce classification bias. Further types of bias, such as recollection bias or reporting bias, might be added if data are collected retrospectively or based on patient records. Patient-reported outcome measures (PROMs) like the EMS overcome these problems, and the EMS has been used in the Swedish Spine Register (Swespine) since 2006 [12].

With the recent development and validation of the patient-derived modified Japanese Orthopaedic Association (P-mJOA) scale [13], it is now possible to use a myelopathy scale that is considered gold standard and patient-reported. Further, as the Swedish translation of the P-mJOA and the EMS has been used in parallel in Swespine since the fall of 2020, a comparison of these two PROMs is now appropriate. In case of concordance, it would also be possible to translate large amounts of long-term EMS-based follow-up data and findings into a more international setting. The comparison with the P-mJOA is therefore fundamental to legitimize and increase the transferability of previous findings, while also serving as a validation of the P-mJOA from a European point of view. For these purposes, the aim of this study was to compare the P-mJOA with the EMS for the assessment of DCM.

This study was approved by the Swedish Ethical Review Authority (2022-06976-01). Written informed consent was waived by the authority for this register study.

Data extraction resulted in 714 included patients (mean age 63.2 ± 11.1 years, range 18–91 years, 42.2% female) who in total had completed 937 pairs of the P-mJOA and the EMS. An additional 80 (7.9%) potential pairs in 37 (4.9%) patients had to be excluded as data were incomplete for either one or both scales.

The mean P-mJOA score was 13.9 ± 3.0 (range 1–18), and the mean EMS was 14.5 ± 2.7 (range 5–18). The mean difference between the P-mJOA and EMS was –0.61 (95% CI –0.72 to –0.51; p < 0.001), and the mean differences between the scales at the three measured timepoints (baseline, 1-year follow-up, and 2-year follow-up) were consistent with the total mean difference (Table 4).

Figure 1 shows a scatter plot comparing P-mJOA versus EMS scores. A Spearman’s ρ of 0.84 (95% CI 0.82–0.86; p < 0.001) demonstrated a strong monotonic association between the P-mJOA and the EMS. Intra-rater agreement measured with the ICC between the P-mJOA and the EMS was 0.83 (95% CI 0.76–0.87; p < 0.001).

The distribution of severity levels between the scales is given in Table 5. Severity levels in the population were significantly higher using the P-mJOA compared with the EMS (Pearson Chi-square; p < 0.001); however, if disease severity was classified as severe with the EMS (n = 32), it also classified as severe with P-mJOA. Further, moderate disease severity classified with the EMS (n = 171) was only four times classified as mild with P-mJOA. There was fair agreement between the P-mJOA and the EMS severity levels with both unweighted kappa (κ = 0.22; p < 0.001) and weighted kappa (κ = 0.34; 95% CI 0.30–0.37; p < 0.001).

Mean subscores for each evaluated item grouped by severity levels for the P-mJOA and the EMS are presented in Table 6. The largest mean difference in subscore points between measurement points classified as severe versus mild was seen in the lower extremity-function item for the P-mJOA (3.1 points) and in the gait-function item for the EMS (3.0 points).

In the supplementary material, a conversion table to compare the scales through a proportional division of the 14 steps (range 5–18) of myelopathy measured with the EMS into the 19 steps of the P-mJOA (range 0–18) is given (Supplementary Table 2). The mean score of this converted version of the EMS (cEMS) was 13.0 ± 3.9. The mean difference between the P-mJOA and the cEMS was at 0.85 (95% CI 0.72–0.98; p < 0.001) slightly higher than that between the P-mJOA and the EMS. The number of equal values between the P-mJOA and the cEMS was 224 (23.4%). A Spearman’s ρ of 0.86 (p < 0.001) demonstrated a strong monotonic association between the P-mJOA and the cEMS. Intra-rater agreement with the ICC between the P-mJOA and the cEMS was 0.80 (95% CI 0.71–0.83; p < 0.001). By severity level, there was moderate agreement with the unweighted κ statistic (κ = 0.53; p < 0.001) and substantial agreement with the weighted κ statistic (κ = 0.64; 95% CI 0.60–0.68; p < 0.001).

This is the first study to compare the newly developed P-mJOA with the established EMS, and our comparison shows that the scales are highly correlated and have high intra-rater agreement in terms of total scores, but only show a fair agreement when assessed by severity level. Naturally, the comparison of the P-mJOA and the EMS was complicated by the difference in number of items, item content, and score intervals between the scales. Despite this, mean scores only differed slightly more than 0.5 points, which is less than the previously reported minimum clinically important difference of the mJOA estimated to between 1 and 2 points depending on severity [19].

When comparing the scales by their classification of patients into three previously used severity levels for each scale [9, 13, 17], the P-mJOA scale classified significantly more patients as severe or moderate. This difference might be important as surgical decision making in accordance with current international guidelines suggests surgical intervention for this patient group, whereas initial non-operative treatment can be considered for patients with mild myelopathy [15]. However, our study also demonstrated that a large proportion of patients classified as having mild myelopathy (41 and 77% for P-mJOA and EMS, respectively) still undergo surgical treatment. As our study population represents a national setting along with previous studies presenting comparable mean preoperative scores [9, 13, 20‐22], these results are likely to reflect a common management strategy for patients with mild myelopathy. This is also in agreement with several studies showing that less severe baseline myelopathy is a predictor of improved functional outcome [17, 23‐25]. Another important aspect is that neither scale is particularly influenced by the degree of pain that the patient experiences, which in clinical practice might be a determining indication for surgery due to radiculopathy present in combination with myelopathy [15, 26‐28], nor the main symptom in other conditions that may be associated with myelopathy [29].

The number of patients classified as severe with the EMS was unexpectedly low at 3.4% but can be explained by the narrow range (EMS 5–8) used to define severe myelopathy in previous literature [6, 9, 17]. In contrast, the P-mJOA uses 12 steps (range 0–11) to define severe myelopathy. In theory, the four steps of severe myelopathy using the EMS could have corresponded to the 12 steps of severe myelopathy using P-mJOA, but this study clearly contradicts that. For both scales, the largest differences at item level between patients classified as severe versus mild were seen for the corresponding lower extremity- and gait-function items. This was somewhat expected as these items have the largest point ranges in both scales. Still, this finding suggests that the degree of walking disability has the largest impact on the severity grading, and it could also be indicative of a more advanced disease progression. The importance of walking disability is further supported by the widespread use of the Nurick scale that only reports walking disability [2, 30]. There was, however, no obvious explanation for the much lower proportion of patients classified as severe with the EMS, which suggests that the interval to classify a patient as severe with the EMS is too narrow.

The proportional conversion table, converting the EMS into the cEMS, proved to be of limited value. Although the kappa statistics increased from fair agreement to substantial agreement with the cEMS, the number of patients classified as severe was instead overestimated when the 0–11 range of P-mJOA was divided into a proportionally equivalent 0–13 EMS range.

Our results indicate that the original EMS has a low sensitivity for detecting severe myelopathy and lacks the ability to subcategorize severe myelopathy compared with the P-mJOA. This limitation could partially be reduced by increasing the upper interval limit of severe myelopathy to an extent where an equal number of patients are classified as having severe myelopathy, but to fully account for the difference, the EMS would need more steps in the lower end of the scale.

Several limitations with this study are related to the comparison of structurally different scales. Although the difference in ranges and intervals might be addressed through a proportional conversion, the difference in content within the items, further complicated by an additional item in the EMS, cannot easily be adjusted for. This also means that the similarity in mean scores and correlations between the scales should be interpreted with great caution. Strengths and limitations of this study might relate to the register-based study design, consent bias, and potential selection bias related to patients not providing complete PROMs. However, less than 10% of the timepoints from the data extraction did not have complete P-mJOA and EMS data and had to be excluded. Selection bias might further occur related to compliance with completing all PROMs, which might be individually influenced positively or negatively by disease severity, motivation, or ability. Completion rates at the different timepoints were, however, evenly distributed, and the large sample size with high nationwide representation and non-exclusive selection criteria employed in this study should at least partially counteract these biases and yield generalizable findings. A register-based study design further always introduces a risk of collection and confounding biases; however, the latter should be limited as we only used paired observations. Finally, comparative subscore analyses of the different items between the scales would have been interesting but were not deemed feasible due to the structural differences.

In this study, the P-mJOA and the EMS had similar mean scores, and intra-rater agreement measured with the ICC was high. However, agreement by severity level measured with kappa statistics only demonstrated fair agreement. Compared with the P-mJOA, the EMS has a low sensitivity for detecting severe myelopathy but shows an increasing agreement with the P-mJOA for milder disease severity. For improved comparison between the P-mJOA and the EMS, a larger interval to define severe myelopathy with the EMS is suggested.