To investigate the external validation and scalability of four predictive models regarding new vertebral fractures following percutaneous vertebroplasty.
Methods
Utilizing retrospective data acquired from two centers, compute the area under the curve (AUC), calibration curve, and Kaplan–Meier plot to assess the model’s discrimination and calibration.
Results
In the external validation of Zhong et al.’s 2015 predictive model for the probability of new fractures post-vertebroplasty, the AUC for re-fracture at 1, 2, and 3 years postoperatively was 0.570, 0.617, and 0.664, respectively. The AUC for Zhong et al.’s 2016 predictive model for the probability of new fractures in neighboring vertebrae was 0.738. Kaplan–Meier plot results for both models indicated a significantly lower incidence of re-fracture in low-risk patients compared to high-risk patients. Li et al.’s 2021 model had an AUC of 0.518, and its calibration curve suggested an overestimation of the probability of new fractures. Li et al.’s 2022 model had an AUC of 0.556, and its calibration curve suggested an underestimation of the probability of new fractures.
Conclusion
The external validation of four models demonstrated that the predictive model proposed by Zhong et al. in 2016 exhibited superior external generalization capabilities.
Hinweise
Xiangheng Dai and Weibin Liao have contributed equally.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
Osteoporosis is a global affliction characterized by a decline in bone mass and alterations in bone structure, resulting in heightened bone fragility and an elevated risk of fractures [1]. Common fracture sites include the spine, hip, and forearm [2]. Osteoporotic vertebral compression fractures (OVCF) may manifest with symptoms such as back pain and kyphotic deformity of the spine. Percutaneous vertebroplasty (PVP) can effectively alleviate back pain and restore partial vertebral height in OVCF [3]. However, it may also give rise to common complications, including new-onset osteoporotic vertebral compression fractures [4]. This often necessitates re-treatment, amplifying patient discomfort and imposing a financial burden on families [5]. Consequently, it becomes particularly crucial to identify risk factors for re-fracture post-PVP and to implement appropriate interventions based on these risk factors.
Clinical predictive models are widely used worldwide to analyze risk factors to assist clinicians in developing individualized diagnoses, treatments, and follow-up plans [6, 7]. Numerous studies have evaluated the probability of new fractures post-PVP using clinical prediction models. In 2015, Zhong et al. [8] developed a predictive model for evaluating new fractures post-PVP by retrospectively analyzing 397 patients (241 in the training set and 156 in the validation set). The model incorporated risk factors such as lower CT values, previous presence of OVCF, cement leakage, treatment of multiple vertebrae, and distribution of cement margins, achieving an AUC value of 0.757. Continuing this line of research, Zhong et al. developed another predictive model in 2016 to assess new fractures in adjacent vertebrae after PVP. The retrospective analysis included 421 patients (256 in the training set and 165 in the validation set), with risk factors such as the previous presence of OVCF and cement leakage. The model exhibited an AUC of 0.78 in the training set and 0.72 in the validation set [9]. Subsequently, in 2021, Li et al. developed a model by retrospectively analyzing 562 patients (403 in the training set and 159 in the validation set). The risk factors included advanced age, bone cement volume (< 4.0 ml or > 4.5 ml), unregulated anti-osteoporotic treatment, cement leakage, poor cement distribution, and cement not contacting the endplate. The AUC value of the model was 0.882 in the training set and 0.869 in the validation set [10]. Finally, in 2022, Li et al. [11] developed a model by retrospectively analyzing 385 patients, incorporating risk factors such as higher BMI, lower BMD, multiple vertebral fractures, previous long-term steroid use, and unregulated anti-osteoporosis treatment. The model achieved an AUC value of 0.726.
Anzeige
Although there are many risk prediction models for predicting new OVCF after PVP, each model uses different parameters for predicting new vertebral fractures, which in turn makes it more difficult for clinicians to decide which model is more useful, and there is still a lack of externally validated data on the above models. External validation involves testing models using data from sources independent of the cohort used in model development. It is crucial to evaluate a predictive model’s ability to provide accurate predictions for individuals from comparable source populations (reproducibility) or diverse clinical settings (portability). In general, predictive models tend to perform better in model-derived cohorts than in other cohorts for different reasons, including differences in case mix and different outcome rates.
This study aimed to externally validate four previously developed predictive models, using data from two medical centers in northern Guangdong, China. By externally validating the above four models, spine surgeons can gain the following information that is instructive for clinical practice: (1) Model applicability: external validation helps to assess the applicability of these models in different patient populations. If the models perform well on different datasets, they are more likely to have a wide range of applications in clinical practice. (2) Accuracy of risk prediction: By externally validating the predictive accuracy of the models, physicians can understand how they perform in real-world clinical settings. If models can accurately predict a patient’s risk, they can help physicians better plan treatment and preventive measures. (3) Individualised treatment decisions: External validation can help you determine which patients benefit from a particular model. Based on the model’s predictions, physicians can develop a more personalised treatment plan for each patient to minimise the risk of complications. (4) Potential for clinical application of models: If these models perform well in external validation, they can become useful tools in clinical practice. Physicians can use these models to aid decision-making and improve patient outcomes. In conclusion, external validation is crucial to assess the effectiveness of these models in practical applications. If the models perform well on different datasets, they can be a powerful support tool for clinical decision making.
Methods
Clinical data and inclusion and exclusion criteria
A retrospective analysis was conducted on patients with OVCF who underwent PVP between April 2016 and May 2021 at Yuebei People’s Hospital and from December 2020 to March 2022 at The First People’s Hospital of Shaoguan, Guangdong Medical University. Approval for the study was obtained from the ethics committees of both hospitals.
Inclusion criteria were as follows: (1) Conformance to the World Health Organization (WHO) diagnostic criteria for osteoporosis, characterized by a bone mineral density (BMD) below the mean peak BMD of a healthy adult by a standard deviation of 2.5 or more [12]; presence of pain or localized pressure consistent with imaging findings, with preoperative spinal X-ray and magnetic resonance imaging (MRI) confirming T1-L5 new fractures; (2) Availability of comprehensive basic patient information, including preoperative and postoperative imaging (X-ray, CT, MRI, dual-energy X-ray absorptiometry bone density); (3) Undergoing PVP through a unilateral or bilateral approach; (4) No history of additional traumatic injuries post-operation; and (5) A follow-up period of at least 2 years.
Anzeige
Exclusion criteria: (1) non-OVCF or compression fractures secondary to other factors, such as pathological fractures caused by metastatic tumours or haemangiomas; (2) no PVP treatment; (3) incomplete patient information or imaging data.
PVP
The patient was placed in prone position and vital signs were monitored. The C-arm X-ray machine was adjusted preoperatively, and the unilateral/bilateral puncture point was located at 3–4 cm adjacent to the spinous process, skin disinfection was routinely performed, and anesthesia was infiltrated to the periosteum using 1% lidocaine. The needle was inserted along the puncture point and when the tip of the needle was in the optimal position, the cement injection was performed after pulling out the occipital core and observing no active bleeding. The ratio of bone cement powder was 1:1, and C-arm X-rays were used to monitor whether the bone cement leaked or exceeded the vertebral midline during the injection process. After the bone cement injection was completed, the needle was withdrawn, and the wound was compressed for 3 min and then bandaged with a sterile dressing. All patients were placed on mandatory bed rest for several hours after the procedure.
Diagnostic criteria for new osteoporotic vertebral compression fractures
The patient’s thoracolumbar pain resolved after surgery and reappeared during follow-up. Compared with postoperative X-ray imaging, the anterior vertebral body height (AVH) had decreased by more than 1 mm and the cobb angle had increased by > 3° at the last follow-up; and MRI examination suggested low signal intensity on T1-weighted images and high signal intensity on T2-weighted images [11, 13]. Other associated diseases, such as tumors or infections, were also excluded. Asymptomatic incidental fractures were defined as changes in vertebral height on follow-up radiographs without acute exacerbation of pain [14], and such fractures were not considered in this study due to the ambiguity of the analysis.
Data collection
General information of the patient: gender, age, date of admission, duration of follow-up, BMI, previous OVCF, steroid use, number of treated vertebral, thoracolumbar vertebral fractures, prescription of anti-osteoporosis, new postoperative vertebral fracture, and whether the new fracture was adjacent to the operated vertebra. Surgery-related data: length of hospitalisation-operation, surgery time, bone cement volume. Radiological data: bone density of lumbar vertebrae was measured by dual-energy X-ray absorptiometry for L1-L4 vertebrae and the corresponding t-values were calculated. CT values in Hounsfield Units (HU): Using the picture archiving and communication system (PACS), measure the CT values below the upper endplate, in the middle of the vertebral body, and above the lower endplate on the CT cross-sectional slices. Cement leakage: Cement leakage is determined when the cement exceeds the upper or lower endplates of the vertebral body and enters the disc on a postoperative X-ray. Endplate contact: Endplate contact is determined when the injected bone cement is shown to be in contact with the endplate on the postoperative X-ray. Cement distribution pattern: Postoperative radiographs assessed type C as dense solid block cement filling the vertebral body and type T as well-dispersed spongy cement. Kummell’s sign: Preoperative CT or X-ray images show radiolucent shadows located in the center or adjacent to the vertebral body endplates; in MRI they usually show low signal intensity on T1-weighted images and high or low signal on T2-weighted images [15, 16]. Reduction rate: The vertebral compression ratio (CR) is the ratio of the height of the fractured vertebrae to the average height of the upper and lower vertebrae, and the reduction rate is preoperative CR minus postoperative CR. Reduction angle: preoperative Cobb angle minus postoperative Cobb angle; Cobb angle is the angle formed by the upper and lower endplates of the fractured vertebral body, with a positive value for anterior convexity and a negative value for posterior convexity.
Radiological assessment
All radiological parameters underwent dual measurements by two spinal surgeons with over 10 years of experience each, independently conducted to mitigate both intra- and inter-observer bias. In our investigation, the intra- and inter-observer correlation coefficient (ICC) for all radiological parameters demonstrated excellence, surpassing 0.83. To address any notable discrepancies, a third experienced evaluator was engaged to provide a deciding vote. Moreover, radiological reports were consulted to aid in the decision-making process when necessary. All radiographic measurements were digitally verified using the picture archiving and communication system (PACS) and its associated computer software within our department.
Data analysis
Continuous variables were described using means and standard deviations, while categorical variables were presented as frequencies; chi-square tests were used for categorical variables, and t-tests were used for continuous variables. Univariate analysis was used to screen for strong risk factors from the 21 potential factors identified. Following the methodologies of previous studies, we generated ROC curves and calculated AUC values to evaluate the discriminatory efficacy of the five clinical prediction models, contextualized to our regional data. For models built using the Cox formula, Kaplan–Meier curves were utilized to assess the model’s ability to distinguish between various risk factors for the occurrence of new vertebral fractures post-PVP. For models built using the logistic formula, calibration curves were plotted to elucidate the disparity between the predicted and actual probabilities of occurrence. All statistical analyses were performed using IBM SPSS Statistics version 25 and Prism software to plot the above ROC, AUC, DCA, etc. P < 0.05 was considered statistically significant.
Results
A total of 296 patients (with a male-to-female ratio of 61:235) met the inclusion criteria. Among them, 60 patients (20.27%) experienced new vertebral fractures post-surgery, with 37 (61.67%) developing new compression fractures of adjacent vertebrae (AVCF).
BMI, age, and bone mineral density values showed no statistically significant differences between patients treated at ShaoGuan First People’s Hospital and Yuebei People’s Hospital. However, there was a statistically significant difference in the amount of polymethylmethacrylate (PMMA) injected between the two hospitals. The average PMMA injection volume at ShaoGuan First People’s Hospital was 3.50 ± 1.02 ml, while at Yuebei People’s Hospital, it was 4.50 ± 1.62 ml, with a significant difference (P < 0.001) and a mean difference of 0.8 ± 0.2 ml. Furthermore, a comparison of the proportion of bilateral and unilateral vertebroplasty (VP) procedures revealed discrepancies between the hospitals. At ShaoGuan First People’s Hospital, only 22.4% (11/49) of the procedures were bilateral, whereas at Yuebei People’s Hospital, this proportion was 63.2% (156/247), indicating inconsistency in surgical approaches. Despite these differences, there were no statistically significant disparities in the rates of subsequent vertebral fractures between the two hospitals. The rate of subsequent fractures at ShaoGuan First People’s Hospital was 14.3% (7/49), while at Yuebei People’s Hospital, it was 21.5% (53/247), with an independent samples t-test yielding a P-value of 0.21.
Anzeige
Additionally, when analyzing all patients collectively, regardless of hospital, those who underwent bilateral VP procedures had a significantly higher PMMA injection volume compared to those who underwent unilateral procedures. Among the 167 patients who underwent bilateral procedures, the average PMMA injection volume was (5.00 ± 1.55) ml, whereas among the 129 patients who underwent unilateral procedures, it was (3.25 ± 1.16) ml, with a statistically significant difference (P < 0.0001). Furthermore, the choice of anesthesia method, whether local or general, did not significantly influence the amount of PMMA injected during VP procedures. Among the 260 cases performed under local anesthesia, the average PMMA injection volume was (4.00 ± 1.6) ml, while among the 36 cases performed under general anesthesia, it was (4.09 ± 1.37) ml, with a non-significant difference (P = 0.2458).
A comparison between the new vertebral fracture group and non-new vertebral fracture group revealed no statistically significant difference in age, gender, BMI, anti-osteoporosis treatment, thoracolumbar fractures, bone cement volume, reduction rate, reduction angle, and contact with endplate (P > 0. 05). Statistical significance was observed in lumbar BMD (P = 0. 028), previous OVCF (P < 0. 001), CT values (P = 0. 005), Kummell’s sign (P = 0. 003), bone cement leakage (P < 0. 001), and Cement distribution pattern (P = 0. 001), as shown in Table 1.
Table 1
Demographics and clinical characteristics of NVCFs group and Well-maintained group
NVCFs group (n = 60)
Well-maintained group (n = 236)
P values*
Age (years)
72.20 ± 8.209
72.56 ± 7.943
0.753
Sex (female, %)
50 (83.3%)
185 (78.4%)
0.398
BMI (kg/m2)
22.32 ± 3.811
22.41 ± 3.426
0.864
BMD T-score
− 3.96 ± 0.845
− 3.66 ± 0.979
0.028*
Preexisting fracture (yes, %)
35 (58.3%)
48 (20.3%)
< 0.001*
HU values
53.79 ± 18.818
62.89 ± 31.111
0.005*
Thoracolumbar (T11–L2) (yes, %)
39 (65.0%)
171 (72.5%)
0.256
Kummell’s sign (yes, %)
14 (23.3%)
22 (9.3%)
0.003*
Cement volume (ml)
4.16 ± 1.624
4.23 ± 1.562
0.773
Bone cement leakage (no, %)
30 (50.0%)
191 (80.9%)
< 0.001*
Cement distribution pattern (C type, %)
44 (73.3%)
119 (50.4%)
0.001*
Prescription of anti-osteoporosis (yes, %)
31 (51.7%)
127 (53.8%)
0.766
Reduction rate (%)
12.63 ± 13.055
10.20 ± 9.703
0.181
Reduction angle (°)
17.55 ± 26.874
19.60 ± 26.282
0.593
*Indicates statistically significant data
Univariate analysis comparing patients with and without new vertebral fractures revealed no statistical significance for gender, age, BMI, steroid use, hospital-procedure time, surgery time, bone cement volume, number of treated vertebral, prescription of anti-osteoporosis, thoracolumbar vertebral fractures, reduction rate, reduction angle, and contact with endplate (P > 0.05). On the other hand, the univariate analysis indicated a significant association between bone cement leakage (P < 0.001), previous osteoporotic vertebral compression fractures (OVCF) (P < 0.001), and Kummell’s sign (P = 0.004) with an increased risk of new vertebral fractures. Conversely, bone cement pattern (P = 0.002), bone mineral density (BMD) (P = 0.029), and CT values (P = 0.032) were significantly associated with a decreased risk of new vertebral fractures. Multifactorial logistic regression analysis further confirmed the significant association of previous OVCF (P < 0.001), bone cement leakage (P < 0.001), and Kummell’s sign (P = 0.006) with an increased risk of new vertebral fractures, as detailed in Table 2.
Table 2
Univariate and multivariate Logistic regression analysis of new vertebral compression fractures after percutaneous vertebroplasty in patients with osteoporosis
Characteristic
Univariate OR (95% CI)
B
P value*
Multivariate OR (95% CI)
B
P value*
Gender (male, female)
1.378 (0.653–2.907)
0.321
0.399
Age (year)
0.994 (0.960–1.030)
− 0.006
0.752
Age (year)(55–74, ≥ 75)
0.939 (0.524–1.682)
− 0.063
0.832
Age (year) (< 60)
1
Age (year)(60–70)
0.346 (0.091–1.313)
− 1.062
0.119
Age (year)(70–80)
0.570 (0.156–2.088)
− 0.562
0.396
Age (year) (> 80)
0.324 (0.080–1.316)
− 1.127
0.115
BMI
0.993 (0.915–1.077)
− 0.007
0.863
BMD T-score
0.707 (0.517–0.965)
− 0.347
0.029*
0.731 (0.502–1.066)
− 0.313
0.103
Preexisting fracture (no, yes)
5.483 (3.000–10.024)
1.702
< 0.001*
4.810 (2.479–9.333)
1.571
< 0.001*
Steroid use (no, yes)
0.547 (0.121–2.474)
0.321
0.433
Hospitalization to surgery (days)
0.902 (0.801–1.016)
− 0.103
0.089
Surgery time (min)
0.993 (0.982–1.004)
− 0.007
0.215
Cement volume (ml)
0.974 (0.812–1.167)
− 0.027
0.772
Cement volume (< 5, ≥ 5 ml)
1.067 (0.592–1.924)
0.065
0.829
Number of treated vertebra
1
(1)
(2)
1.305 (0.632–2.695)
0.266
0.472
(≥ 3)
2.416 (0.771–7.572)
0.882
0.130
Location (TL junction, non-TL junction)
0.706 (0.386–1.289)
− 0.348
0.257
CT values (HU)
0.988 (0.978–0.999)
− 0.012
0.032*
0.997 (0.985–1.010)
− 0.003
0.691
< 40HU
1
40-63HU
1.385 (0.655–2.928)
0.325
0.394
63-83HU
1.153 (0.511–2.598)
0.142
0.732
≥ 83HU
0.255 (0.068–0.952)
− 1.365
0.042*
Bone cement leakage (no, yes)
4.244 (2.326–7.744)
1.446
< 0.001*
3.537 (1.795–6.973)
1.263
< 0.001*
Distribution of the bone cement (C type, T type)
0.370 (0.198–0.692)
− 0.995
0.002*
0.538 (0.267–1.083)
− 0.620
0.082
Endplate contact (no, yes)
1.023 (0.580–1.806)
0.023
0.937
Kummell’s sign (without, with)
2.960 (1.410–6.217)
1.085
0.004*
3.246 (1.393–7.567)
1.177
0.006*
Prescription of anti-osteoporosis (no, yes)
0.917 (0.520–1.618)
− 0.086
0.766
Reduction rate (%)
1.021 (0.995–1.047)
0.021
0.112
Reduction angle (°)
0.997 (0.986–1.008)
− 0.003
0.592
Restoration of the middle height in the fractured vertebra (< 7, ≥ C7%)
1.049 (0.594–1.854)
0.048
0.869
*Indicates statistically significant data
Univariate analysis comparing patients with and without new AVCF indicated no statistical significance for gender, age, bone cement volume, number of treated vertebral, prescription of anti-osteoporosis, thoracolumbar vertebral fractures, reduction rate, and CT values (P > 0. 05). Conversely, univariate analysis revealed a significant association between Kummell’s sign (P = 0.019), bone cement leakage (P < 0.001), and previous osteoporotic vertebral compression fractures (OVCF) (P < 0.001) with an increased risk of developing new AVCF. Multifactorial logistic regression analysis confirmed that previous OVCF (P = 0.001), bone cement leakage (P < 0.001), and Kummell’s sign (P = 0.034) were significantly associated with an increased risk of new AVCF development, as detailed in Table 3.
Table 3
Univariate and multivariate Logistic regression analysis of new AVCF after percutaneous vertebroplasty in patients with osteoporosis
Characteristic
Univariate OR (95% CI)
B
P value*
Multivariate OR (95% CI)
B
P value*
Gender (male, female)
1.393 (0.553–3.508)
0.331
0.482
Age (year) (55–74, ≥ 75)
1.238 (0.615–2.492)
0.213
0.551
CT values (HU)
1
< 40HU
1.474 (0.601–3.616)
0.388
0.397
40-63HU
1.080 (0.399–2.920)
0.077
0.880
63-83HU
0.145 (0.018–1.204)
− 1.928
0.074
Number of treated vertebra
1
(1)
(2)
2.044 (0.910–4.589)
0.715
0.083
(≥ 3)
2.341 (0.610–8.983)
0.851
0.215
Location (TL junction, non-TL junction)
1.316 (0.593–2.920)
0.275
0.499
Bone cement leakage (no, yes)
4.983 (2.435–10.197)
1.606
< 0.001*
4.236 (2.010–8.924)
1.444
< 0.001*
Preexisting fracture (no, yes)
4.170 (2.050–8.485)
1.428
< 0.001*
3.859 (1.823–8.172)
1.351
< 0.001*
Cement volume (< 5, ≥ 5 ml)
1.280 (0.633–2.589)
0.247
0.492
Kummell’s sign (without, with)
2.762 (1.180–6.463)
1.016
0.019*
2.725 (1.076–6.901)
1.003
0.034*
Prescription of anti-osteoporosis (no, yes)
0.911 (0.457–1.816)
− 0.093
0.792
Restoration of the middle height in the fractured vertebra (< 7, ≥ C7%)
1.133 (0.566–2.270)
0.125
0.724
*Indicates statistically significant data
Anzeige
Predictive Performance Analysis of Zhong et al. in 2015.
To understand the discrepancies between our data and the modeling data provided by Zhong et al. in 2015, we conducted a single-sample t-test based on the data provided in their article. Our analysis revealed slight decreases in age and CT value compared to the modeling data. However, there were no differences in gender proportions (Fig. 1a).
Fig. 1
Predictive Performance Analysis of Zhong et al. in 2015. a Single-sample t-test revealed discrepancies in age and CT value between our data and Zhong et al. in 2015’s modeling data. No differences were found in gender proportions. *P < 0.05. b Kaplan–Meier analysis demonstrated significant predictive power of the model, dividing patients into high-risk (> 8.5) and low-risk (≤ 8.5) groups for new vertebral fractures. High-risk group: approximately 50% free from new fractures after 3 years; low-risk group: proportion exceeding 80%. c ROC curve analysis indicated better predictive ability for long-term occurrence of new vertebral fractures with 1-year AUC of 0.570, 2-year AUC of 0.617, and 3-year AUC of 0.664
×
In the Kaplan–Meier (KM) analysis, it is evident that the division of patients into high-risk (> 8.5) and low-risk (≤ 8.5) groups based on the model demonstrates statistical significance in predicting subsequent vertebral fractures. The high-risk group exhibits a proportion of patients free from new fractures after 3 years at approximately 50%, while the low-risk group shows a proportion exceeding 80% (Fig. 1b).
However, in the overall receiver operating characteristic (ROC) curve analysis, we observed that the 1-year AUC was 0.570, the 2-year AUC was 0.617, and the 3-year AUC was 0.664 (Fig. 1c). These results indicate a better predictive ability for long-term occurrence of new vertebral fractures.
Anzeige
Predictive Performance Analysis of Zhong et al. in 2016.
To further assess the disparities between our data and the modeling data provided by Zhong et al. in 2016, we conducted a single-sample t-test using the data provided in their article. Our analysis revealed a slight decrease in age compared to the modeling data, while CT value and gender proportions showed no differences (Fig. 2a).
Fig. 2
Predictive Performance Analysis of Zhong et al. in 2016. a Single-sample t-test revealed a slight decrease in age compared to Zhong et al. in 2016’s modeling data, while CT value and gender proportions showed no differences. *P < 0.05. b Kaplan–Meier analysis demonstrates significant predictive power in stratifying patients into high-risk, intermediate-risk, and low-risk groups for adjacent new vertebral fractures. c Receiver operating characteristic (ROC) curve analysis yielded an AUC of 0.738, indicating robust predictive ability for adjacent new vertebral fractures
×
In the KM analysis, it is evident that the division of patients into high-risk (6 points), intermediate-risk (2 and 4 points), and low-risk (0 points) groups based on the model demonstrates statistical significance in predicting subsequent adjacent vertebral fractures. The proportion of patients free from adjacent new fractures was over 93% in the low-risk group, over 81% in the intermediate-risk group, and lower than 50% in the high-risk group (Fig. 2b).
Moreover, in the overall receiver operating characteristic (ROC) curve analysis, we observed an AUC of 0.738, indicating a strong predictive ability for adjacent new vertebral fractures (Fig. 2c).
Predictive Performance Analysis of Li et al. in 2021.
To assess the disparities between our data and the modeling data provided by Li et al. in 2021, we conducted a single-sample t-test using the data provided in their article. Our analysis revealed slight differences in various parameters between our and the modeling data. Specifically, in the Control group (no new vertebral fracture), CT value, BMI, available anti-osteoporotic treatment, bone cement leakage, contact between bone cement and endplate, and bone cement dispersion showed minor variances compared to the modeling data, while no differences were observed in age and gender proportions. Similarly, in the NVCF group (new vertebral fracture group), available anti-osteoporotic treatment, age groups of 60–70 years and over 80 years, and CT value exhibited slight differences compared to the modeling data. However, BMI, age groups under 60 and between 70 and 80 years, bone cement leakage, contact between bone cement and endplate, bone cement dispersion, and gender proportions showed no disparities (Fig. 3a).
Fig. 3
Predictive Performance Analysis of Li et al. in 2021. a Comparison of our data with Li et al. in 2021’s modeling data revealed minor differences in parameters for both Control and NVCF groups. *P < 0.05. b The ROC curve analysis of Li et al.in 2021’s predictive model resulted in an AUC of 0.518. c The calibration curve demonstrated that the model tended to overestimate the actual number of new fractures
×
We observed that the predictive model developed by Li et al. in 2021 exhibited a calculated AUC of 0.518 (Fig. 3b). Additionally, upon examination of the calibration curve, it was evident that the model predicted a high probability and tended to overestimate the actual number of new fractures (Fig. 3c).
Predictive Performance Analysis of Li et al. in 2022.
To assess the disparities between our data and the modeling data provided by Li et al. in 2022, we conducted a single-sample t-test using the data provided in their article. Our analysis revealed slight differences in various parameters between our Control group (no new vertebral fracture) and the modeling data. Specifically, in the Control group, variables such as steroid use, multiple vertebral fracture, anti-osteoporosis therapy, surgery time, bone mineral density (BMD), age, and BMI showed minor variances compared to the modeling data, while no differences were observed in age, gender proportions, bone cement leakage, and injection volume. Similarly, in the NVCF group (new vertebral fracture group), variables such as steroid use, anti-osteoporosis therapy, bone cement leakage, hospitalization to surgery duration, BMD, and age exhibited slight differences compared to the modeling data. However, surgery time, BMI, injection volume, and gender proportions showed no disparities (Fig. 4a).
Fig. 4
Predictive Performance of Li et al. in 2022. a Comparison of our data with Li et al. in 2022’s modeling data revealed minor differences in parameters for both Control and NVCF groups. *P < 0.05. b ROC curves were plotted to evaluate the predictive model developed by Li et al. in 2022, resulting in a model AUC of 0.556. c The calibration curves indicated that the model tended to predict a low probability of new fractures and consistently underestimated the actual number of occurrences
×
The ROC curves were constructed to evaluate the predictive model developed by Li et al. in 2022, resulting in a model AUC of 0.556 (Fig. 4b). Additionally, the calibration curves indicated that the model tended to predict a low probability of new fractures and consistently underestimated the actual number of occurrences (Fig. 4c).
Discussion
We conducted a study across two medical centers in northern Guangdong, validating four predictive models for forecasting new vertebral fractures post-PVP in 296 patients. The model by Zhong et al. in 2016 outperformed others with an AUC of 0.738. K-M plots for Zhong et al.’s (2015, 2016) models demonstrated a positive risk score-fracture likelihood correlation. However, Li et al.’s model in 2022 consistently underestimated, while Li et al.’s (2021) overestimated probabilities. These findings highlight areas for improving accuracy and calibration, guiding future optimization for precise application in fracture risk prediction.
The K-M plot from Zhong et al.’s model in 2015 indicates increased risk of new OVCF at 1, 2, and 3 years post-PVP for patients with scores exceeding 8.5, facilitating effective risk stratification. However, AUC values for predicting new OVCF at these time points are < 0.7, suggesting inadequate differentiation. We uncovered slight decrements in age and CT value compared to their modeled data. Our univariate analysis contradicts their finding, showing no statistical significance between treated vertebrae number and new fractures, consistent with previous studies [17]. This discrepancy warrants further investigation, as some studies [18, 19] suggest a potential link between more treated vertebrae and increased fracture risk. Conversely, our study identifies preoperative Kummell’s sign as significant for new fractures, contrasting the analyses conducted by Zhong et al. in both 2015 and 2016. Kummell’s sign’s association with bone nonunion and dynamic instability underscores its relevance [20‐22] However, conflicting findings exist [23], necessitating ongoing debate on Kummell’s sign’s impact post-PVP.
The 2016 predictive model by Zhong et al. effectively stratified risk for new AVCF, supported by K-M plots and a high AUC of 0.738, indicating strong differentiation in our dataset. Our analysis indicated a minor decline in age when compared to the modeled data, with no discernible differences observed in CT value or gender proportions. Univariate and multivariate analyses confirmed cement leakage and previous OVCF as significant predictors, consistent with their findings. Cement leakage, a consistent risk factor in multiple studies [14, 24], concentrates stress and contributes to intervertebral disc degeneration [25], altering biomechanics and increasing AVCF risk. Similarly, prior OVCF emerged as an independent risk factor, aligning with existing literature highlighting its role in adjacent fractures [19, 26, 27]. However, they did not provide the original model formula, thus precluding the analysis of calibration curves, and lacked validation results pertaining to clinical benefits. Therefore, further prospective data research is needed for the clinical application and generalization of our findings.
External validation of Li et al.’s model in 2021 revealed an AUC value of 0.518, and the calibration curve indicating overestimation of new OVCF probability. Univariate analysis identified cement leakage and distribution patterns as significant, while gender and BMI were not, consistent with Li et al. (2021). However, age, cement volume, endplate contact, reduction angle, anti-osteoporotic prescription, and steroid use showed no significance. Indeed, the disparities between our dataset and theirs may have contributed to these limitations. However, previous OVCF was significant [19, 26, 27], contrary to Li et al. (2021). Literature presents conflicting views on bone cement volume’s effect [24, 28] with variability in vertebral body volume emphasizing the need for consistent-level studies. Increased cement injection may elevate leakage risk [29], an independent factor for new OVCF. Uncemented endplate contact’s role remains controversial [15, 30, 31], as analyses showed no significance, consistent with prior studies.
The external validation by Li et al. (2022) revealed an AUC of 0.556 for their predictive model, with the calibration curve indicating underestimation of new OVCF probability. Univariate analysis identified BMD as significant, consistent with Li et al. (2022), but BMI, anti-osteoporotic prescription, steroid use, and multiple OVCFs did not show significance. Multifactorial analysis similarly found no significance for BMD and CT values, contrary to Li et al. (2022). Conflicting views on BMI’s role as a risk factor persist [32‐34], while BMD values below −2.5 signify osteoporosis [35], yet alone insufficient for independent risk classification [36]. Anti-osteoporotic therapy inhibits bone resorption [28], but patient compliance may impact its efficacy. Steroid use’s role remains contentious [37, 38]. Tailored postoperative anti-osteoporosis plans are crucial, considering individual patient circumstances [29].
During the external validation of the four prediction models, we noticed a noteworthy trend: models with more incorporated risk factors didn’t necessarily excel in external expansion. Notably, Zhong et al.’s model in 2016, incorporating only two factors, demonstrated the most effective expansion capability. This observation aligns with the goal of enhancing clinical applicability while avoiding potential overfitting issues. However, none of the models provided raw equations, hindering the accurate plotting of DCA curves. Future articles should include original equations to enable a thorough analysis of predictive effects and clinical benefits. This approach would improve model assessment, aiding clinical practice and informing future decisions.
This study is subject to certain limitations, most notably the absence of validation using prospective data for the predictive models. The model-building process itself did not incorporate prospective data, and while retrospective validation offers valuable insights, it cannot replace the gold standard of prospective validation. Nevertheless, these findings suggest potential avenues for further research. While we have garnered initial insights into the real-world efficacy and applicability of these models across various patient populations, a more comprehensive assessment is warranted. Our objective is to gather prospective data to ensure robust validation and facilitate the application of these models in clinical settings. Despite its limitations, this study lays a foundation for future exploration in this area.
Conclusions
By applying data from our region to externally validate four clinical prediction models for new OVCF post-PVP, we conclude that the prediction model developed by Zhong et al. in 2016 performed better in terms of external expansion. In contrast, the efficacy of other models may necessitate additional external validation and refinement, leveraging data from a broader spectrum of medical centers. This result emphasizes the possibility that model performance varies across regions or populations. To more fully and accurately assess the validity of different prediction models, we encourage more research centers to participate in the external validation of clinical prediction models. This collaborative approach aims to identify more reliable and universally applicable clinical prediction models, enhancing their utility in real-world medical settings. Through multi-center validation, we can gain deeper insights into the generalization capabilities of these models, thereby establishing a more dependable foundation for clinical decision-making. This concerted effort not only enhances the existing prediction models but also contributes significantly to the ongoing advancement of the clinical prediction field.
The authors have no competing interests to declare that are relevant to the content of this article.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Mit e.Med Orthopädie & Unfallchirurgie erhalten Sie Zugang zu CME-Fortbildungen der Fachgebiete, den Premium-Inhalten der dazugehörigen Fachzeitschriften, inklusive einer gedruckten Zeitschrift Ihrer Wahl.
Der Einsatz von Operationsrobotern für den Einbau von Totalendoprothesen des Kniegelenks hat die Präzision der Eingriffe erhöht. Für die postoperative Zufriedenheit der Patienten scheint das aber unerheblich zu sein, wie eine Studie zeigt.
Mit dem Hebelzeichen-Test lässt sich offenbar recht zuverlässig feststellen, ob ein vorderes Kreuzband gerissen ist. In einer Metaanalyse war die Vorhersagekraft vor allem bei positivem Testergebnis hoch.
Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.
Bewegungs-, Dehnungs- und Entspannungsübungen im Wasser lindern die Beschwerden von Patientinnen mit Fibromyalgie besser als das Üben auf trockenem Land. Das geht aus einer spanisch-brasilianischen Vergleichsstudie hervor.
Update Orthopädie und Unfallchirurgie
Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.
