Congenital diaphragmatic hernia: automatic lung and liver MRI segmentation with nnU-Net, reproducibility of pyradiomics features, and a machine learning application for the classification of liver herniation

We enrolled 39 inborn patients, born between 01/01/2012 and 31/12/2020, with isolated CDH from singleton pregnancies, taken in charge at the Fetal Surgery Unit of the Fondazione IRCCS Ca’ Granda Ospedale Maggiore Policlinico (Milan, Italy) before the 30th week of gestation. The only exclusion criterion was a pre- or postnatal diagnosis of non-isolated CDH.

A retrospective data collection of clinical and radiological variables from newborns’ and mothers’ medical records was performed for eligible patients (Astraia, Astraia Software GmbH, Ismaning, Germany; NeoCare, GPI SpA, Trento, Italy). In addition, the native sequences from fetal MRI were collected, with separate acquisition for the lungs and liver.

The imaging software used was Synapse PACS and Synapse 3D (FUJIFILM Medical Systems Lexington, MA, US). Lung volumes were calculated on the T2 HASTE sequences, selecting the best image quality plane without motion-induced artifacts [22]. Liver volumes were calculated on T1 VIBE sequences [23]. A pediatric radiologist with 15 years of experience in fetal MRI performed the manual segmentation of lung and liver volumes. In each slice, left and right lung and liver areas were determined separately by tracing freehand regions of interest (ROIs), excluding the pulmonary hila and mediastinal structures. The areas were automatically added to obtain the entire organ volume, multiplied by the sum of slice thickness and intergap by the software.

The DICOM files were then anonymized, converted to the NIFTI format for easier manipulation, and fed to the segmentation pipeline.

A publicly available segmentation pipeline based on DL achieved automatic lung and liver MRI segmentation. The pipeline was the no-new-Net (nn-NET), a specialized DL framework for medical image segmentation [24]. The framework is based on the U-Net architecture, a popular convolutional neural network that is particularly effective for biomedical image segmentation. It was developed to address the challenge of designing neural network architectures well-suited for various medical imaging tasks without requiring manual configuration or architectural modifications for each new task. nnU-Net automatically adapts its architecture to the specific characteristics of the dataset. It analyzes the dataset and decides on the most appropriate network architecture, preprocessing steps, and training strategies. This includes decisions about the network depth, convolutional kernel sizes, and the number of feature maps. This automation reduces the need for manual tuning and expert knowledge, making high-quality segmentation accessible even to those who might not be specialists in deep learning or medical image analysis. This network can achieve good segmentation results even with datasets of limited size. The nnU-Net segmentation pipeline is organized in several steps: (1) dataset structuring to a format compatible with the software; (2) the extraction of a dataset “fingerprint” containing dataset-specific properties, used to build various 2D/3D configurations, among which the best is “3D cascade”; (3) model training and validation, which we performed in the default fivefold cross-validation scheme. The software automatically gives Sørensen–Dice and Jaccard coefficients for segmentation quality evaluation. We ran the pipeline on a Server Supermicro 2023US-TR4, 2 CPU AMD Rome 7282 16C/32 T 2.8G 64 MB, equipped with 256 GB RDIMM DDR4 RAM and GPU Nvidia Tesla V100 32 GB HBM2 PCIe 3.0 (property of INFN, the Italian National Institute for Nuclear Physics, branch of Lecce). A cross-validation fold of each configuration took about 1 full day of calculations.

After segmentation, several standard 3D radiomics features were calculated. Pyradiomics was chosen for feature calculation [25]. This software package is freely available and allows the computation of many variables both from the original images and after preprocessing by various filters (e.g., wavelets or LoG, Laplacian of Gaussian). It also allows automatic reslicing with a chosen interpolator. The computed features, a subset of those available in pyradiomics, and after removing some correlated ones, are listed in Table 1. For gray level co-occurrence matrix (GLCM) and neighborhood gray tone difference matrix (NGTDM) calculation, only pixel pairs separated by a distance of 1 pixel were considered.

The MR images were preliminarily resized to all have the same voxel size of 1 × 1 × 1 mm³; the sitkBSpline interpolator was used for this purpose.

In order to test if the features calculated from manually and automatically segmented ROIs had similar values, a Wilcoxon rank-sum test and tests based on the intraclass correlation coefficients (ICCs) were performed. ICC is a statistical measure ranging from 0 to 1, with values close to 1 representing stronger feature reproducibility in segmentations. McGraw and Wong [26] defined 10 forms of ICC. In this study, we calculated the interrater reliability by employing a two-way mixed effect, absolute agreement, single rater/measurement model considering the variation between two or more raters who evaluate the same group of subjects (Eq. 1) [27]:where MS_R is the mean square for rows, MS_E is the mean square error, MS_C is the mean square for columns, k is the number of observers involved, and n is the number of subjects.

A freely available code was used for ICC computation [28]. According to ICC values, we stratified the features into four groups as having excellent (ICC >  = 0.75), good (0.60 <  = ICC < 0.75), fair (0.40 <  = ICC < 0.60), or poor (ICC < 0.40) reproducibility [29]. The reproducibility within groups of features was also assessed using the Wilcoxon rank-sum test with a p value threshold set at 0.05.

To test the possibility of predicting liver herniation as a dichotomous variable (up/down), several ML forecasting algorithms were implemented in the Matlab environment and Python, according to the experimenters’ convenience, using features calculated by pyradiomics. Several ML approaches were tested, such as decision trees, linear and non-linear artificial neural networks (ANN), and support vector machines (SVM) with various standard kernels.

Decision trees are a widely used method in ML, for both classification and regression tasks. A decision tree works by breaking down the classification procedure into a series of steps, each represented by a tree node (or leaf), so that an associated decision tree is incrementally developed. Each step asks a question that has a “yes” or “no” answer and redirects the flow towards different branches as you move down to another node or a tree lead, depending on the answer. The path from root to the final leaves (the classes) gives the overall classification rule. ANNs are inspired by the structure of the human brain. They consist of layers of interconnected nodes, known as neurons, which process information. Each connection between neurons has a weight that adjusts as the ANN learns from data. This structure allows ANNs to learn complex patterns and make predictions or decisions. ANNs can be linear or non-linear, depending on how these nodes and layers are arranged and interact. In simple terms, ANNs are like complex webs that learn to recognize patterns from the data they are trained on. Support vector machines (SVM) are another method used for classification and regression tasks. SVMs work by finding the best boundary that separates data into classes. This boundary is chosen to maximize the margin, or distance, between the boundary and the closest data points from each class, known as support vectors. SVMs are efficient in high-dimensional spaces and are versatile, as they can use various kernels (mathematical functions) to transform data so that a non-linear boundary can be used linearly.

For this part, only left-sided CDH patients were considered because of their larger numerosity, homogeneity, and variability in liver position, leaving outright CDH cases in which the liver is almost always herniated. The results obtained with the features computed in the manually segmented ROIs of the liver and lungs were compared with those obtained with those calculated in the nnU-Net segmented ROIs.

Since the MRI scans were very dissimilar in gray-level content, only shape features were used, discarding variables computed on the gray levels to avoid further image manipulation (intensity standardization). This choice left 22 features (Table 1), considering the liver and the lungs.

We trained and validated the models with a Leave One Patient Out (LOPO) scheme, in which each patient was chosen as the only element in the validation set, and the remaining patients built the training set. Classification quality was expressed as the area under the receiver operating characteristic (ROC) curve and by confusion matrices.

Segmentation results showed a very good accordance between manual and automatic methods. In Fig. 2, we reported an example of segmentation results of two single MRI cases of the liver and lungs, in which perfect accordance was observed. Nonetheless, quality varied for other images, and in some lung segmentation tests, one of the two lungs was lost during automatic segmentation. The Jaccard coefficient values for the whole dataset expressed as box plots are reported in Fig. 3. The average Jaccard coefficient for lung segmentation was 0.65, while liver segmentation showed better results with an average value of the Jaccard coefficient of 0.75. A Jaccard coefficient of 1 indicates perfect agreement, while a coefficient of 0 indicates no agreement.

Figures S1 and S2 (Supplemental Materials) show, respectively, for lungs and liver, the scatterplots of the features obtained for each variable, the values calculated in the manual ROIs (x-axis), and the corresponding values calculated in the automatic ROIs (y-axis). In case of perfect correspondence, the points should be located on the quadrant bisector.

To state the agreement between manual and automatic feature groups, we employed the Wilcoxon rank-sum test within each group of features. We also computed and examined ICCs across single features for testing interrater reliability. Table 2 provides results for single-measure ICCs under a two-way mixed model with absolute agreement.

Based on the approach chosen by Owens et al., we then classified the 103 features into four groups according to their ICC values having excellent (ICC ≥ 0.75), good (0.60 ≤ ICC < 0.75), fair (0.40 ≤ ICC < 0.60), or poor reproducibility (ICC < 0.40) [29]. Results are visually reported in Fig. S3 (Supplemental Materials) with a heat map. Of the 103 features, 46 (45%) showed excellent reproducibility, 11 (11%) exhibited good reproducibility, 22 (21%) showed fair reproducibility, and in 24 features (23%), reproducibility was poor.

As previously stated, only MRI shape features were used to automatically classify up vs. down liver herniation. In order to test whether the features considered at high reproducibility were more predictive in detecting liver herniation than the others, we also used ICC values as cut-offs for feature selection. In the first test, all the features were used without exclusion (case 1: no selection). In subsequent attempts, three thresholds were selected: 0.60, 0.70, and 0.75 (cases 2 to 4), and only the radiomic features with ICC values not lower than the threshold were considered. The features of the lung and liver were included for each specific case, and the corresponding results are shown in Table 3.

The best results were obtained without feature selection. Figure 4 shows the ROC curves for liver herniation (up/down) acquired by the best-tested classifier (a linear SVM). Without feature selection, the AUC obtained for the dataset of manually segmented ROIs and the one for the automatically segmented ROIs were equal to 0.86 and 0.84, respectively. The confusion matrices (cm) obtained and the corresponding values for sensitivity, specificity, and accuracy are reported in Table 4.