Development and validation of the Sorting non-trauMatIc adoLescent knEe pain (SMILE) tool – a development and initial validation study

This study consists of the development and evaluation of the Sorting non-trauMatIc adoLescent knEe (SMILE) tool for non-traumatic knee pain in adolescents. The study was conducted at the Center for General Practice at Aalborg University between February 2019 – February 2020 and inspired by requests from our reference group of general practitioners (GPs).

After an initial pilot evaluation of the SMILE tool with one medical doctor and one medical student, we modified the layout based on feedback, and subsequently assessed and iterated the SMILE tool on two different test days. The content of the different test days and how we optimised and evaluated the SMILE tool after the test days is described in Fig. 1.

Baseline characteristics for adolescents were calculated using descriptive statistics. To test the validity of the SMILE tool we calculated the percentage agreement without and with the SMILE tool between the diagnoses given by the assessors and the gold standards diagnoses. We tested the inter-rater reliability of the SMILE tool using Fleiss kappa statistics and McNemar’s test to determine differences in proportion correct diagnoses with and without the SMILE tool. Descriptive statistics were performed using Microsoft excel version 16.34. Fleiss kappa and McNemar’s test were calculated in R version 3.5.3.

The systematic search revealed a total of 13,429 articles of which 81 were screened by full text. After full-text screening 23 articles were eligible for data extraction (Fig. 2).

Based on the input from experts, the first version of the SMILE tool included the most common complaints which were growing pain, Osgood Schlatter, Sinding Larsen Johansson disease, Patellar tendinopathy, patellofemoral pain, and Iliotibial band syndrome as the main diagnoses in the tool (Fig. 3). We subsequently included the following differential diagnoses which were mentioned as important in the included articles and following feedback from experts and clinicians: Pes anserine tendinopathy, Baker’s cyst, popliteus tendinopathy, referred pain, inflammatory arthritis, osteochondritis dissecans, infection, and malignancy (Fig. 3). Figure 3 shows the final version of the SMILE tool. Detailed description of the development process is in Additional file 2 and methods section.

In total 27 adolescents (aged 10–17) participated (see Table 2). Of the 11 participants on test day 2, eight (72.7%) participated in one of the previous test days and were assessed by the two gold standards on separate occasions.

The seven assessors (three female and four male) had 0–4 years of clinical experience (mean 2.4 ± 1.5). Six were medical students in the final year of their medical education. One was a medical doctor with authorisation to practice. One had a special interest in musculoskeletal pain. The assessors came from 2 different Universities in Denmark.

Agreement with the gold standards increased with the use of the SMILE tool on both test days, with 77.3% percent agreement with the final version of the SMILE tool (Table 3).

One participant had differing diagnoses by the two assessors used as gold standard on the different test days; PFP and OSD respectively. Among the junior assessors, seven out of seven diagnosed the participant with OSD without the tool on the two test days. The participant reported pain directly at the tibial tuberosity with a bony prominence. Therefore, the participant was considered to have OSD.

Inter-rater reliability increased with the use of the SMILE tool on both test days, with a moderate agreement on the final version of the SMILE tool. On the final test day (test day 2) Fleiss’ kappa increased from poor agreement without the tool k = − 0–04 (95% CI, − 0.12 to 0.04), to moderate agreement with the SMILE tool k = 0.56 (95% CI, 0.40 to 0.72).

To our knowledge, this is the first study to develop and initially evaluate a support tool for clinical practice to improve management of adolescents with non-traumatic knee pain. The percentage of correct diagnoses and reliability reached nearly 80% using the final version of the tool. This reflected an approximate fourfold increase in junior medical doctors’ correct diagnoses, compared to without the SMILE tool. The reliability of the final version of the SMILE tool between assessors was moderate.

Our validation is based on a relatively small sample size and external validation is needed. We purposefully recruited medical students to reflect inexperienced doctors. This choice may mean that the different in correct diagnosis may be larger among this sample compared to a sample of more experienced medical doctors. We cannot generalise our results to other professions in sports medicine, orthopaedics or other areas where this patient population consults. Further each participant’s cause of knee pain was unknown at recruitment and we were unable to recruit according to specific diagnoses and therefore certain conditions, i.e. growing pain, were not represented among the participants. However, our primary focus was on non-traumatic knee injuries, which were well represented and for which we showed high validity.

Our testing was not conducted as part of a normal clinical day of clinical practice. This tool was designed for non-specialist doctors (GPs). The assessors had 10 min for each to ensure comparability to clinical practice. The flow of patients was not identical to a normal clinical day as all patients had knee pain, which made it easier for the assessors to focus only on knee pain. Future studies may focus on the implementation of the SMILE tool in a clinical practice setting including the need for more explicit guidelines on when to refer patients for example if oncologic or inflammatory etiologies are considered. Finally, this tool was designed for GPs, and has not yet been tested with other types of medical doctors. The tool was largely tested by medical trainees in a general practice setting to specifically target junior doctors in this setting. In this study we recruited patients with non-traumatic knee pain. The patients included, had many of the diagnoses outlined in tools, as we developed this to consider the most common diagnoses. Further research is needed to evaluate whether the tool can adequately support clinicians with patients with more diverse conditions, and correctly rule out serious pathology as this has not been evaluated in this initial study.

Diagnosing knee pain can be challenging, especially during adolescence where growth related injuries (Osgood Schlatter, Sinding-Larsen Johansson) are common and unique to this population. It is also important to differentiate growing pains and growth related non-traumatic knee pain. In this study, the assessors only diagnosed about 20% of adolescents correctly compared to specialists (gold standards) without using the SMILE tool, with a poor reliability between assessors. Given the importance of diagnoses for treatment [12, 14], this tendency of low diagnostic accuracy is concerning.

Diagnostic clarification is important for adolescents when consulting for knee pain [14]. However, GPs find it challenging to give the correct diagnosis [2, 13]. We know that non-traumatic knee complaints are overlooked compared to traumatic knee complaints [3, 4]. Our assessors focused on traumatic causes of knee pain when they assessed patients without our tool (data included in Additional file 3). This underlines younger GPs’ or junior medical doctors’ need for guidance when assessing adolescents with non-traumatic knee pain.

Current available clinical tests primarily focus on a single diagnosis [5, 18‐23]. Our simple SMILE tool does not require any clinical tests except palpation and pain localisation, which was highlighted in the interviews by the assessors that this made the SMILE tool extremely user-friendly without any training. Furthermore, the assessors highlighted that this tool is easy to use and time efficient, which may facilitate implementation into a GP consultation. Most importantly, it does not need any pre-training or specialist introduction which may facilitate use and implementation.

We aimed to create a tool that includes the most common presentations of non-traumatic adolescent knee pain, as ease-of-use and time requirements are important factors for GPs and medical professionals [24, 25]. The high level of agreement between our SMILE tool and the gold standard suggest that despite having several diagnoses it has a high validity. We showed that our inexperienced assessors improved their diagnostic accuracy almost 4 times with the assistance of the SMILE tool. Further, their focus shifted from thinking about traumatic causes of knee pain, even in presentations without trauma, to thinking about non-traumatic causes (data shown in Additional file 3). This shift may be favourable in general practice, as non-traumatic causes of knee pain are the most prevalent in adolescents [3].

This tool was developed for a general practice setting and therefore we developed a simple tool feasible to use in the setting of a 10-min consultation without previous knowledge of the patient. This tool is primarily made for GPs who have a wide range of different patients daily, where it is still necessary to maintain a high level of expertise in all topics to meet the patients’ expectations and provide the appropriate management. Our SMILE tool gives an easy overview of the most common non-traumatic knee complaints seen in adolescents without the need of much introduction and can thereby easily fit into a general clinical practice setting with a variety of different patients. Despite being designed for general practice setting, this tool could influence other medical specialities including orthopaedic surgery and pediatric rheumatology. Patients are often co-managed between specialities and if correct diagnosis is achieved in initial assessment in general practice, this could lead to more relevant referrals and targeted treatment.