CRISIS AFAR: an international collaborative study of the impact of the COVID-19 pandemic on mental health and service access in youth with autism and neurodevelopmental conditions

A workgroup of ASD/NDD experts (ADM, LG, SG, PP, AT, BV) led the adaptation from the CRISIS Parent/Caregiver Baseline Form. Adapting the caregiver survey was prioritized in order to immediately capture the acute phase of the pandemic across the widest possible range of ASD/NDD youth using the same method. The adaptation aimed to include assessments of clinical domains relevant to ASD/NDD and services, while maintaining the existing structure of CRISIS [25, 42]. At the time, empirical evidence on the impact of disasters on ASD/NDD was limited to one study reporting worsening in adaptive functioning [43]. Therefore, along with adaptive skills, we prioritized the assessment of symptoms known to be affected by or to impact adjustment needs. These encompassed restricted and repetitive behaviors/interests (RRB) [44‐46], externalizing, internalizing symptoms, and sleep problems that often co-occur in ASD/NDD [8, 12]. Parent/caregiver questions were developed to target observable behaviors rather than attempting to seek reports on internal states. In keeping with this goal and to contain the survey’s length, new questions on a range of co-occurring psychiatric symptoms replaced the Mood State domain. The Substance Use domain was also removed given that it is more accurately measured by self-report, which is beyond the scope of the present study [47, 48]. Like CRISIS, symptoms were rated on a Likert-scale based on (1) the three months prior to the start of COVID-19 pandemic in the respondent’s geographical area and (2) the two weeks prior to completion (Prior and Current time points, respectively).

To evaluate changes in service access, we derived items from a survey developed during the pandemic and piloted with people with syndromic intellectual disabilities and their caregivers [26]. Questions queried changes in therapeutic services typically received both within and outside school settings in the respondents geographical area, following the start of the pandemic. The remainder of the original CRISIS was unchanged, except for some rewordings or additional response options (e.g., sleep problems) as summarized in Table 1 and detailed in Additional file 1: Methods.

Like CRISIS, questions of the parent/caregiver survey were developed for individuals aged five to 21 years; a later review identified a subset of questions developmentally applicable for children as young as three years. The initial adaptation was developed in English and then translated into five other languages and updated with rewordings in consultation with the larger AFAR network. The final version 0.5.1 of AFAR Parent/Caregiver Baseline Form (3–21) included 96 independent items, with 34 questions asked twice for Prior and Current time points. Clinical contacts with caregivers of youth with ASD/NDD during the acute phase of the pandemic informed the adaptation process; at its completion, several sites reached out to caregivers of ASD/NDD youth to review the survey; they favored its distribution for data collection (see Additional file 1: Methods for more details). AFAR is freely available for use by other investigators [42].

Following the AFAR data collection, we used exploratory and confirmatory factor analyses to identify the underlying structure of each of the domains developed a priori and to yield summary scores for quantitative investigations (see Additional file 1: Methods for more details). As detailed in Additional file 1: Results and in Tables S5 and S6, factor analyses yielded a stable and replicable factor structure, including a single factor for Adaptive Living Skills, two factors RRB largely reflecting lower- and higher-order RRB [46, 49], and four for Co-occurring Problem Behavior (i.e., Anxiety/Affect, Oppositional Behavior, Sleep Problems, Activity/Attention), among the AFAR-specific domains. Further, consistent with the general population [25], results yielded a single COVID-19 Worries factor, and multiple factors for the Life Changes and Behavior/Media domains (two and five, respectively).

AFAR Parent/Caregiver Baseline surveys were collected in 15 samples at 14 research and/or clinical institutions in Europe and North America. Data were collected cross-sectionally over the first six months of the pandemic (April–October 2020; Fig. 1, Additional file 1: Methods and Table S1). Parents of children aged three to 21 years with previously established clinician-based DSM-IV/5 [6, 50] or ICD-10 [5] diagnosis of ASD and/or other NDD were invited to complete AFAR. As discussed below, only data from individuals aged five years and older are included in the present analyses.

Along with diagnostics, when feasible, previously collected information on intellectual functioning and symptoms was shared. IRB approval for collection and sharing of de-identified AFAR and related data was obtained at each institution. For the present analyses, only data from AFAR surveys with available AFAR responses for the variables used in clustering analyses and completed within the time interval of 90% of a given sample were included. This served to minimize outliers regarding to COVID-19 infection rates and related responses.

Data from 1275 youth, 5- to 21-year-old, aggregated across the 15 contributing samples met inclusion criteria for subsequent analyses. As shown in Figs. 1 and 3, contributing samples were from Europe (49%) and North America (51%). They included seven samples collected across research and clinical institutions in Italy, one each from Greece and Ireland, four samples collected in the USA (two in New York, and one each in California, and Missouri), and one sample collected across five centers in Ontario, Canada, as part of a COVID-19 multi-network collaboration in pediatrics, including the Province of Ontario Neurodevelopmental Disorders COVID Mental Health collaboration (POND-CMH) Network [61‐63]. Key demographics (age and sex a birth), diagnosis, and intellectual functioning are summarized in Fig. 3 for the aggregate and each contributing sample. Most individuals (79%, n = 1004) met diagnostic criteria for ASD; 17% (n = 214) had ADHD without ASD, and 4% (n = 57) had other NDDs, without ASD. Reflecting the known male bias of ASD/NDD [64, 65], the sample predominantly included males (88%). Among 938 (80%) youth with available intelligence estimates, 62% (n = 624) were in the Average/Above Average range, 14% (n = 141) in the Borderline range, and 24% (n = 173) had mild to profound intellectual disability. Further, over half (64%, n = 811) of the caregivers had at least a college degree; the remaining had either a high school degree (30%), or elementary education (6%). Fifty-six percent of the aggregate dataset was of European/British ancestry (see Additional file 1: Table S4). Information on prior (i.e., baseline) AFAR symptom factor severity and number of services received within and outside school are in Fig. 5; complementary characteristics are in Additional file 1: Results, Figures S1, S2 and Tables S2–S4.

Based on the NbClust approach, the four-cluster solution resulting from hierarchical clustering was optimal (see Figs. 4 and 5 and Additional file 1: Table S7). In other words, clustering analysis yielded four subgroups of ASD/NDD youth capturing distinct outcome profiles of symptom and service changes (Fig. 4). Based on their profile of deviations from the aggregate’s average of symptoms and/or service changes, the four outcome subgroups were defined as: broad symptom worsening only (20%), primarily modified services (23%), primarily lost services (6%), and average symptom/service changes (53%). The subgroup defined as broad symptom worsening only was characterized by worsening across all symptom domains as indexed by z scores ranging between 1 and 3 (i.e., 1 to 3 standard deviations above the aggregate’s mean) but by marginal service changes relative to the aggregate’s average (i.e., within 0.5 standard deviation from the aggregate’s average). The three remaining subgroups, totaling n = 1024 (80% of the aggregate), showed symptom changes within the aggregate sample’s average (z scores < 0.5) but differed in service access. As indicated by their name, one subgroup had the most services modified, another had the most services lost, and the third subgroup had the number of services lost and continued like those of the aggregate. These differential profiles of changes in symptom and services were confirmed by one-way ANOVAs and Tukey pairwise group mean comparisons (FDR q < 0.05; Additional file 1: Table S7). Since, as expected by our naturalistic study design, differences in subgroup distributions existed between contributing samples (Additional file 1: Fig. S3), we conducted follow-up analyses including “contributing sample” as covariate. Statistical group differences were unchanged. Similarly, secondary cluster analyses on data subsets distinct by survey completion rates yielded similar results (Additional file 1: Results).

Finally, Additional file 1: Table S8 provides descriptive statistics and subgroup comparisons regarding key demographics (age and sex at birth) and intellectual functioning. As detailed in Supplementary Results in Additional file 1, the four subgroups did not significantly differ in sex distribution; there was a main effect of subgroup for age with the subgroup average service and symptom change being significantly older (11.6 ± 3.5 years) than the three other subgroups which included children in mid-childhood (mean and SD range across subgroups: 9.7–10.8 ± 3.1–3.8 years). Most youth fell in the average/above average intelligence category across subgroups except for the primarily lost services subgroup, which was characterized by significantly greater intellectual impairment.

The RF classification model predicted subgroup membership with 81% accuracy (precision/sensitivity = 82%, recall/specificity = 75%). The top-ranked predictor (OOBE: 12%) was the number of services received at school before the pandemic (Fig. 6a). Six predictors followed, with OOBE ranging from 6 to 1%. In rank order, they included the number of services received in and outside of school, GS index, Lifestyle Stress, COVID-19 Worries, new COVID-19 infections, and child’s age. The remaining features had OOBE < 1% and, thus, were considered negligible predictors.

Across the top seven predictors, each outcome subgroup was characterized by unique combinations of increases or decreases relative to the other subgroups. For example, the primarily modified services subgroup had a greater number of services before the pandemic relative to the broad symptom worsening only and to the average symptom/service changes subgroups. This latter subgroup, however, had lower ratings of COVID-19 Worries and Lifestyle Stress and included older youth in comparison with the other three subgroups. The primarily lost services subgroup had a similarly higher number of pre-pandemic services received but had the lowest GS index and COVID-19 new infection rates, as well as lower COVID-19 Worries and Lifestyle Stress relative to the other subgroups. Notably, the broad symptom worsening only subgroup had the highest COVID-19 Worries and Lifestyle Stress ratings, even with lower infection rates and GS index relative to the primarily modified services subgroup. These subgroups’ differences across predictors were confirmed by one-way ANOVAs and Tukey pairwise group mean comparisons (FDR q < 0.05; see Table 3). Since differences in subgroup distributions existed between contributing samples (Additional file 1: Fig. S3), we conducted follow-up analyses covarying for “contributing sample” in comparisons for the total baseline services received, Lifestyle Stress, COVID-19 Worries, and child age, but not for GS index and COVID-19 rates, as they reflect known differences across contributing samples per study design; statistical subgroup differences remained unchanged.

Notably, the predictor ranked eighth—i.e., pre-pandemic global severity—negligibly contributed to the classification model. Nevertheless, for illustrative and interpretation purposes, it is plotted in Fig. 6b and its descriptive statistics for each of the subgroups are summarized in Table 3. Similar to intellectual functioning, the four outcome subgroups did not differ with respect to pre-pandemic global severity except for the primarily lost services subgroup which was significantly more severe than the other three subgroups. Given that the primarily lost services subgroup was characterized by significantly greater intellectual impairment than the other three, we repeated supplementary RF analyses including intellectual categories in the subset of youth with intellectual functioning category available (n = 926). As detailed in Supplementary Results in Additional file 1, results were virtually unchanged relative to those from primary analyses; the feature indexing intellectual functioning had negligible predictive value for subgroup membership.

Although this is the most comprehensive and systematic study of the covariation of symptom and service changes in youth with NDD/ASD during the pandemic, results should be interpreted in light of several limitations. First, considering time constraints on questionnaire completion, albeit comprehensive, AFAR could not assess all possible domains of impact and/or prediction. For example, symptoms least expected to change over a short period of time, were given lower priority, most notably, social communication impairments [70]. Given the protracted nature of the pandemic, future studies should include assessment of longer-term changes in social communication skills. Similarly, although family demographics, parent education, and parent being an essential worker were assessed and included in our predictive model, parent’s mental health, recently reported to relate to children’s outcomes during the pandemic [71‐74], was not assessed. Second, because of the urgency of the pandemic, we did not systematically involve other stakeholders in the survey adaptation process. However, the input of caregivers of children with ASD was accounted for in a number of informal ways during and at completion of the survey. Third, although the aggregate sample included youths with clinician-based diagnoses, previously collected measures of severity varied by contributing sample, and assessments of the role of prior severity were based on parent responses in the AFAR survey. Nevertheless, we found that the AFAR baseline global severity scores correlated with available standardized measures. Fourth, although relatively large, our aggregate convenience sample could not address all demographics. For example, females were underrepresented, and preschoolers were not included. Although consistent with a systematic review of pandemic studies in youth [4] sex at birth was not a robust predictor of impact, given recent reports of a male to female ratio of 3:1 in ASD [64], future studies should oversample females. Fifth, the present study focused on impact over the first six months of the pandemic using a cross-sectional design. Longitudinal coordinated study designs and infrastructures (e.g., common assessment measures) are needed to capture long-term outcomes and define stability of the subgroups over time. Finally, this first study relied exclusively on parent reports in order to capture the acute phase of the pandemic across a widest possible range of ASD/NDD youth and abilities using the same approach. Future studies should include self-report and/or direct observation to complement the parent’s perspective.