Effectiveness of clinical training on improving essential newborn care practices in Bossaso, Somalia: a pre and postintervention study

This quasi-experimental prepost intervention study was conducted in Bossaso, Somalia from August 2016 through December 2018. Mixed data collection procedures included health worker knowledge and skills evaluations; observations of essential newborn care practices during labor, birth, and immediately after birth; postnatal interviews of mothers at home or via phone on the 7th -9th day after birth; and in-depth interviews and focus group discussions with health workers.

Preintervention baseline measurements were recorded from August to October 2016. An intervention package consisting of the clinical training for health workers, provision of newborn medical kits, and installation of a newborn health record system was implemented from October 2016 to April 2018. The postintervention endline measurements were taken from April to December 2018 (Fig. 1).

In Somalia, the health system has four levels: hospitals, referral health centers, health centers, and health posts [20]. Health centers are also known as maternal and child health centers. In Bossaso city, six health centers and one public hospital provide maternal and child services to internally displaced persons (IDPs) and to the host community. In consultation with the Ministry of Health, we selected four of the six health centers serving IDPs based on predefined selection criteria: the health facilities were open 24 h a day, 7 days a week and had an average of at least 40 deliveries per month. The total catchment population of the four health facilities was estimated at 134,735 persons, including both the IDPs and the host community.

Pregnant women 15–49 years of age who sought childbirth care at one of the study facilities during the study period were eligible and approached for consent and enrollment in the study. Women who were immediately referred to a hospital prior to childbirth were excluded. Women who had a stillbirth or early newborn death defined as death from 0 to 7 days of life were excluded from the postnatal interview out of respect for the family. Health workers who were responsible for service provision in the labor/maternity unit and the in-charges at the four health facilities were approached for consent and included in the study.

The successful translation of the Field Guide to Practice necessitates the training of health workers in essential newborn care practices, the provision of newborn medical supplies and drugs, and the installation of data collection systems. The curriculum for the training of health workers was based on the AAP HBS program and comprised the Helping Babies Breathe (HBB), Essential Care for Every Baby (ECEB), and Essential Care for Small Babies (ECSB) [12]. Supplemental modules on maternal health, intrapartum care, identification and management of maternal complications, and supportive supervision were also included in the 8-day course. The training was taught by two experienced clinicians from Kenya with expertise in the subject area and experience as educators. The 8-day essential newborn care course was taught through various teaching methods: didactic lectures, videos developed for educational purposes, small group discussions, and skills practice with NeoNatalie, MamaBreast simulator, and a partograph [21‐24]. Skills practices were performed in pairs and covered the assessment of a newborn, immediate newborn care, newborn resuscitation, and completion of a partograph on simulated case stories. The trainees were midwives, registered nurses, and in-charges at the four health centers. A 5-day refresher training was conducted 6 months after the initial course focusing on review of the topics covered during the initial training. Knowledge and skills evaluations were conducted pretraining, immediately after training (posttraining), and at an 18-month follow-up.

Twelve health workers (three per health center) were trained, representing 43–46% of all registered nurses and midwives in the four health facilities. Overall, there were 26 registered nurses or midwives working at the four health facilities at baseline, and 28 registered nurses or midwives at endline. We were unable to train all the health workers due to funding limitations and the requirement for some providers to remain on duty.

Newborn kits and delivery kits containing the medical supplies and drugs recommended by the Field Guide for the primary health facility level were distributed to the four health centers. The kits included the supplies and medicines necessary to monitor labor, attend childbirth, perform newborn examinations, perform newborn resuscitation, provide routine predischarge care, antibiotics, and medications for maternal health (Supplement 1: List of medical supplies and drugs).

As the existing labor and delivery registers lacked key information on the newborns, a supplemental newborn register was installed. The newborn register included information on gestational age, date and time of birth, birth weight, and any newborn complications observed (Supplement 2: Newborn register book). The newborn register was created in English and translated into Somali. The Somali version of the register was printed and distributed to the health facilities; health workers were trained and received orientation on how to complete the register book and extract data monthly.

The primary outcome was a composite indicator of the essential newborn care practices and services provided to newborns at birth and immediately after birth and was measured via direct observation of clinical practices using an observation checklist. The observation checklist was adapted from the WHO’s Managing Complications in Pregnancy and Childbirth Guide [25], which has been validated in African and conflict settings [26, 27]. The observers were female from the community and with a health background (midwifery and nursing students). All observers received didactic and video-based simulation training that demonstrated the recommended essential newborn care practices. The observation tool was piloted for 2 days during the preintervention baseline assessment by pairing observers. Interobserver agreement was 94.6%.¹ The primary outcome, essential newborn care, was defined as the proportion of newborns that were observed to receive at least two essential newborn care interventions. Additional variables measured were care received during pregnancy (history), predischarge education given to the mother, and maternal and newborn outcomes. Postnatal interviews with mothers were conducted either by phone or in person on the 7th to 9th day after birth. The postnatal interviews captured the status of the newborn, the mother’s knowledge of danger signs and newborn care practices, and the mother’s level of satisfaction with the care received at the health facility.

Scores of the health worker knowledge test and skills evaluation were collected for all training participants. Twelve health workers participated in the training and were evaluated at pretraining, posttraining, and at the 18-month follow-up. The knowledge and skill evaluation tools, the multiple-choice questionnaire (MCQ) and scenario-based Objective Structured Clinical Examinations (OSCE), were adapted from the AAP course. The lifesaving skills emergency obstetric training case studies on partograph use were applied to evaluate skills in the accurate completion and use of the partograph [24].

Qualitative assessment was performed at the endline. Three Focus Group Discussions (FGDs) were held with 30 health workers at the four health centers (one with providers who received the training, two with providers who had not), and in-depth interviews (IDIs) were conducted with four in-charges. The qualitative assessment aimed to examine whether and how the 8-day course and refresher training were shared among health workers who had not participated in the training; to gather information from those who attended the training on what had changed in their clinical practice after course completion; to gain insight into the perspectives of the health workers (trained and untrained) on the utility of the newborn commodities and medical supplies; and to collect their feedback on the installed newborn record/register system. The FGDs and IDIs were either conducted in English with a Somali interpreter or in Somali and audio recorded then translated and transcribed into English.

We assumed a baseline prevalence of the provision of essential newborn care of 30%, a power of 80%, a 5% probability of a Type I error, and a nonresponse rate of 20%. To detect an absolute difference in the primary outcome, provision of essential newborn care, of at least 15%, the number of mother-newborn pairs needed was 203 at both pre and postintervention. The sample size for each facility was allocated using a proportional to estimated size where the size measure was based on historic data on number of childbirths per health facility [28].

Descriptive statistics were used to summarize the data. Proportions with 95% CI, mean (standard deviation), and median (interquartile range) were generated to characterize the study population. Missing and “don’t know” responses were analyzed as missing and excluded from the analysis. To estimate the effect of the training on the trainees’ knowledge and skills, paired Student’s t-tests were used. Differences in two proportion z-tests were used to test for differences in proportions in maternal and newborn characteristics and essential newborn care practices pre and postintervention. To accommodate for possible correlations of changes in score with health facility, i.e., single level of clustering at the health facility level, we ran a generalized estimating equation (GEE) with robust option. The GEE utilized health facility as the cluster, the indicators of interest as an outcome, and the preintervention versus postintervention indicator as predictor to estimate the adjusted odds ratio, F-statistic and p-value. We applied logistic regression to estimate the risk ratio, an approximation of the odds ratio for rare events, of early neonatal mortality and stillbirth. Statistical significance was considered at a p-value < 0.05. STATA (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP) was used for the quantitative analysis. For the qualitative analysis, audio recordings from the IDIs and FGDs were transcribed into English and imported into MAXQDA Analytics Pro (VERBI Software, 2017) for coding and analysis. Researchers engaged in content analysis applying an initial coding list based on the interview and focus group discussion guides. New codes were added as they emerged during the analysis. Once the coding framework was finalized, a subset of transcripts was coded by a primary and a secondary analyst to determine the intercoder reliability. The themes that emerged were consolidated into four subthemes.

Approval for the study was sought from the Puntland Ministry of Health, Save the Children and the Centers for Disease Control and Prevention (CDC). Approval was obtained from the Puntland Ministry of Health, the Save the Children ethics review committee, and a nonresearch determination approved by the CDC. Consent information was read to the women in the local language, and verbal consent was sought at each point of contact. Those who consented were included in the study. Personal identifiers collected to facilitate postnatal follow-up visits were destroyed immediately after completion of the data collection process. Verbal consent was obtained from health workers and the in-charges who were responsible for service provision at the four health facilities.

All 12 registered nurses and midwives who attended the training completed the multiple-choice knowledge test questionnaire and the scenario-based objective structured clinical skill evaluation at three time points. Their knowledge scores improved from pre to posttraining and at the 18-month follow-up. The mean difference in score from posttraining to pretraining was + 11.9% [95% CI: 7.2, 16.6; p-value < 0.001] and from 18-months after training to posttraining was + 10.9% [95% CI: 4.7, 17.0; p-value < 0.001] (Fig. 3). The score on the accurate completion of a partograph at baseline was 28.5% (95% CI: 11.7, 45.3), which improved at posttraining with a mean difference in score of + 68.5% (95% CI: 52.7, 84.3; p-value < 0.001) and declined at the 18 month follow-up from posttraining with a mean difference in score of − 30.3% (95% CI: − 13.5, − 47.1; p-value = 0.002). Skills in newborn resuscitation with bag and mask improved from pretraining to posttraining with a mean difference in score of + 65.1% (95% CI: 53.4, 76.7; p-value < 0.001), and the skill was retained at the 18-month follow-up from the posttraining score with a mean score difference of + 0.4 (95% CI: − 6.6, 7.4; p-value = 0.903) (Fig. 3).

The primary composite outcome, proportion of newborns who received two or more essential newborn care practices (skin-to-skin contact, early breastfeeding, and dry cord care), improved from 19.9% (95% CI: 4.9, 39.7) at baseline to 94.7% (95% CI: 87.7, 100.0) at endline with a difference in proportion of + 74.8% (95% CI: 69.1, 80.5; p-value < 0.001). In the adjusted model, the odds of receiving two or three newborn practices at endline versus baseline was 64.5 (95% CI: 15.8, 262.6, p-value < 0.001). The proportion of newborns who received three newborn care practices improved from 0.8% (95% CI: 0.0, 1.7) at baseline to 61.4% (95% CI: 37.8, 77.0) at endline, with a difference in proportion of + 60.6% (95% CI: 54.6, 66.5; p-value < 0.001) (Table 3). All newborns who had birth asphyxia were successfully resuscitated both pre and postintervention. The difference in the proportion of newborns who received skin-to-skin contact was + 64.6% (95% CI: 58.2, 71.0, p-value < 0.001) from a baseline of 8.5% (95% CI: 5.4, 12.7) and in early breastfeeding was + 53.6% (95% CI: 46.4, 60.9, p-value< 0.001) from a baseline of 30.1% (95% CI: 24.4, 35.8) (Fig. 4).

The differences in essential newborn practices was one directional (improvement) for all health facilities for most of the indicators. However, for predischarge care and education, handwashing by the attendant, and use of the sterile delivery kit, the direction of change varied by health facility. Overall, predischarge education provided to mothers related to newborn care at pre vs postintervention on the topics of skin-to-skin contact (3.7% (95% CI: 1.3, 6.1) vs 10.0% (95% CI: 6.4, 13.7)), breastfeeding (16.5% (95% CI: 11.8, 21.1) vs 44.2% (95% CI: 38.2, 50.3)), and danger signs of newborn illness (9.1% (95% CI: 5.4, 12.7) vs 5.0% (95% CI: 2.4, 7.7)) remained suboptimal (Fig. 5).

This study was not powered to detect changes in mortality. From the data gathered, the proportion of stillbirth was 2.8% (95% CI: 1.1, 5.6) at baseline and 2.6% (95% CI: 1.0, 5.3) at endline, with a risk ratio of 0.9 (95% CI: 0.3, 2.6; p-value = 0.899). Early newborn mortality was 1.7% (95% CI: 0.4, 4.9) at baseline and 2.0% (95% CI: 0.7, 4.6) at endline, with a risk ratio of 1.2 (95% CI: 0.3, 4.9, p-value = 0.804).

The themes that emerged from the FGDs and IDIs were summarized in four subthemes: knowledge and skills gained; dissemination of training; applicability of the medical supplies and kits received; and use of the newborn register. In the FGDs and IDIs, health workers reported that the training taught them new skills and improved their previously acquired skills and knowledge.

“There’s an encouragement for the staff to conduct their services. Even your mind changes – our mind has changed when we got that training and knowledge.” (In-depth interview participant)

Interventions that resulted in self-reported areas of change included Kangaroo Mother Care (KMC) and recognition of the golden minute for newborn resuscitation. Participants stated that they had been taught about the importance of skin-to-skin contact previously as an intervention for low birth weight babies, but after the study intervention period, they now used skin-to-skin contact for all babies delivered in their facilities.

Notably, changes in care resulting from the intervention were reported in the FGDs with health workers who had not attended the training as well as those who attended the training, most likely due to dissemination efforts within the facilities. All participants from the FGDs and IDIs noted that there was a diffusion of learning within their facilities following the training. Many participants shared that this diffusion of knowledge and materials was an expectation within their facilities. Information was shared through didactic presentations, dissemination of training materials, hands-on demonstrations, on-the-job training, or through a combination of these methods. In some facilities, health workers who had attended the training were paired for multiple shifts with a colleague who had not attended the training.

“You know some of our staff get trained – they share with other colleagues. That is normal.” (Focus group discussion participant)

“Every trained person was assigned to train other untrained staff during shifts, one trained and one untrained in one shift.” (Focus group discussion participant)

Participants discussed the newborn medical supplies and drugs that the health centers received as part of the study. They commented that some of the medical supplies and drugs in the kit were useful and that they would have liked to have more delivery kits, baby caps, towels, vitamin k, and antibiotics. Some supplies were not used because they either did not know how to use the medical equipment (for example, the vacuum extraction delivery kit) or the supplies were not installed (for example, the table for baby reanimation with overhead heater). The participants also discussed their preference for single use medical supplies, such as disposable newborn suction devices, as they found cleaning and sterilizing instruments challenging.

The participants discussed that the newborn register documents more information on the newborn than they had collected in the past, and since the installation of the newborn register, they have changed the way they record data. For example, newborn birth weight, previously recorded in kilograms, is now recorded in grams. The register also facilitated easier review of information. However, participants voiced challenges associated with the register, they reported that they recorded large amounts of information and that it added to their workload.

Interrupted time-series study design is increasingly used for the evaluation of public health interventions, and it is particularly well suited to interventions introduced over a clearly defined time-period. Our study had a clearly defined time for pre and postintervention measurements, and the interventions were introduced after the preintervention measurement was completed, decreasing the likelihood that concurrent changes in practice influenced the outcome. The 18-month follow-up enabled us to assess the retention of knowledge and skills. The use of observation rather than self-reported or register-based information was a strength of this study. However, the study had several limitations. First, we were unable to quantify individual (provider) levels of change in practice. Second, our estimates had wide confidence intervals overall, reflecting the small number of clusters in the study, leading to corrections widening the confidence intervals. This wide confidence interval is much more visible in the GEE model, where there were also small numbers present in the comparison groups. Third, we were unable to train all of the health workers at the four health centers. Fourth, we lost some mothers to follow-up at the postnatal visit, which might have affected the outcome. There is also the possibility of the Hawthorne effect, where providers might have performed differently under observation than in routine practice [35]. Fifth, the study was performed at four health facilities, and the implementation package and findings of this study might not be generalizable to national or large-scale initiatives. Finally, temporal changes in context or policy during the implementation period could have influenced the results.