Evaluation of Normalization after Implementation of the Digital Dutch Obstetric Telephone Triage System: Mixed Methods Study with a Questionnaire Survey and Focus Group Discussion

Background: The Dutch Obstetric Telephone Triage System (DOTTS) was developed to improve the quality of acute obstetric care. To achieve optimal effect, the DOTTS should be adopted in the daily care process by triage staff. Objective: The primary aim was to evaluate the degree of implementation (ie, normalization) of the DOTTS, and the secondary aim was to evaluate which lessons can be learned from its current implementation in Dutch hospitals. Methods: An evaluation study with a mixed methods design was performed. All triage staff in 9 Dutch hospitals that implemented the DOTTS before September 1, 2019, were invited to complete the Normalization Measure Development (NoMAD) questionnaire between December 2019 and July 2020. The questionnaire is based on the Normalization Process Theory (NPT). This self-reported questionnaire provides insights into the work people do in order to integrate and embed new practice in routine care. The NPT is based on the following 4 constructs: coherence, cognitive participation, collective action, and reflexive monitoring. Within the questionnaire, each construct is represented by 4-7 questions. Questions are scored on a 5-point normalization process scale. Descriptive statistics were used for analysis of questionnaire scores. Subsequently, the questionnaire findings were discussed during a focus group. Template analysis following the 4 constructs was used for analyzing the results of the focus group. Results: Overall, 173 of 294 (58.8%) triage staff members completed the NoMAD questionnaire, and 90.2% (156/173) of the participants had used the DOTTS for over 6 months. The digital application was used as much as possible or always by 137 of 173 (79.2%) participants. The overall normalization process score was 3.77 (SD 0.36). The constructs coherence, cognitive participation, collective action, and reflexive monitoring scored 4.01 (SD 0.47), 4.05 (SD 0.45), 3.5 (SD 0.45), and 3.72 (SD 0.47), respectively. Analysis of the focus group discussion showed that the added value of the DOTTS was seen as a quality improvement for the care of pregnant women. Dedication of the complete multidisciplinary implementation team was important for facilitating normalization. Support from the medical staff and proper use by all disciplines involved in the triage were seen as facilitating factors. Participants appreciated training and evaluation, and indicated a need for ongoing training and evaluation in relation to goal achievement. Conclusions: The DOTTS has been integrated into normal care in daily practice. Evaluation by the NoMAD questionnaire provided a positive overall score. These results are in line with or, in some aspects, better than the results of other evaluation studies. Key factors in the normalization process of the DOTTS in obstetric triage are the shared added value for stakeholders, the dedication of the complete multidisciplinary implementation team, and implementation plans that are tailor made in the practical context of the hospital.