Successes of and lessons from the first joint ehealth program of the Dutch university hospitals: Evaluation study

Background: A total of 8 Dutch university hospitals are at the forefront of contributing meaningfully to a future-proof health care system. To stimulate nationwide collaboration and knowledge-sharing on the topic of evidence-based eHealth, the Dutch university hospitals joined forces from 2016 to 2019 with the first Citrien Fund (CF) program eHealth; 29 eHealth projects with various subjects and themes were selected, supported, and evaluated. To determine the accomplishment of the 10 deliverables for the CF program eHealth and to contribute to the theory and practice of formative evaluation of eHealth in general, a comprehensive evaluation was deemed essential. Objective: The first aim of this study is to evaluate whether the 10 deliverables of the CF program eHealth were accomplished. The second aim is to evaluate the progress of the 29 eHealth projects to determine the barriers to and facilitators of the development of the CF program eHealth projects. Methods: To achieve the first aim of this study, an evaluation study was carried out using an adapted version of the Commonwealth Scientific and Industrial Research Organization framework. A mixed methods study, consisting of a 2-part questionnaire and semistructured interviews, was conducted to analyze the second aim of the study. Results: The 10 deliverables of the CF program eHealth were successfully achieved. The program yielded 22 tangible eHealth solutions, and significant knowledge on the development and use of eHealth solutions. We have learned that the patient is enthusiastic about accessing and downloading their own medical data but the physicians are more cautious. It was not always possible to implement the Dutch set of standards for interoperability, owing to a lack of information technology (IT) capacities. In addition, more attention needed to be paid to patients with low eHealth skills, and education in such cases is important. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the success of the 29 projects. The in-depth interviews illustrated that a novel eHealth solution should fulfill a need, that partners already having the knowledge and means to accelerate development should be involved, that clear communication with IT developers and other stakeholders is crucial, and that having a dedicated project leader with sufficient time is of utmost importance for the success of a project. Conclusions: The 8 Dutch university hospitals were able to collaborate successfully and stimulate through a bottom-up approach, nationwide eHealth development and knowledge-sharing. In total, 22 tangible eHealth solutions were developed, and significant eHealth knowledge about their development and use was shared. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the successful progress of the projects and should therefore be closely monitored when developing novel eHealth solutions.