TY - JOUR
T1 - External Validation of a Prediction Model for Falls in Older People Based on Electronic Health Records in Primary Care
AU - Dormosh, Noman
AU - Heymans, Martijn W.
AU - van der Velde, Nathalie
AU - Hugtenburg, Jacqueline
AU - Maarsingh, Otto
AU - Slottje, Pauline
AU - Abu-Hanna, Ameen
AU - Schut, Martijn C.
N1 - Funding Information: The authors are grateful to all participating GPs and the data managers of the academic network of general practice at VU University medical center in Amsterdam (ANH VUmc) for their time and effort in contributing routine care data for this study. This work was supported by the Netherlands Organization for Scientific Research (NWO) (grant number 628011026), the Hague, the Netherlands. The sponsor did not have any role or influence in study design analysis or reporting. Funding Information: This work was supported by the Netherlands Organization for Scientific Research (NWO) (grant number 628011026), the Hague, the Netherlands. The sponsor did not have any role or influence in study design analysis or reporting. Publisher Copyright: © 2022 The Authors
PY - 2022/10
Y1 - 2022/10
N2 - Objective: Early identification of older people at risk of falling is the cornerstone of fall prevention. Many fall prediction tools exist but their external validity is lacking. External validation is a prerequisite before application in clinical practice. Models developed with electronic health record (EHR) data are especially challenging because of the uncontrolled nature of routinely collected data. We aimed to externally validate our previously developed and published prediction model for falls, using a large cohort of community-dwelling older people derived from primary care EHR data. Design: Retrospective analysis of a prospective cohort drawn from EHR data. Setting and Participants: Pseudonymized EHR data were collected from individuals aged ≥65 years, who were enlisted in any of the participating 59 general practices between 2015 and 2020 in the Netherlands. Methods: Ten predictors were defined and obtained using the same methods as in the development study. The outcome was 1-year fall and was obtained from free text. Both reproducibility and transportability were evaluated. Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (ROC-AUC), and in terms of calibration, using calibration-in-the-large, calibration slope and calibration plots. Results: Among 39,342 older people, 5124 (13.4%) fell in the 1-year follow-up. The characteristics of the validation and the development cohorts were similar. ROC-AUCs of the validation and development cohort were 0.690 and 0.705, respectively. Calibration-in-the-large and calibration slope were 0.012 and 0.878, respectively. Calibration plots revealed overprediction for high-risk groups in a small number of individuals. Conclusions and Implications: Our previously developed prediction model for falls demonstrated good external validity by reproducing its predictive performance in the validation cohort. The implementation of this model in the primary care setting could be considered after impact assessment.
AB - Objective: Early identification of older people at risk of falling is the cornerstone of fall prevention. Many fall prediction tools exist but their external validity is lacking. External validation is a prerequisite before application in clinical practice. Models developed with electronic health record (EHR) data are especially challenging because of the uncontrolled nature of routinely collected data. We aimed to externally validate our previously developed and published prediction model for falls, using a large cohort of community-dwelling older people derived from primary care EHR data. Design: Retrospective analysis of a prospective cohort drawn from EHR data. Setting and Participants: Pseudonymized EHR data were collected from individuals aged ≥65 years, who were enlisted in any of the participating 59 general practices between 2015 and 2020 in the Netherlands. Methods: Ten predictors were defined and obtained using the same methods as in the development study. The outcome was 1-year fall and was obtained from free text. Both reproducibility and transportability were evaluated. Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (ROC-AUC), and in terms of calibration, using calibration-in-the-large, calibration slope and calibration plots. Results: Among 39,342 older people, 5124 (13.4%) fell in the 1-year follow-up. The characteristics of the validation and the development cohorts were similar. ROC-AUCs of the validation and development cohort were 0.690 and 0.705, respectively. Calibration-in-the-large and calibration slope were 0.012 and 0.878, respectively. Calibration plots revealed overprediction for high-risk groups in a small number of individuals. Conclusions and Implications: Our previously developed prediction model for falls demonstrated good external validity by reproducing its predictive performance in the validation cohort. The implementation of this model in the primary care setting could be considered after impact assessment.
KW - Accidental falls
KW - electronic health records
KW - external validation
KW - fall prevention
KW - prediction models
KW - routinely collected data
UR - https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85136281565&origin=inward
UR - https://www.ncbi.nlm.nih.gov/pubmed/35963283
UR - http://www.scopus.com/inward/record.url?scp=85136281565&partnerID=8YFLogxK
U2 - https://doi.org/10.1016/j.jamda.2022.07.002
DO - https://doi.org/10.1016/j.jamda.2022.07.002
M3 - Article
C2 - 35963283
SN - 1525-8610
VL - 23
SP - 1691-1697.e3
JO - Journal of the American Medical Directors Association
JF - Journal of the American Medical Directors Association
IS - 10
ER -