Comparing machine learning to a rule-based approach for predicting suicidal behavior among adolescents: Results from a longitudinal population-based survey

Introduction: Suicidal thoughts and suicide attempts are one of the most prominent public health concerns in adolescents and therefore early detection is important to initiate preventive interventions and closer monitoring. Method: We examined whether the Machine Learning models Random Forest and Lasso Regression better predict future suicidal behavior than a simple decision rule that classifies every adolescent with history of suicide ideation at baseline as at risk (current practice). We used data from a general population of students in second and fourth year of secondary education in Amsterdam, the Netherlands. Results: Both the Random Forest and the Lasso Regression resulted in slightly better prediction. The AUC of the Random Forest (0.79) and Lasso regression (0.76) were both higher than the AUC of the decision rule (0.64). The Random Forest achieved slightly (but non-significantly) higher sensitivity than the decision rule (0.37 versus 0.34), with the same specificity (0.94). With Lasso Regression the sensitivity increased significantly (0.52), but at the expense of the specificity (0.85). Limitations: The loss of cases after merging the data, the use of self-reported data, confidential data collection and the use of only four questions to measure suicidal behavior. Conclusions: This is the first study applying Machine Learning techniques to predict future suicidal behavior on survey data collected in a general population of adolescents. Our study showed that integrating machine learning techniques in screening practice will result in a small improvement in the ability to predict suicide. The models need to be further optimized to improve accuracy.