Automated Machine Learning For Risk Prediction Of Incisional Hernia In Abdominal Surgery Patients
Ankoor A. Talwar1, Abhishek A. Desai1, Phoebe B. McAuliffe1, Robyn B. Broach1, Tony Liu1, Lyle Ungar1, John P. Fischer1
1University of Pennsylvania, Philadelphia, PA
Background: Incisional hernia (IH) is a common, morbid long-term complication following abdominal surgery. Incidence is 500,000 annually. Our group previously developed a logistic regression model to predict risk of IH. The purpose of this study was to determine if automated machine learning (AutoML) is superior to logistic regression (LR) in assessing risk of incisional hernia, and understanding which clinical features are salient for IH formation.
Methods: This retrospective cohort study reviewed adult patients who underwent colorectal surgery at our institution from 2005 - 2016. Development of IH was noted. Two sets of clinical features were tested: a limited set of 18 features previously studied, and an expanded set of 246 clinical features. The four models generated were: LR with limited features, LR with expanded features, AutoML with limited features, and AutoML with expanded features. Primary outcome was the AUC generated by each model. Secondary outcomes included differences in predictions at varying true positive rates and Shapley values for feature importance.
Results: 20,516 patients were included, of which 12.3% developed IH (n=2,519). 67% of patients were used to train the models (n=12,871). The other 33% were the test cohort (n=6,340). AUCs were calculated: LR limited 0.599, LR expanded 0.682, AutoML limited 0.706, AutoML expanded 0.747 (Figure 1). At a true positive rate of 0.8, the AutoML expanded had a False Positive Rate (FPR) of 0.64, compared to AutoML limited FPR of 0.71, LR expanded of 0.78, and LR limited of 0.82 (all p < 0.0001). Shapley values of both the limited and expanded feature sets revealed the most critical factors predicting risk of IH were age and BMI followed by others (Figure 2). However feature importances shifted moving from limited to expanded features.
Conclusion: Automated machine learning is better at predicting IH development over logistic regression. Predictions generated from expanded features are also better than from limited features. Finally, the importance of clinical features shift when using a larger feature set.
Back to 2022 Abstracts