Christiana Care Health System (CCHS) had used a machine learning model to inform population segmentation. The initial model used “black box” algorithms to predict risk that care managers didn’t have input on or understand. CCHS leaders and experts wanted an efficient model that they understood and trusted to predict 90-day inpatient admission. CCHS used a feature selection process to build the simplest model possible—and AI insight tools for selecting the best model, setting triggers for action, and explaining how the model worked.
- Feature selection reduced the model complexity from 236 data features to just 16 data features (7 percent of the original data set).
- Both models, the one with 236 data features and the one with 16 data features, had an AUROC of 0.78 and an AUPR of 0.15, suggesting no degradation of predictive performance due to the lower number of features selected.
- CCHS care managers have confidence in the predictive model, and they are successfully using the output of the machine learning tool to engage with an average of 857 distinct members each week, completing more than 2,520 tasks for those members.