Predictive Analytics: Making Patients Safer Through Event Reporting and Prediction

For patients, safety in hospitals and health systems remains a serious concern, justified by the fact that medical errors are now the third leading cause of death in the U.S. Artificial intelligence (AI), and predictive analytics have made a difference throughout many industries, improving a large variety of outcomes and driving improvement. With the help of machine learning, it is expected that hospitals and health systems can also take advantage of AI and predictive analytics to find new opportunities to improve patient safety and outcomes.

Allina Health is a large healthcare system serving the greater Minneapolis/St. Paul area. Determined to improve patient safety in its four large metropolitan hospitals, eight smaller community hospitals, and 85 clinics, Allina Health turned to data analytics to standardize and expand safety event reporting and plans to eventually develop a system of predictive alerts to respond to emerging safety concerns.

PATIENT SAFETY REMAINS A MAJOR CONCERN FOR HOSPITALS

More than 21 percent of people in the U.S. report experiencing a medical error in their own care, and 31 percent report an error in the medical care of a relative or friend.¹ Despite a national push to improve the care in U.S. hospitals, lack of safety and resulting patient harm remain a major concern to hospitals and patients, fueled by the fact that medical errors are now the third leading cause of death in the U.S.²

One mechanism for identifying patient safety issues and patient harm is voluntary event reporting. Unfortunately, despite a reliance on this passive form of surveillance, voluntary event reporting only identifies a small fraction of incidents that threaten patient safety and security. With the growth of artificial intelligence and predictive analytics in multiple industries, including healthcare, a better method to understand and improve patient safety may lie in deeper analysis of existing patient data.

Allina Health is a large healthcare system centered around the Minneapolis/St. Paul area. It includes four large metropolitan hospitals, eight smaller community hospitals, 85 clinics, 5,000 physicians, and approximately $4 billion in revenue. Allina Health is determined to improve the safety of the patients cared for at its facilities by learning from past adverse events, identifying and addressing root causes, and using this information to identify and support interventions to prevent harm.

LIMITATIONS ON VOLUNTARY REPORTING HINDER SAFETY IMPROVEMENTS

The Allina Health Board Quality and Population Health Committee and executive leadership team realized patient safety was not being addressed as effectively as it could be by the health system. Quality meetings were heavily focused on quality measures and did not focus on patient safety as much as was needed, with the safety topic frequently falling to the end of the agenda. As a result, Allina Health’s Board of Directors and executive leadership team formed a separate, dedicated patient safety committee to address the issue and develop a plan for improvement. The committee subsequently grew to include a focus on employee safety, raising the visibility of both patient and employee safety.

After convening the safety committee, Allina Health identified the following challenges when it came to improving patient safety in the health system:

• There was a large variation in culture across Allina Health, and some employees reported not feeling safe in speaking up about issues affecting patient safety.
• Many sites were doing excellent work, but almost no two sites approached patient safety in the same way.
• There was no reliable or consistent way of communicating what was learned from site to site when a safety event occurred. Senior leaders at each location were aware of events at their site but did not know about all the safety events across the system. Consequently, senior leaders lacked insight into the scope and number of safety events in the system, leaving them less informed than they would have been if they had a broader perspective.
• The voluntary safety event reporting process was not capturing all events across the organization, thus limiting the information available about the type of events, and what needed to be done to remedy them.
• Identifying patient safety events was made even more difficult due to the requirements for multiple and differing classifications of events, and inadequate, labor-intensive, and incomplete reporting.

ANALYTICS AND CULTURAL SHIFT SPUR PATIENT SAFETY IMPROVEMENTS

To overcome challenges to patient safety and develop meaningful improvement efforts throughout Allina Health, the patient safety committee drafted these initial strategies as their first steps:

1. Assess safety culture using the Agency for Healthcare Research and Quality (AHRQ) Safety Culture Survey tool and act on the results.
2. Assess and redesign reporting associated with safety events.
3. Improve the identification of safety events through more accurate and comprehensive data analytics.

Establishing a culture of patient safety

The Allina Health patient safety committee began its work by assessing the safety culture using the AHRQ Safety Culture Survey tool. As a result of the insight gained from this survey, the committee identified the need to create a culture of safety throughout the health system that included both a willingness to speak up when patient or employee safety was at risk and a mechanism for widespread communication of safety events across the system.

To encourage speaking up, Allina health instituted the practice of tiered safety huddles—the practice of having daily meetings of key frontline individuals to address issues of safety. During the safety huddle, events occurring in the last 24 hours were reviewed. The discussion included the risk of those events occurring again, as well as other potential events that could occur in the next 24 hours. The information was passed up through the tiered huddle system, which culminates in a daily huddle of all Allina executive leaders. With this new information, Allina Health ensured that patient risks were identified, and staff would be available to intervene as needed to prevent harm from occurring.

It was determined that the safety officer and the entire executive leadership team needed to be informed on every event across the system when it occurred. Once this practice was instituted, it was eye-opening for everyone, as leadership was previously unaware of many of the events that were occurring. Allina Health soon learned that widespread sharing of events requires a balance between communicating early and communicating accurately to avoid misunderstanding and confusion. Some level of validation and fact-checking is required before communicating to effectively facilitate early identification and appropriate intervention for potential safety concerns.

Reporting patient safety events

Voluntary reporting systems are effective in picking up some notable safety events and enable organizations to comply with mandatory reporting requirements. Every health system needs to have a voluntary reporting system, and hospital employees need to be able to bring up safety events if and when they occur. At Allina Health, having regular safety huddles helped to reinforce a culture of safety by providing a non-threatening opportunity for everyone to speak up when he or she felt the conditions were unsafe for patients or staff. The huddles improved communication and contributed to learning from events as they occurred.

However, information gathered from the safety huddles also highlighted the shortcomings of the voluntary reporting system. Because such systems are dependent on individuals recognizing and reporting harm, many events and near misses simply go unidentified when only voluntary reporting is utilized. Since voluntary reporting identifies only a subset of safety events and the degree of accuracy varies widely among organizations, comparisons with other organizations are misleading at best, making benchmarks for safety challenging to develop. Reporting of safety events is also complicated by the lack of consistent data and the numerous ways that the data is classified.

Allina Health realized that without being able to identify and investigate all-cause harm and near misses, it was missing the opportunity to identify and learn from patterns and system failures and use that information to determine how to address these weaknesses in its organization. The health system needed to improve safety event reporting and prediction, by assessing and redesigning the reporting of safety events to include all-cause harm and near misses.

There are various national and state reporting requirements, accreditation reporting requirements, and other programs for voluntary reporting of safety events. However, these requirements do not always match, and the methodology is not always the same. There are many ways to classify events: some are lists, some are classification systems, and some are a combination.

To improve the reporting of safety events at Allina Health, the patient safety committee felt it would be necessary to develop the system’s own interactive, comprehensive, and agile reporting capabilities to identify near misses and accommodate multiple definitions of harm. They turned to analytics to help provide a new system of reporting safety events.

IDENTIFY ALL PATIENT SAFETY EVENTS AND NEAR-MISSES

Leveraging analytics to identify patients in need

Allina Health determined that to identify most, if not all, patient safety events and near-misses, it needed to automate the identification of safety events with the help of data analytics. Building on its experience with the Health Catalyst^® Analytics Platform and broad suite of analytics applications, Allina Health decided to investigate the advantages of using the Health Catalyst Patient Safety Monitor™: Surveillance Module, which uses machine learning to identify and implement triggers that predict when a patient may be harmed or indicate if harm may have occurred.

Triggers are clinical and demographic data points that have been found to correlate with a specific safety event. The triggers that are used will vary based on the type of event, the definition of harm being used, and the level of desired sensitivity, i.e., false positives versus missed events.

The patient safety committee decided to pilot this application and trigger tool concept to see if it could be used to get to all-cause potential and actual harm cases. The intent was to reveal a larger range of harm rather than focusing on one specific type of harm, such as reportable harm. To support this type of investigation, specific definitions of harm are built into the trigger tool, and the application pulls data from the EMR into the tool.

Validating the effectiveness of the trigger tool

The Allina Health patient safety committee decided to perform a patient safety surveillance overview to see how the application performed compared to its current processes in accurately uncovering actual harm events related to three areas (see Figure 1):

• Administration of ephedrine.
• Emergency department (ED) or inpatient readmission within 30-days of surgical procedure.
• Partial thromboplastin time (PTT) >130 after intravenous (IV) heparin administration.

Some of the advantages the safety committee identified in using the analytics application were the ability to organize data by multiple factors such as severity, location, and harm type, which could not be done before. The committee also gained a systemwide view of performance with standardized definitions, and up-to-date information much closer to real-time data than what was previously available.

In general, the difficulty they found was not in the tool itself, but rather in operationalizing it. Initially, the tool uncovered so many additional harm events that they had to spend a lot of time cross-matching information from focus groups to determine if harm had actually occurred, and then learn how to “tweak” the triggers to get more accurate results. The safety committee became a bit overwhelmed by the magnitude of the number of cases that voluntary reporting had not identified previously, but that only strengthened their resolve to improve the ability to predict and prevent patient safety events.

In all three pilot studies utilizing triggers, three themes began to emerge:

1. The need to improve EMR documentation. Non-discrete data (free-text notes) are more difficult to draw data from, and data was frequently missing because documentation was not standardized or was not in a retrievable field.
2. The positive impact on patient safety of having consistent use of standard order sets and protocols that reduce unwarranted variation in patient care.
3. The opportunities to improve care are uncovered by investigating both safety events and near-misses.

The investigation into safety events related to the administration of ephedrine highlighted the need for consistent orders, protocols and order sets, and clear criteria to use in selecting one protocol rather than another. Consistency in dosing and notification criteria make it easier for those carrying out the protocol to do it correctly. It also uncovered the need to document the successful completion of notification and the patient response to the medication.

The analysis of the information obtained about ED or inpatient readmission within 30 days of surgery uncovered new opportunities to intervene more effectively and improve the safety of patients. For example, the safety committee learned:

• 47 percent (111) of patients returned within seven days of their procedure, indicating that improvements could be made in the discharge instructions and educational opportunities at discharge and/or earlier follow up with these patients after their procedures to avoid the need for readmission.
• Multiple returns to the ED or clinic may indicate continuity of care issues or concerns not being resolved during the initial readmit or clinic visit.
• There is a potential opportunity to more effectively manage chronic conditions that seemed to flare up following a procedure.
• Patients returning from a skilled nursing facility due to inadequate care highlighted the need for better communication and transitions of care.

The investigation into patients with a PTT >130 after heparin reinforced the need for consistent documentation of elevated PTT and IV heparin rate changes. Inconsistencies in the documentation were problematic in that they failed to support good communication among caregivers in addition to making the use of triggers less effective in identifying patients who were at risk of harm or being harmed.

RESULTS

The Allina Health patient safety committee determined that the use of analytics and triggers improved patient safety reporting and surveillance, leading to the identification of more actual and potential patient safety events, as well as possible contributing factors. Allina Health’s patient safety improvement efforts have revealed the following:

• Safety huddles are effective in building accountability and a strong safety culture, encouraging employees to speak up about potential problems.
• The trigger tool successfully identified more safety events than were identified by voluntary reporting.
• Investigating the events uncovered opportunities for improvement, enabling the organization to better understand its performance, and adjust documentation in the EMR to portray a more complete picture of the patient.
• Standardizing documentation further improved the identification of safety events and near misses, which in turn identified more causes and interventions to use to avoid future safety events, most notably the elimination of unwarranted clinical variation.
• The analytics application has provided the ability to organize data by multiple factors such as severity, location, and harm type, which could not be done before.
• The committee also gained a systemwide view of performance with standardized definitions, and up-to-date information much closer to real-time data than what was previously available.

“The ability to predict patient harm events by identifying patients at highest risk will allow us to move resources from reacting to safety events to preventing them.”

– Robert Quickel, MD, FACS
Vice President, Surgery and Procedural Care
Allina Health

WHAT’S NEXT

When the pilot teams shared their findings on using the analytics application and triggers with the Allina Health patient safety committee, the committee decided to expand the concept of safety huddles and use them in a tiered system from the frontline through middle management and up through the organization. It was determined that this would further strengthen the safety culture and improve communication across the organization.

While the information uncovered in the pilot was used to improve the care of those three patient populations, the safety committee determined that it needed to dig deeper into additional patient safety topics to determine if the knowledge it gained from the safety event investigation could be used to improve documentation, feed predictive analytics, and be used to accurately predict safety events, thus allowing clinicians to intervene in time to prevent harm for an even greater number of patients.

REFERENCES

1. Institute for Healthcare Improvement, National Patient Safety Foundation, & NORC at the University of Chicago. (2017). New survey finds 21 percent of Americans report personal experience with medical errors [Press Release].
2. John Hopkins Medicine. (2016). Study suggests medical errors now third leading cause of death in the U.S. – Physicians advocate for changes in how deaths are reported to better reflect reality.