Call us toll-free: 800-878-7828 — Monday - Friday — 8AM - 5PM EST
by Cheryl Clark, for HealthLeaders Media
Some 25 hospitals, including 520-bed Ronald Reagan UCLA Medical Center in Los Angeles, received a failing grade of F, while another 121 hospitals, including the Cleveland Clinic, got a barely passing D in the release today of the controversial Leapfrog Group Hospital Safety Score, an effort to inform patients and payers which hospitals are most likely to cause avoidable harm.
The grades reflect the risk that a patient will suffer a preventable medical error, an injury, an accident, or an infection while hospitalized. The D and F grades, given today for the first time, “represent the most hazardous environments for patients in need of care,” Leah Binder, Leapfrog Group’s CEO, explained in a news conference Tuesday.
In an interview with HealthLeaders Media, Binder said that in talking with low-scoring hospitals, “Most realize they have a problem with patient safety, but they don’t realize how serious that problem is until they see how their data compares nationally.”
These scores should guide patient choices, she says. And if low scores impact hospitals’ bottom lines, so be it.
“If it were me or someone in my family, frankly, I would hesitate before going to a hospital that scores poorly (below an A) on a hospital safety score, especially if there’s an alternative hospital that got a higher score that meets my needs,” she says.
Executives of several hospitals that received F scores were incensed said they were caught off guard. One declared that Leapfrog was attempting to “extort” hospitals to join Leapfrog’s voluntary reporting system, while others said the methodology is flawed or unknown to anyone except Leapfrog.
“UCLA is clearly not an ‘F’ hospital in quality and safety,” says Tom Rosenthal, MD, the hospital’s chief medical officer. “And if UCLA is not an ‘F’ hospital, it seems to me there must be flaws in the Leapfrog methodology.”
A nonprofit quality improvement group formed by employers 12 years ago, Leapfrog launched its first hospital safety report card in June. That gave 2,651 hospitals an A, B, C, or a “grade pending,” which Leapfrog officials said was a surrogate for a D or F grade, to give the hospital six months for more recent data to show improvement. In today’s update, all “grade pending” scores now are listed as a D or an F.
The grades are based on how each hospital scores on 26 separate measures of safety divided among three categories of harm or risk of harm:
- Preventable adverse outcomes of hospital care such as postoperative respiratory failure, pressure ulcer development, puncture or laceration, foreign object retention, bloodstream infections, or falls and trauma.
- Process measures such as appropriate use of antibiotics or prophylaxis for patients at risk of blood clots.
- Structural measures, such as whether the hospital uses computerized physician order entry systems or staffs an around-the-clock intensivist in its intensive care units.
The measures combine data that hospitals voluntarily report to Leapfrog or to the American Hospital Association with that which is publicly reported through Medicare data from the Centers for Medicare & Medicaid Services Hospital Compare website.
Binder says all 2,618 acute care hospitals that met requirements for a grade—having more than 25 beds, are not critical access or specialty hospitals, and are not located in the state of Maryland—were given a chance to see their raw numbers as well as their letter grades in the weeks and days before today’s public release, and refute them if there was an error.
Of those 2,618 hospitals, 790 received an A, 678 received a B, and 1,004 received a C.
While 58% of the hospitals got the same grade as in the June report card, 23% got a higher grade and 19% got a lower grade. Binder says that even with improvement, most hospitals “have a long way to go” to make their hospitals as safe as possible.
When Leapfrog first released its grades in June, many hospitals and their trade groups were incensed, and criticized many aspects of the scoring system, from the measures it uses to the weights applied to each of the 26 measures. They also accused Leapfrog of not alerting hospitals about the process and being secretive.
The American Hospital Association said the safety scores were inaccurate and advised patients to avoid using them to make decisions about where they should receive care. They even criticized Leapfrog for requiring that hospitals pay a fee of between $5,000 to $12,000 to use their letter grade in marketing materials.
Poorly performing hospitals contacted by HealthLeaders Mediain the last two days used many of the same arguments to refute their scores. They vehemently criticized Leapfrog scoring methodology and the weights given to certain measures. Some hospital officials said they didn’t know anything about the scorecard’s November update and were caught by surprise.
UCLA’s Rosenthal is among the most unhappy. He says that an unfortunate case of one air embolism in a liver transplant patient in the spring of 2010 pulled down the medical center’s score. If that single patient had been excluded, UCLA would have gotten at least a C and maybe an A, he says.
Rosenthal also says the Leapfrog methodology favors hospitals that voluntarily report data to the organization, and because of that, he called the safety scorecard “a form of extortion” to get more hospitals to send their safety indicators to Leapfrog’s national database.
“It certainly feels that way, and you can’t draw any other conclusion when 35% of the [Leapfrog] equation is derived from self-reported measures.”
When Leapfrog released its report card in June, he says, the hospital was listed with a “grade pending” to indicate a D or F, and he should have complained then. Had UCLA officials done that, he says, “Leapfrog probably would have fixed it, or they would have put an asterisk by our score.”
Nancy Foster, vice president of quality and patient safety for the American Hospital Association, last June heatedly criticized the scoring system on a variety of points. But in an e-mail to address the updated report card, she says that Leapfrog “has made some important changes in its scoring methodology, in that they minimize scoring disparities between those that report to AHA.”
However, she says the AHA is “still concerned that they [Leapfrog] continue to use data that is being retired from Hospital Compare because the measures do not provide an accurate picture of the quality efforts in many hospitals today.”
She added that patients should use all tools at their disposal, including talking with their doctors, nurses, friends, and family, in making healthcare choices, and not rely solely on Leapfrog’s safety scores.
Shannon Phillips, MD, quality and patient safety officer at the Cleveland Clinic, regretted that her hospital showed up with a D, which was even lower than the C it received in June.
“Many of these measures come from the public space (reflected in statistics posted on Hospital Compare) and we know our performance in them, and we know many of them have improved,” Phillips says, due to the efforts of the hospital’s dedicated quality improvement team. “But [the data Leapfrog uses] is a rolling three-year average. It takes a long time for data they’re looking at to reflect our current work.”
She says that she has yet to delve into the methodology to see all the reasons the Cleveland Clinic did so poorly. But she notes that one element is that the Cleveland Clinic several years ago stopped voluntarily reporting to Leapfrog.
Doug Patten, MD, senior vice president of medical affairs for 101-year-old Phoebe Putney Memorial Hospital, a 445-bed facility in Albany, GA, says he “patently disagrees” with his hospital’s F grade, saying that’s not reflected in current federally reported data.
“Much of what Leapfrog is using is three or four years old,” he says, “and is based on some proprietary methodology, capriciously assigning adverse grades to someone.”
“Across the board, we’re at or near national and Georgia averages, and in some cases above.”
“It appears that Leapfrog takes great pride in using the facilities that do report on their other [Leapfrog] measures, which they chose. I don’t disagree with the importance of intensivists and electronic medical records and things like that, but if you don’t, it impacts on your scores adversely.”
Binder refutes much of the criticism, saying that after some objections were raised in June, the methodology committee changed the weights so that hospitals that report to the American Hospital Association for CPOE and intensivists receive the same amount of credit, 65 points, as they would if they reported those functions to Leapfrog. In the June scoring, a non-Leapfrog-reporting hospital received only 15 points for each of those measures.
But she’s aware that hospitals receiving low grades will still say that the methodology is invalid. “We have no problem, and are expecting that they will be critical of our methods,” she says.
“But I would strongly suggest that they not lose focus on what’s important, which is reassuring the public they serve that they care about patient safety.
“And they should take a lesson from other industries outside hospitals that are very accustomed to very high levels of transparency. The auto industry has lots and lots of publications that compare different autos from different automakers with negative or positive ratings. But I rarely hear automakers complaining that a particular rating is bad or uses a bad methodology. They wouldn’t waste their breath, because that’s not going to get them more sales. What they do is they get better.”
Some executives for smaller, rural hospitals felt they were disproportionately affected by the methodology. For example, James Barrett, CEO of 38-bed Richardson Memorial Center in Rayville, LA, says that not only did Leapfrog fail to contact him, but smaller, more rural hospitals are disproportionately represented among the low-scoring hospitals.
For example, he lamented, “a lot of the things they surveyed for we don’t have. It’s not a fair process.” Because of that, one bad score on a measure that they are scored on can skew the entire grade downward, he says.
Binder deflected criticism that the methodology is flawed, saying that it was developed by a team of the nation’s leading healthcare quality experts, such as Ashish Jha, MD, Sara Singer, and Lucian Leape, MD, all of Harvard University; John Birkmeyer, MD, of the University of Michigan; Arnold Milstein, MD, of Stanford University; and Peter Pronovost, MD, of Johns Hopkins University.
Binder also says that Leapfrog’s safety score now has its own free mobile app that patients who travel may download so they know with a few clicks the safest nearby hospitals to get care if they’re away from home.
The 25 hospitals that received an F are shown below, followed by the safety score they received in June:
- Wiregrass Medical Center, Geneva, AL – Grade Pending
- Hazel Hawkins Memorial Hospital, Hollister, CA – Grade Pending
- Palo Verde Hospital Blythe, Palo Verde, CA – Grade Pending
- University of California Ronald Reagan UCLA Medical Center, Los Angeles – Grade Pending
- Western Medical Center, Anaheim, CA – Not Scored
- Phoebe Putney Memorial Hospital, Albany, GA – Grade Pending
- Madison Memorial Hospital, Rexburg, ID – Grade Pending
- Jackson Park Hospital and Medical Center, Chicago – Not Scored
- Jersey Community Hospital, Jerseyville, IL – Grade Pending
- Loretto Hospital, Chicago – Grade Pending
- Norwegian American Hospital, Chicago – Grade Pending
- Roseland Community Hospital, Chicago – Grade Pending
- Daviess Community Hospital, Washington, IN – Not Scored
- Richardson Medical Center, Rayville, LA – Not Scored
- County Memorial Hospital, Houston – Grade Pending
- Holy Rosary Medical Center, Ontario, OR
- Clarendon Memorial Hospital, Manning SC – Not Scored
- Wallace Thomson Hospital, Union, SC – Not Scored
- Christus Spohn Hospital Beeville, Beeville, TX – Grade Pending
- Gulf Coast Medical Center, Wharton, TX – Grade Pending
- Renaissance Hospital, Houston – Not Scored
- Renaissance Hospital Dallas, Dallas – Grade Pending
- Renaissance Hospital Terrell, Terrell, TX – Grade Pending
- Buchanan General Hospital, Grundy, VA
- Valley General Hospital, Monroe, WA