Image by Getty Images
Image by Getty Images

Hospitals reach for the stars

Health systems are focusing on online care ratings of facilities and physicians.


Would patients give your hospital care a five-star rating? As a hospitalist, it can be tough for you to determine and even harder for potential patients to find out.

In recent years, more and more health systems have started posting star ratings for individual physicians as part of the “Find a doctor” feature on their websites. But by and large, filtering the search to hospitalists yields few results.

Many hospitalists haven't garnered enough patient reviews to provide a sample, but the biggest challenge of rating hospitalists is one of attribution, said Ira Nash, MD, FACP, executive director of Northwell Health Physician Partners in Manhasset, N.Y.

Accordingly, his health system has posted reviews of more than 1,500 doctors, all outpatient physicians, which are informed by more than 150,000 Press Ganey surveys at any given time and feature about 45,000 unedited narrative comments. Even the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Survey asks about “the doctors” a patient saw rather than singling out a single doctor in charge.

“Hospitals have struggled, I think understandably, to figure out who's the appropriate person to pin that on,” said Dr. Nash.

Rating individual hospitalists may be tricky business, but health systems are finding workarounds to collect patient narratives and leverage them to improve the patient experience, said Tara C. Lagu, MD, FACP, an academic hospitalist and associate professor at the University of Massachusetts Medical School–Baystate who researches physician reviews.

“Health systems can get the data today, and they can turn that data into improvement efforts tomorrow. That's where I think the cutting edge of this work is right now,” she said.

Real-time data

For hospitalists to get serious about patient reviews, it may take new approaches to garnering feedback.

At Cedars-Sinai Medical Center in Los Angeles, for example, inpatients were surveyed through HCAHPS for more than a decade. But “Up until July of 2018, because of really low survey returns and a lack of useful data, I would say most hospitalists didn't really engage with patient experience,” said chief patient experience officer Alan Dubovsky, MBA.

However, over the past two years, the system, which has multiple groups of hospitalists, developed an inpatient real-time survey, moving to an almost fully electronic process for gathering patient feedback via email or phone shortly after discharge, he said. “Through that, our patient narratives (the comments in particular) have increased dramatically,” Mr. Dubovsky said. “We get about 45,000 comments per year across the system.”

The narrative piece of patient reviews is crucial to understanding their context, said Anish K. Agarwal, MD, MPH, an emergency medicine physician and National Clinician Scholars fellow at the Perelman School of Medicine at University of Pennsylvania in Philadelphia. “Just a one- or five-star [review] is a good place to start, but it's more helpful to know what is driving that rating. . . . Getting at the ‘why’ is oftentimes the challenging part, but online reviews typically provide a place for organic, free text, which provides hints on where to look,” he said.

Those free-text comments don't always come in the form of hospital surveys. With Yelp, Google, and other websites that post reviews, patients don't need a prompt to make their hospital experience public. “These reviews are real-time, anyone can post something—it can be a patient, it can be their family member, a friend, or a loved one—and a lot of them capture insights very different than what is asked on Press Ganey, which is the current standard,” said Dr. Agarwal.

Taking a closer look at online reviews may even improve care, as high HCAHPS scores and Yelp ratings are tied to lower hospital mortality and readmission rates, according to a study published in the March 2013 BMJ Quality & Safety. “There's some evidence that there's a relationship between what patients see in the hospital and their safety outcomes,” said lead author Naomi S. Bardach, MD, MAS, an associate professor of pediatrics and policy at the University of California, San Francisco (UCSF).

In a follow-up study, published in the November 2016 BMJ Quality & Safety, her group found that while patients and family members both focused on safety in their reviews, family members and friends commented on safety more than patients did. “The HCAHPS Survey just asks questions of the patients, so we're missing the family members' perspective,” noted Dr. Bardach, who calls family members “the closest observers of care.”

While the HCAHPS Survey asks about care from physicians, including how well they listen and explain concepts to the patient, it doesn't quite capture the full care experience, said Dr. Lagu. “So much of what we struggle with as hospitalists are system problems. We could take those comments and recognize that it's not just one-patient things; there are patterns,” she said.

In fact, hospital Yelp reviews may cover far more topics than the HCAHPS Survey, according to a study published in the April 2016 Health Affairs. While Yelp reviews included seven of 11 HCAHPS domains, they featured 12 additional domains not covered by the standard survey, including cost of the hospital visit, compassion of staff, family member care, and quality of technical aspects of care.

However, comments on social media platforms don't necessarily provide the actionable feedback hospitals are looking for, according to a study published in the January 2016 Journal of Hospital Medicine. Dr. Lagu and her research team assessed the Baystate Medical Center Facebook page, which solicited feedback and generated comments about staff, inattention to pain control, and hospital parking and amenities.

Only a small number of patients accounted for 30% of the narratives. “It was a lot of data to sift through for not that many novel ideas,” Dr. Lagu said. “This was not the way to do it, but I do think it's a novel idea in the sense that we're really taking what our patients say to heart.”

To get even more real-time feedback, some hospitals are reaching out while patients are still on the wards. In a pilot study, Dr. Bardach tested a text-message intervention in 2017 on one medical-surgical unit at UCSF Benioff Children's Hospital, which asked patients' family members if they had experienced safety issues that day. If they had, a link took them to a mobile-responsive website containing a daily safety report tool, which they could use to submit and categorize a free-form response. More than 100 safety events were reported in a nine-month period, and more than 25% of reports focused on positive safety events. Physicians and staff used the feedback to change their communication with families about medications, as well improve communication between teams.

Machine-learning techniques may also improve understanding of patients' hospital experiences. Of more than 51,000 hospital reviews posted on Yelp from 2005 to 2017, 41.9% gave one star and 31.6% gave five stars, for an average rating of 2.8 stars, according to a study published online in February 2019 by the Journal of General Internal Medicine. An algorithm that pulled the most common words found that the single word most correlated to negative reviews was “told,” whereas the words most correlated with positive reviews were “great” and “friendly.”

Dr. Agarwal, who led the study, said that using the word “told” in a negative review speaks to a breakdown in communication. “I feel like the word ‘told’ invokes an image of a one-way street. . . . There might be a component of not really understanding and not really having a true conversation—more like just one person speaking at somebody,” compared to the words “explained” or “answered,” he said.

Of course, sorting through comments is not without its challenges. Just the time spent poring over comments and organizing them is about a full-time equivalent's worth of work, according to a return-on-investment analysis by the group at Cedars-Sinai. To address this issue, the health system works with a patient feedback-processing company to categorize, theme, and look for trends in patient comments for performance improvement purposes and training, Mr. Dubovsky said.

Taking action

If hospitals are going to take action on their reviews, timely feedback is crucial.

Until the switch to electronic surveys in July 2018, Cedars-Sinai would mail an 80-plus-question survey to patients after discharge. “It would take forever, and we'd get back just a handful per physician. . . . By the time patients got to the end of it, very seldom did we get comments,” Mr. Dubovsky said. Now, a 14-question email survey is sent within hours of discharge, and many hospitalists are on track to receive a few hundred surveys per year, Mr. Dubovsky said.

Each month, each hospitalist receives individual feedback reports, including comments, from all the patients who were discharged under his or her name. Hospitalists can also see each other's ratings and comments. “It's on our internal drive, and at any given point anyone can log in,” said Mr. Dubovsky.

To actually do something with the feedback, Cedars-Sinai engaged with Practicing Excellence, an online skills lab that helps physicians improve their patient experience skills. Some of the online modules are specific to hospitalists, such as how to explain to patients what a hospitalist is and how they function, and many hospitalists have used the training, said Mr. Dubovsky.

The health system also offers coaching for hospitalists who need help improving. “We've had them shadow and talk through ways in which to improve the patient experience, so that's been extremely helpful as well,” he said.

Physician-to-physician coaching can improve patient experience scores, according to a 2019 paper published in Patient Experience Journal. At Scripps Health in San Diego, emergency medicine physician and chief experience officer Ghazala Sharieff, MD, MBA, coaches seven hospitalist groups in both group and one-to-one sessions. After the coaching sessions, the hospitalist groups have improved their HCAHPS scores, sometimes to dramatic effect.

The secret to success is keeping it simple, said Dr. Sharieff, who is also corporate vice president at Scripps. “We can throw data at physicians right and left and it doesn't mean anything because some question the statistical significance—’My patients are sicker than your patients' or ‘The survey count is too low,’” she said. “But we made it very simple.”

She asks hospitalists to do three things: knock, sit, and ask. Knocking helps patient privacy, and sitting at the bedside has been shown to increase patients' perception of time spent with their physician, Dr. Sharieff said. “And then the one thing that has been resonating is asking the patient their greatest concern for the day.”

Sitting has also been a theme at Cedars-Sinai, where one hospitalist was consistently getting feedback on an inability to connect with patients, Mr. Dubovsky said. After watching a video on the power of sitting down when talking to patients, the hospitalist's scores improved significantly, he said. Dr. Sharieff added that making more folding chairs available has facilitated sitting at the bedside at her institution.

The “ask” piece of her coaching comes from her own clinical experience more than a decade ago. A child in the ED had presented with leg pain and fevers at three other EDs, each of which found a normal X-ray. By the time she saw him, he had no leg pain or fever, but she decided to ask the child's grandmother what her greatest concern was. “She said, ‘I'm worried he's got cancer.’ My lightbulb went off: roving leg pain, negative X-ray, unexplained fever. I said, ‘You know what? Let's get a blood test,’” Dr. Sharieff said. “And that child had leukemia. I would have missed it had I not asked that simple question.”

In addition, she asks hospitalists to do “one thing different” than they are already doing to impact patients' care experience. For example, one hospitalist chose to end all patient encounters by asking if there's anything he can explain before leaving, since he gave a lot of information, she said. His scores went from the 77th percentile to the 90th percentile.

Another hospitalist's “one thing different” was to put his hand on each patient's shoulder, make eye contact, and say, “You are in really good hands.” When he was asked about the impact of this change in practice, he said he could physically feel the patient relax under his touch.

Not all hospitalists are as proactive when it comes to patient feedback. Although those at Cedars-Sinai have a new appetite for improvement, most still question how much of the feedback applies to them directly, Mr. Dubovsky said. “In every hospitalist meeting I go to, I still get people that say, ‘Well, this patient had four hospitalists. I was just the final one that got tagged to her,’” he said.

In part due to these and other concerns, health systems still aren't quite to the point of publicly reporting reviews of hospital physicians.

Although Cedars-Sinai has enough individual-level data to do so, there has been some reluctance, Mr. Dubovsky said. One reason is that hospitalists and ED physicians make the point, unique among them, that their patients have virtually no way of choosing them. “The pushback is fair,” he said. “At the same time, we have a lot of patients who are in house who are interested in feedback about the physicians that are seeing them.”

Overall, as patient and family feedback moves toward more narrative, real-time processes, health systems will have more opportunities to improve care, said Dr. Lagu. “We are bringing the patients into the process of improvement, and that in and of itself might be the innovation and the cultural change,” she said.