Physician-rating websites lack star power

Reviews aren't numerous enough to provide meaningful information.

Physician-rating websites are prevalent, but a recent study found that they're no superstars when it comes to providing patients with enough data to compare doctors.

Across 28 such websites, researchers searched for reviews of 600 U.S. physicians, finding a median of seven total reviews for each doctor who had at least one (although 34% had none). Only 42% of the doctors had one or more narrative reviews, according to results published online Feb. 21 by JAMA.

Headlines about physician ratings—”Can you believe what your patient said about you online?”—piqued the interest of lead author Tara Lagu, MD, MPH, FACP, and her research team, who published their first research on the topic in 2010 in the Journal of General Internal Medicine.

Then and now, they found major limitations to the websites. “We really didn't find an adequate number of reviews, either on any one site or across all 28 sites that we looked at, that would allow you to draw conclusions about the experience of care with any given doctor,” said Dr. Lagu, a research scientist and an academic hospitalist at Baystate Medical Center in Springfield, Mass., and associate professor of medicine at the University of Massachusetts Medical School at Baystate.

She recently spoke with ACP Hospitalist about her findings.

Q: What led you to conduct the most recent study?

A: I was motivated as we were starting to see articles. A [2012] article in the Journal of Medical Internet Research noted a huge increase in the number of reviews for doctors, and then there was a [2014] article in JAMA that said more than 60% of patients are looking for information about their doctors online. We just sort of realized, “Oh, it's time to revisit this topic. Potentially, these sites have changed a lot and maybe they have a lot of reviews now.”

Q: What did you think of your findings?

A: In the original study, more than 70% of doctors had no review on any site. Reviews were very scarce originally. In this new snapshot...I really was anticipating that we would find more reviews, but what we found was that, although there are more reviews than there were before, the issue is that for any given doctor on any site, more than half are going to have zero narrative or star reviews. Among doctors with reviews, even on the sites with the most reviews—Healthgrades and Vitals—we still only found a median of four reviews per doctor. It really does leave this space where people are hungry to read narratives about the experience of care with doctors, and these sites are just not providing them.

Q: What might explain the lack of reviews?

A: Writing a review of your doctor is like writing a review of your mother: You have a long-term relationship with this person. Although your review of your mother may be mixed, you're not going to throw her under the bus because she's your mother and you love her, so as a result, you're going to be very cautious about what you say. This long-term relationship we have with our doctors probably affects the willingness of patients to write reviews about them.

Q: Were there other interesting findings?

A: We had one doctor who had 1,389 quantitative reviews and 140 narrative reviews on Vitals, which was 42% of all the reviews from all the doctors we looked at....That same doctor [also] had hundreds of reviews on Patient Fusion, and that was a site where we had no reviews for any other doctors. So zero reviews for everybody, and then one person had hundreds, was very suspicious. We didn't know how that happened, either....It could be incentives, it could be soliciting, it could be hiring a company to leave fake reviews. It could be any number of possibilities, but it was very strange.

Q: What other patient feedback methods may eventually be an alternative to these commercial sites?

A: Over the last three to five years or so, CMS and the Agency for Healthcare Research and Quality have funded this initiative to systematically collect narrative data, and it's the team that designed the CAHPS [Consumer Assessment of Healthcare Providers & Systems] surveys and the HCAHPS surveys. The plan is to develop a systematic, scientific method for collecting narratives and eventually post those narratives on a public reporting website like Physician Compare.

Health systems have [also] started to try to address this. Health systems realized that they do experience surveys and have access to all the data as part of their public-reporting requirement. Because they have the data, a few systems decided to compile it and post it on the biographical web pages of their doctors. The University of Utah did that first, but others rapidly followed. We also did it at Baystate. You can see some benefits from this because...a health system gets between 30 and 100 reviews per doctor per year, so you're looking at between 30 and hundreds of reviews and many, many narratives.

Q: Are there concerns about bias in health systems' reviews?

A: The first challenge is that it's not like review sites where any patient can go on and write. It's systematically collected survey data, so you have to be one of the patients who [were] selected and you have to receive a survey....The second thing is, of course, the health systems own the data, so they are making choices about what comments get posted. So we don't really know what this means. Most of the sites, including Baystate's, put up a disclaimer that says “We're committed to posting both positive and negative reviews.” We mean it, and the reason we mean it is we know that people will not take it seriously unless the reviews include some negative reviews.