The July effect: Real or urban myth?

At teaching hospitals, the learning process restarts each summer. Does hospital quality dip at this crucial point?.

Susan George, FACP, was reviewing some data before patient safety rounds with residents at Saint Vincent Hospital when she noticed a slight uptick in medication reconciliation errors during July and August of last year.

There were no serious adverse events and the discrepancies were typically minor, involving omission of vitamins or as-needed medications rather than more worrisome medications like beta blockers, said Dr. George, associate program director of the internal medicine residency program at the Worcester, Mass. hospital. Still, medication reconciliation is just one example of the many tasks and skills new doctors have to pick up during residency that can greatly impact patient safety.

Illustration by David Cutler
Illustration by David Cutler.

At teaching hospitals, the learning process restarts each summer, when a new brood of freshly hatched medical students fills the hallways and other residents move a rung up on the teaching ladder. The effect of that transition on hospital quality has been a long-standing question. Does it dip at this crucial point in the learning cycle?

Opinions vary widely about whether the phenomenon dubbed “the July effect” even exists, and if it does, about its underlying causes and the best solutions to it. Dr. George believes in the July effect, which is why her hospital has additional checks in place until the new interns and residents are comfortable in their roles. Joanne Conroy, MD, chief health care officer for the Association of American Medical Colleges, is more doubtful, saying it “may fall more into that urban legend category. We've never really been able to prove it. And it's not for lack of trying.”

Until recently that was generally true. A series of studies—mostly focused on specific specialties such as obstetrics or trauma—revealed no summertime pattern. Last August, though, a large-scale study published in the Journal of General Internal Medicine reached a different conclusion. The analysis, based on nearly three decades of death certificate data, identified a 10% spike in fatal medication errors during the month of July in counties with teaching hospitals. Counties without teaching hospitals showed no spike.

Explaining the findings

David Phillips, PhD, the study's lead author and professor of sociology at the University of California, San Diego, credited the size of the database he used, based on death certificates from 1979 to 2006, in explaining why his study was able to isolate a pattern where others have not.

“It's clear that there is a spike,” Dr. Phillips said. “We can say that it's something associated with the introduction of new medical residents.” Beyond that, it will be up to the next set of researchers to identify the precise reasons why the influx of new residents influences medication-related fatalities, he said.

A significant increase was not identified for several other categories of mortalities, including deaths from all causes and deaths related to surgical errors. One possible explanation for that difference might be related to team interactions, Dr. Phillips said.

“In the case of surgical errors, you've got a whole team working and you've got multiple redundancies, checks on the behavior of inexperienced residents,” he said. “But in the case of medication errors, a single resident can prescribe something unchecked by the rest of the team.”

Why would the 10% increase in fatal medication errors, though, vanish after July? Dr. Conroy asked. “If there actually was a July effect, you would expect to see some type of tail into August and September as people got more experience.”

Dr. Phillips' response: Nobody knows how long it takes for medical residents to learn their jobs. “If the average resident learns in two weeks, the trail might stop by the end of July,” he said. Also, it may be that problems do persist, but so subtly that they can't be detected, he noted.

Prior to the JGIM study in 2010, the bulk of the evidence sided with those who say quality can be maintained amid the July reshuffling of residents. For example, one 2007 study in the Journal of Perinatology didn't find any statistically significant difference in the rate of cesarean section or in a litany of delivery-related complications, including severe lacerations and anesthesia-related problems. The study, which relied on a national inpatient data sample from 1998 to 2002, compared complication rates in July with August and June.

The July effect may be difficult to capture because it may not permeate all specialties or types of hospitals, said Robert Huckman, PhD, associate professor of business administration at Harvard Business School.

Dr. Huckman co-authored a 2005 National Bureau of Economic Research analysis which found that the average major teaching hospital had a slight increase in length of stay—1% to 2%—from July through year's end. Mortality also increased by 2% to 4% during the same stretch, according to the unpublished working paper.

The studies to date provide enough suggestive evidence that it's worth continuing to research the July effect, said Don Goldmann, MD, senior vice president at the Institute for Healthcare Improvement. Such an effect is inherently logical, he added. “You'd have to be pretty naïve to say categorically that a bunch of new junior doctors coming onto a ward is without any possible deleterious effect.”

Finding solutions

To mitigate any potential impact, academic medical centers have invested increasing time and resources on lengthy orientations and closer monitoring of residents, Dr. Goldmann noted.

Today's orientation for residents looks much different than when physicians like Dr. Conroy launched their careers. “I showed up July 1 and I was put into an operating room,” she said, recounting her first day as an anesthesia resident in the early 1980s.

First-year residents now arrive a week or two prior to July 1 for a barrage of training that encompasses not only hospital policies and procedures, but also the idiosyncrasies of that facility's computer systems, Dr. Conroy said. At the same time that the new residents are learning the ropes, the other residents are adjusting to increasing levels of responsibility all the way up to the attending level, she said.

That hospital-wide shift, and resulting communication challenges, could play a significant role in the July effect, if it does exist, she noted. “It probably has more to do with how we communicate with each other than the level of experience or expertise,” said Dr. Conroy. For first-year residents, incorporating simulated training sessions as part of orientation can help, she said. The simulations can also allow attending physicians to gain an early sense of which interns may have problems communicating with other clinicians and address those problems.

At Saint Vincent Hospital, part of the orientation includes the use of standardized patients in various clinical scenarios to teach skills like how to break bad news, Dr. George said. A key component is teaching effective handoffs, so the residents know how best to pass along patient information to the next resident.

Recently Saint Vincent also consolidated what used to be a series of handouts into a compact pocket booklet, Dr. George said. The booklet covers a variety of common issues or questions, from administrative to clinical, including the optimal way to dictate notes and strategies to handle pain management.

Each spring, Saint Vincent clinicians also hold a retreat for those first-year residents who are on the cusp of assuming their second-year positions, so they can gain some training in the types of supervisory roles that they will be asked to assume, she said.

Differing skill levels

In any teaching hospital, attending doctors quickly pick up on who are the stars among the newbie residents, as well as who is floundering, Dr. Goldmann said. “It's in the middle ground where there is a lot of variability, in terms of how fast people are learning and getting it,” he said.

The challenge is how to provide a bit more support, possibly through an apprenticeship model, without it carrying the sting of humiliation for the resident involved, Dr. Goldmann said. “That's where the innovation is required,” he said. “How do we know who needs more help? How do we rush help to them?”

One untapped resource, Dr. Goldmann said, are nurses. Perhaps their staffing could be adjusted so savvy nurses could better assist those middling-level residents, he said. “The people who really save the interns are the nurses,” he said. “The nurses know that and a smart doctor knows that.”

At Massachusetts General Hospital, beginning this year, the nurses will start providing their input on the residents' evaluations, along with attending physicians, explained Hasan Bazari, ACP Member, director of the medical residency program. The more eyes and input the better, he said. “Our blind spots are identified only when others point out what we can't see.”

Shorter shifts on the way

Massachusetts General also has been working to restructure its system of oversight and checks to respond to the shorter shifts of first-year residents that will be mandated by new regulations taking effect in July, Dr. Bazari said. Among the requirements, a first-year resident's shift can't extend longer than 16 hours.

New residents will be more rested, but the tradeoff is additional patient handoffs, he said. To deal with that, attending physicians will round twice each day to provide additional supervision. “You don't want a July intern to be handing off to somebody who is equally unprepared and doesn't even know the patient,” he said.

Even more attention will be devoted at Saint Vincent's to patient handoffs, once the new resident shifts begin with the July interns, Dr. George said. The intensive care areas, such as the medical ICU, will be going to 12-hour shifts and the residents will overlap between the two shifts. It's vital that they provide a complete update on each patient's status and care, she said. Communication will have “to become water tight,” she said.

The effectiveness of increased supervision was indicated in Dr. Huckman's study, which found that the increase in mortality from July onward was less pronounced as the size of the teaching hospital increased. Hospitals with a large number of residents may be more proactive, given the proportionate risk involved, he said. “Organizations that can effectively sensitize themselves to the risk and provide supervision as needed may be able to offset the effects that we are seeing.”

Supervision, whether human or electronic, may be particularly helpful in dealing with the specific problem of fatal medication errors, which “should be catchable with better systems,” according to Dr. Bazari. To guard against such errors, various checkpoints have been implemented at Massachusetts General and elsewhere, both through information technology systems and a closer review by pharmacists, he said.

Dr. Bazari counts himself among those who believe that heightened scrutiny, even hyper-vigilance as he describes it, can eliminate any July effect. “People early in the year give very explicit communications about what to do, what kind of dosing,” he said. “There is a lot more explicit teaching and communication. Everyone really focuses and makes sure there is no ambivalence about communication, no assumptions made.”

But that focus mustn't falter as July recedes. “I worry actually just as much at the end of the year,” Dr. Bazari said. There might still be a gap in an intern's expertise depending upon their exposure to a particular clinical situation, he said. “Later in the year I actually worry that people will assume that somebody knows how to do x, y or z.”