A New Way of Ranking Medical Schools: Social Mission

.
In the June issue of the Annals of Internal Medicine, Fitzhugh Mullan, Candice Chen, Stephen Petterson, Gretchen Kolsky, and Michael Spagnola, mostly from the Department of Health Policy at the George Washington University (and one from the Robert Graham Center) report on “The social mission of medical education: ranking the schools”. This study, sponsored by the Josiah Macy, Jr. Foundation as part of the Medical Education Futures Study (MEFS) is the first report to look at this area, and should be eye-opening to those who assume that “good” medical schools are “good” at everything. Most ranking systems, most notably those of US News and World Report, are based on NIH research funding, grades and test scores of entering students, “competitiveness” (how low a percent of applicants they accept) and reputation. Obviously, the grades and test scores are related to competitiveness and reputation is a tautology, because it reinforces itself. It should depend upon what you are looking at, of course. I addressed this in “Rankings of Medical Schools: Do they tell us anything?” (September 25, 2009), and observed that what they tell us is who does well in what is measured, and that this should only be important to us if those are the outcomes we value.

Mullan and colleagues evaluate different outcomes, the degree to which medical schools meet their “social mission”, or to put it another way, the degree to which they produce the physicians that will take care of the American people. More to the point, since it can be argued that most medical school graduates take care of some American people, physicians who will take care of those people who need it the most because they don’t already have doctors. This means largely those in poor communities, rural communities, and minority communities (and especially those communities that are two or three of these). They look at 3 characteristics of graduates: 1) what percent of their graduates are practicing primary care, 2) what percent of their graduates are practicing in designated Health Professions Shortage Areas (HPSAs), and 3) what percent of their graduates are members of underrepresented minority groups? This is pretty straightforward, and they take two other steps to try and ensure that this is an accurate reflection.

The first is that they examined students, in the graduating classes of 1999-2001, 8 years after graduation. As I have pointed out several times (Primary Care and Residency Expansion, January 7, 2010; Funding Graduate Medical Education, May 25, 2009), the “credit” medical schools claim for students entering “primary care” residencies includes all those entering internal medicine programs, the vast majority of whom will enter subspecialty training. By looking at students 8 years after graduation, after they have completed residency and subspecialty fellowship training, they are able to get a much more accurate picture of who is actually doing primary care. Similarly, it also means that those who are practicing in HPSAs have been doing it for several years. Overall, public schools did much better than private schools, and Southern, Midwestern, and Western schools better than Northeastern schools.

In terms of underrepresented minority students, the percent of graduates, for public medical schools, is compared to the percent of the underrepresented minority population for the state, while for private schools, which are presumed to draw from a more national base, it is the national percentage (26.5%). Thus, for example, the University of Iowa has a positive ratio with 8.1% minority students in a state that is only 6% minority, while the Universidad de Puerto Rico en Ponce has a negative ratio because, even though their students are 82.5% underrepresented minorities, their “state” is 98.8%. The underrepresented minority scores for the 3 historically black medical schools, Morehouse, Meharry and Howard are so high, compared to the national average (as they are all private) that they are easily the top 3 in the overall social mission score. This tends to wash out the significant differences between them on the other two areas. For example Meharry does well in producing primary care physicians (49.3%, or 2 standard deviations [SD] above the national mean) compared to Howard, which at 36.5% is only 0.19 SD above the mean; Howard, however, does better at placing students in HPSAs (33.7%, +0.78 SD) than Meharry (28.1%, +0.12 SD). Ponce, despite its negative underrepresented minority score and also a negative primary care physician score (-0.31 SD*), ranks #9 nationally in total social mission score. This is based on its high rate of physicians practicing in HPSAs (43.8%, +1.94 SD), because so much of its service area are HPSAs.

The data can be analyzed in a number of ways. Osteopathic schools have a much higher rate of producing primary care doctors, but none were in the top 20 because their percent of underrepresented minorities are low. Adding only the two dimensions of primary care and HPSA practice shows only 7 schools with a standardized score above 3, all of them public allopathic schools and 4 of them “community based” medical schools with a specific commitment to primary care (as is Wright State, the #4 ranked school in overall social mission). A few top NIH research schools (4, to be exact) “defied the trend” and were in, at least, the top quartile of social mission scores, again all public schools. Other than the historically black schools, private schools were nowhere to be seen.

The schools that traditionally do well on rankings such as that of US News tended to be at the bottom of this scale. They are overwhelmingly private (14 of the bottom 20) and generally highly NIH-funded. Comments from the leaders of those schools, unsurprisingly, tended to disparage the study and its methods, and to assert, essentially, that “our school does well on all of its missions.” If those missions include the social missions of meeting the health needs of the American people by producing minority and primary care physicians, and those that practice in underserved areas, they clearly do not. And, while some are better than others, no medical schools are doing very well at enrolling underrepresented minorities (except the historically black schools and the Puerto Rican schools) or at producing physicians for rural areas at anything approaching the percent of Americans who live in those areas (Primary Care and Rural Areas, April 28, 2010).

This is not to say that other missions of medical schools, such as biomedical research, cutting-edge medical care, and training of the future generations of academics, including the MD/PhDs who will be laboratory-based researchers, are unimportant. Lawrence G. Smith and Veronica M. Catanese emphasize this point in their accompanying editorial, “The Many Missions of Medical Schools”, as well as noting various possible ways in which the production of physicians who fulfill a social mission might have been underestimated by Mullan, et. al. They also note, as I have above, that different schools do better or worse on the different social mission measures, but also that success in the social mission needs to be pursued, as success in all other missions: “The lack of concordance among the 3 elements of Mullan and colleagues' social mission score suggests that medical schools that accept this mission—as they must—cannot define social mission narrowly. They must have multipronged initiatives and not simply 'wait' for programs aimed at recruiting and retaining underrepresented minority students, or at specifically incentivizing primary care, to bear fruit.”

The key point is that the data produced by Mullan and colleagues that shows that schools which are historically highly ranked do relatively poorly in social mission is not due to a flawed methodology. “The level of NIH support that medical schools received was inversely associated with their output of primary care physicians and physicians practicing in underserved areas.” It is because these schools do not emphasize the characteristics that they are measuring, combined into the concept of social mission, nearly as highly as they do their other missions and do not put as much energy, time, or especially money into them. Mullan et. al. conclude that “Some schools may choose other priorities, but in this time of national reconsideration, it seems appropriate that all schools examine their educational commitment regarding the service needs of their states and the nation. A diverse, equitably distributed physician workforce with a strong primary care base is essential to achieve quality health care that is accessible and affordable, regardless of the nature of any future health care reform.”

The authors note that “Medical schools, however, are the only institutions in our society that can produce physicians”. It is up to the people of the US, particularly the communities in need and the policy makers who represent them, to decide how high a priority producing physicians who will meet our social need by practicing in specialties, in areas, and with populations who do not have doctors, and how to use the public coffers to achieve this. The time is long past, however, for these characteristics not to be measured. We can no longer, in self-indulgence or ignorance, assume that those schools that are the “best” on US News rankings because of NIH research funding, selectivity, and “reputation” are the best in every area. In producing the doctors most needed by this nation’s most needy, they are, with few exceptions, mostly the worst.

*Like the other Puerto Rican Schools, Ponce has a low production of family physicians, ranking 117 out of 128 medical schools for producing FPs in the 10 years from 1999-2008, as reported by the American Academy of Family Physicians (AAFP) annual analysis.
.