The Wednesday post displayed a screen shot from Placentia-Linda Hospital's website; the screen shot showed a current four-minute wait time at the hospital.
The Thursday post talked about the wait times at Scripps Mercy Chula Vista, and how they had decreased to 25 minutes.
So if I think I'm going to get sick, should I go to Placentia? After all, statistics show that I'll be seen 21 minutes more quickly than I would in Chula Vista. And statistics don't lie, do they?
Um, you'd better ask Samuel Clemens about that:
Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: "There are three kinds of lies: lies, damned lies, and statistics."
(Incidentally, it's uncertain that Disraeli actually said this, but Clemens was a damned liar, and a damned good one.)
When looking at a single statistic in isolation, such as emergency room wait times, there are a number of things that need to be examined. Here are a few examples:
- The two wait time statistics are measured differently. Although we don't know the details of Placentia's measure, it is a measure over a brief period of time. The quoted Chula Vista time was an average, possibly taken over a month, or several months. What would happen if I looked at Placentia's number right after a bunch of AYSO soccer games? Or at 11:00 pm on July 4?
- When you measure an elapsed time, you have to define the start and the end time. What are the start and end times for the Placentia and Chula Vista measurements? Does the time end when the clerk calls you up to the counter? Does it end when a nurse sees you? Does it end when a doctor sees you? Let's face it - if you want to, you could set a target emergency room wait time of 1 second and consistently meet it, provided that you define your start and end times properly.
- I know nothing about the hospital in Chula Vista, but I have gained significant knowledge about the Placentia hospital by driving up Rose Drive, and my extensive knowledge suggests to me that Placentia-Linda is a relatively small hospital.
- But even if two hospitals are of the same size, their locations could easily affect any statistics that are generated. Is one hospital next to an AYSO soccer field? Is one next to an urgent care clinic? Is one in a community where everyone has lost their jobs, and their health care coverage?
As it turned out, my Thursday post contained a supposedly off-topic reference to my alma mater.
I consulted the leading authority on hospital reputations, Yelp. (Hey, I'm a Reed College graduate; U.S. News and World Report is not an option.)
In fact, if you followed the link, you will see that Reed's concerns about U.S. News and World Report's annual college survey (or a similar survey from Money magazine) raise similar objections to the isolated use of numbers.
Reed College has actively questioned the methodology and usefulness of college rankings ever since the magazine's best-colleges list first appeared in 1983, despite the fact that the issue ranked Reed among the top ten national liberal arts colleges. Reed's concern intensified with disclosures in 1994 by the Wall Street Journal about institutions flagrantly manipulating data in order to move up in the rankings in U.S. News and other popular college guides. This led Reed's then-president Steven Koblik to inform the editors of U.S. News that he didn't find their project credible, and that the college would not be returning any of their surveys.
A subsequent president, Colin Diver, provided additional thoughts:
Rankings, he says, are grounded in a "one-size-fits-all" mentality. "They are primarily measures of institutional wealth, reputation, influence, and pedigree. They do not attempt, nor claim, to measure the extent to which knowledge is valued and cultivated" on each campus. Reed doesn’t rank its students. "Why should we participate in a survey that ranks colleges?" he asks.
And anyone who agreed with Reed's approach seemed to have their concerns validated during the first year that Reed refused to participate:
The year the college refused to submit data, the magazine arbitrarily assigned Reed the lowest possible in several categories and relegated the college to the lowest tier in its category, the most precipitous decline in the history of its ratings.
And then:
The following year, responding to widespread criticism of its retribution, the magazine trumpeted Reed in its "best colleges" press release as being new to the "top tier" of national liberal arts schools.
For the record, Reed would prefer not to be in the survey at all, but then U.S. News' survey would be considered incomplete, so the magazine prefers to keep them in there.
And they perform other surveys, including surveys of hospitals and law schools. The latter has not endeared U.S. News and World Report to an American Bar Association committee:
A special committee of the American Bar Association has issued a report criticizing the impact of U.S. News & World Report rankings on law schools. The report says that the magazine's methodology "tends to increase the costs of legal education for students" because colleges are rewarded for spending more, that the rankings "discourage the award of financial aid based upon need" because law schools are rewarded for enrolling more students with high LSAT scores and so will award merit aid, and that for similar reasons "the current methodology tends to reduce incentives to enhance the diversity of the legal profession."
U.S. News and World Report raised objections:
Robert Morse, who leads the college rankings at U.S. News, said that the ABA never contacted the magazine to discuss the rankings, and he offered his own critique of the study. "I think the ABA report is one sided and does not tell the whole story," he said. "The ABA ignores that it's highly likely that there are other factors besides U.S. News behind rising tuitions at law schools, why law schools are offering more merit aid and that some law schools aren't accepting enough at-risk low-LSAT students. In other words, it's easy to blame U.S. News for many of the negative practices at law schools. Law schools and the ABA need to take far more direct responsibility for these trends."
Of course, not everyone agrees with Reed College and the ABA. Take Grace College in Winona Lake, Indiana:
It has historically been among the top schools of its size and listed in U.S. News & World Report as one of America’s Best Colleges.
But nearby Kosciusko Community Hospital in Warsaw doesn't display its emergency room wait times. Maybe that's why U.S. News and World Report characterizes the hospital as "average."
