If ever there was a lesson in the utter uselessness of government data, it's the story circulating today that air traffic control system errors are way up.
It's based on a late-to-the-party story from the Associated Press:
In the 12 months ending on Sept. 30, 2010, there were 1,889 operation errors -- which usually means aircraft coming too close together, according to the Federal Aviation Administration. That was up from 947 such errors the year before and 1,008 the year before that. Before 2008 the FAA used a different counting method.
The FAA administrator says the higher number of known errors is due to better reporting and technology that can determine more precisely how close planes are in the air.
The bottom line: Maybe it's a safe system. Maybe it's not.
In either case, it's not a new story. The Washington Post reported in December that errors were up in 2010 by more than 50 percent, using more data than the AP story did. But Tim Noah of Slate looked into the numbers and found there was some dispute whether the increase was due to a change in reporting within the FAA that removed discipline penalties if air traffic controllers self-reported, as the FAA administrator insisted during testimony at the Capitol this week.
Eventually, Noah ran into the buzzsaw of the "government spokesman," whose missions seem to be to convince you that you didn't care enough about the numbers to go through the process of getting a straight answer anyway.
Brown also gave me the missing numbers for category A and B (i.e., the more-scary) air-controller errors for fiscal year 2009. They added up to 329. That means (I later calculated) that A and B errors dropped by about 26 percent between 2009 and 2010. But that good news is tempered by an increase of errors in the absolute scariest category, A. These rose from 37 to 44, i.e., by 19 percent.
I asked Brown why the FAA didn't answer the Post story by removing the duplicates from the regular database so we wouldn't have to guess what the apples-to-apples trend was for all categories of air-traffic-control errors from year to year. "I understand what you're saying," she replied, "but that's not how we keep the data." But, I protested, you have both databases. You surely have the means to identify every error that gets reported to the FAA--where it happened, what it was. They number fewer than 2,000 per year!
The FAA spokeswoman also said an indication of the safety of the skies is the fact there hasn't been a major air disaster in the country since a regional carrier crashed near Buffalo two years ago tomorrow.
We do know that there were more serious air traffic controller errors at Minneapolis St. Paul International Airport than the year before. One more.
Bob, I'm curious if the increase in numbers last year compared to the 2 prior years has anything to do with "economic recovery"?
While absolute numbers have some meaning they can be misleading if the number of opportunities for error increase significantly. Using an inappropriate sports analogy: If a hockey goalie allows 1 goal in a game that has only 10 shots on goal, but allows 2 goals in a game where there are 30 shots on goal, which is the better performance? In the first case he allows 1 of 10. In the second 1 of 15. His absolute "error" count doubles but his save percentage goes up from .900 to .933.
Do the reports say anything about the traffic level?