http://spectrum.ieee.org/computing/software/automated-to-death/0
The Flight 124 crew had fallen prey to what psychologist Lisanne Bainbridge in the early 1980s identified as the ironies and paradoxes of automation. The irony, she said, is that the more advanced the automated system, the more crucial the contribution of the human operator becomes to the successful operation of the system. Bainbridge also discusses the paradoxes of automation, the main one being that the more reliable the automation, the less the human operator may be able to contribute to that success. Consequently, operators are increasingly left out of the loop, at least until something unexpected happens. Then the operators need to get involved quickly and flawlessly, says Raja Parasuraman, professor of psychology at George Mason University in Fairfax, Va., who has been studying the issue of increasingly reliable automation and how that affects human performance, and therefore overall system performance.
In many ways, operators are being asked to be omniscient systems administrators who are able to jump into the middle of a situation that a complex automated system can’t or wasn’t designed to handle, quickly diagnose the problem, and then find a satisfactory and safe solution. And if they don’t, the operators, not the system’s designers, often get the blame.
To put this accident in perspective, however, it was only the second fatal crash involving Washington, D.C.’s Metro in its 33 years of operation. In 2008, customers took 215 million trips on the system. Not counting train-vehicle accidents, a total of 27 people were killed and 324 people were injured in train accidents in the United States in 2008. This compares with statistics from 1910, when W.L. Park, general superintendent of the Union Pacific Railroad, asserted that ”one human being is killed every hour, and one injured every 10 minutes.”
Not only has automation improved train safety, but travel by plane, ship, and automobile is safer too. According to Boeing, in 2000 the world’s commercial jet airlines carried approximately 1.09 billion people on 18 million flights and suffered only 20 fatal accidents. The NTSB estimates that traffic deaths in the United States may drop by 30 percent after electronic stability control becomes mandatory in 2012 for automobiles.
