Post by Moses on Feb 21, 2005 11:25:08 GMT -5
February 20, 2005
Who Do You Trust More: G.I. Joe or A.I. Joe?
By GEORGE JOHNSON
IN a story by Isaac Asimov, three technocrats are sitting in an underground cavern stuffed with electronics discussing how, with a computer named Multivac, they won the war.
The attack had come from an enemy that seemed inscrutable - not Shiite fundamentalists or Al Qaeda terrorists but beings from the star system Deneb, a shining light in the constellation Cygnus, who had threatened Earth with weapons of mass destruction.
But the earthlings, relying on the help of an artificial, dispassionate intelligence - this sprawling subterranean computer - had ultimately prevailed.
Recent reports that the Pentagon is planning to spend tens of billions of dollars over the next decade to perfect computerized warfare sound like science fiction. In fact, the plan, Future Combat Systems, was first dreamed up years ago. Its designers envisioned a 21st-century fighting force of automated tanks, helicopters and planes, remote missile launchers and even troops of robot soldiers - all coordinated by a self-configuring network of satellites, sensors and supercomputers. A way to get the human out of the loop.
Already schematized into PowerPoint presentations with tangled colored arrows, talking points and an Orwellian lexicon - "soldier systems," "networked lethality," "war fighter machine interfaces" - the rationale for the effort is laid out in an Army promotional video: "Either you create your future or you become the victim of the future someone creates for you."
Ever since the catapult, warfare has been technology's driving force. Computers were first developed to calculate missile trajectories and break enemy codes. But so far it's been only in science fiction that anyone has dared to turn over decision making to machines.
The Pentagon planners - they've probably all seen the movie "Fail-Safe" - emphasize that in their own futuristic vision, people will remain firmly in charge. The promise is that this won't be like Skynet in "Terminator 2," the self-evolving military electronics that develops a rebellious attitude of its own.
But that leads to a conundrum: The whole point of automation is to rise above human fallibility - knee-jerk decisions, misunderstood orders, cowardly retreats. Machines are faster, more focused, impermeable to propaganda and, at least for now, they don't talk back.
As the thinking machinery continues to evolve, the strategists will keep asking themselves the same question: Is there still a good reason to trust ourselves or should we defer to a computer's calculations?
In the Asimov story, "The Machine That Won the War," published in 1961 in The Magazine of Fantasy and Science Fiction, one programmer admits that as the Denebian battles dragged on, he came to distrust the system. The information the politicians and the generals were providing, the input for Multivac, struck him as so self-serving that he felt compelled to make corrections. So he juggled the data, relying on his own intuition.
Another of Multivac's keepers, charged with analyzing the results of its computations, makes his own confession. Because of bad parts and a shortage of good technicians, he suspected the hardware had become unreliable. He gave the numbers his own spin.
It didn't really matter, as the director of the program makes clear in the end. He more than anyone had been unable to put his trust in a computer. Nor did he entirely trust himself. When it came time to make the ultimate decision - attack here or there, advance or retreat - he threw out reason altogether. He flipped a coin.
Multivac would seem pretty lame stacked next to the machinery the Pentagon has put out to bid. But no matter how advanced the system becomes, how do you know when to believe it? Microprocessors and disk drives malfunction and programs can be buggy or catch a virus, but all that is true of human brains. If the computer says something funny, maybe the fault is with you. So who, or what, makes the diagnosis?
The Pentagon's promotional material doesn't mention whether the contract for the Future Combat Systems, said to be the biggest in American military history, includes a line item for philosophers. But they may be best equipped to judge whether computers, despite their faster speeds, greater bandwidth and bigger memories, are inherently different - less trustworthy than the gut feelings and hunches of their keepers.
John Searle, the philosopher at the University of California at Berkeley whose most recent book, "Mind: A Brief Introduction," came out last year, has argued for decades that the brain is not just a computer strung together from neurons. Whatever is happening in the head - and nobody really knows - it is not computation. Confuse reason with calculation, he argues, and disaster lies ahead.
But that has become the minority view. Attend a conference of the Society for Neuroscience or the Cognitive Science Society and you would be hard-pressed to find anyone who doesn't assume deep down that the brain is some kind of information processor. For the time-being, people excel at certain tasks, like recognizing faces and making sense of ambiguous data, but that may be only because of wiring details and variations in the algorithms - things that eventually could be simulated electronically. The result would be machinery that can do anything a person can, but faster and better.
If that comes to pass, doubting some future incarnation of Multivac might be an act of mutiny. Yet there would always be a nagging suspicion: The machine will have been designed by the imperfect species called homo sapiens. What if we got something wrong?
Who Do You Trust More: G.I. Joe or A.I. Joe?
By GEORGE JOHNSON
IN a story by Isaac Asimov, three technocrats are sitting in an underground cavern stuffed with electronics discussing how, with a computer named Multivac, they won the war.
The attack had come from an enemy that seemed inscrutable - not Shiite fundamentalists or Al Qaeda terrorists but beings from the star system Deneb, a shining light in the constellation Cygnus, who had threatened Earth with weapons of mass destruction.
But the earthlings, relying on the help of an artificial, dispassionate intelligence - this sprawling subterranean computer - had ultimately prevailed.
Recent reports that the Pentagon is planning to spend tens of billions of dollars over the next decade to perfect computerized warfare sound like science fiction. In fact, the plan, Future Combat Systems, was first dreamed up years ago. Its designers envisioned a 21st-century fighting force of automated tanks, helicopters and planes, remote missile launchers and even troops of robot soldiers - all coordinated by a self-configuring network of satellites, sensors and supercomputers. A way to get the human out of the loop.
Already schematized into PowerPoint presentations with tangled colored arrows, talking points and an Orwellian lexicon - "soldier systems," "networked lethality," "war fighter machine interfaces" - the rationale for the effort is laid out in an Army promotional video: "Either you create your future or you become the victim of the future someone creates for you."
Ever since the catapult, warfare has been technology's driving force. Computers were first developed to calculate missile trajectories and break enemy codes. But so far it's been only in science fiction that anyone has dared to turn over decision making to machines.
The Pentagon planners - they've probably all seen the movie "Fail-Safe" - emphasize that in their own futuristic vision, people will remain firmly in charge. The promise is that this won't be like Skynet in "Terminator 2," the self-evolving military electronics that develops a rebellious attitude of its own.
But that leads to a conundrum: The whole point of automation is to rise above human fallibility - knee-jerk decisions, misunderstood orders, cowardly retreats. Machines are faster, more focused, impermeable to propaganda and, at least for now, they don't talk back.
As the thinking machinery continues to evolve, the strategists will keep asking themselves the same question: Is there still a good reason to trust ourselves or should we defer to a computer's calculations?
In the Asimov story, "The Machine That Won the War," published in 1961 in The Magazine of Fantasy and Science Fiction, one programmer admits that as the Denebian battles dragged on, he came to distrust the system. The information the politicians and the generals were providing, the input for Multivac, struck him as so self-serving that he felt compelled to make corrections. So he juggled the data, relying on his own intuition.
Another of Multivac's keepers, charged with analyzing the results of its computations, makes his own confession. Because of bad parts and a shortage of good technicians, he suspected the hardware had become unreliable. He gave the numbers his own spin.
It didn't really matter, as the director of the program makes clear in the end. He more than anyone had been unable to put his trust in a computer. Nor did he entirely trust himself. When it came time to make the ultimate decision - attack here or there, advance or retreat - he threw out reason altogether. He flipped a coin.
Multivac would seem pretty lame stacked next to the machinery the Pentagon has put out to bid. But no matter how advanced the system becomes, how do you know when to believe it? Microprocessors and disk drives malfunction and programs can be buggy or catch a virus, but all that is true of human brains. If the computer says something funny, maybe the fault is with you. So who, or what, makes the diagnosis?
The Pentagon's promotional material doesn't mention whether the contract for the Future Combat Systems, said to be the biggest in American military history, includes a line item for philosophers. But they may be best equipped to judge whether computers, despite their faster speeds, greater bandwidth and bigger memories, are inherently different - less trustworthy than the gut feelings and hunches of their keepers.
John Searle, the philosopher at the University of California at Berkeley whose most recent book, "Mind: A Brief Introduction," came out last year, has argued for decades that the brain is not just a computer strung together from neurons. Whatever is happening in the head - and nobody really knows - it is not computation. Confuse reason with calculation, he argues, and disaster lies ahead.
But that has become the minority view. Attend a conference of the Society for Neuroscience or the Cognitive Science Society and you would be hard-pressed to find anyone who doesn't assume deep down that the brain is some kind of information processor. For the time-being, people excel at certain tasks, like recognizing faces and making sense of ambiguous data, but that may be only because of wiring details and variations in the algorithms - things that eventually could be simulated electronically. The result would be machinery that can do anything a person can, but faster and better.
If that comes to pass, doubting some future incarnation of Multivac might be an act of mutiny. Yet there would always be a nagging suspicion: The machine will have been designed by the imperfect species called homo sapiens. What if we got something wrong?