Cheaper than printing it out: buy the paperback book.

Out of Control

Danny Hillis has come to the same conclusion. He is serious when he says he wants his Connection Machine to evolve commercial software. "We want these systems to solve a problem we don't know how to solve, but merely know how to state." One such problem is creating multimillion-line programs to fly airplanes. Hillis proposes setting up a swarm system which would try to evolve better software to steer a plane, while tiny parasitic programs would try to crash it. As his experiments have shown, parasites encourage a faster convergence to an error-free, robust software navigation program. Hillis: "Rather than spending uncountable hours designing code, doing error-checking, and so on, we'd like to spend more time making better parasites!"

Even when technicians do succeed in engineering an immense program such as navigation software, testing it thoroughly is becoming impossible. But things grown, not made, are different. "This kind of software would be built in an environment full of thousands of full-time adversaries who specialize in finding out what's wrong with it," Hillis says, thinking of his parasites. "Whatever survives them has been tested ruthlessly." In addition to its ability to create things that we can't make, evolution adds this: it can also make them more flawless than we can. "I would rather fly on a plane running software evolved by a program like this, than fly on a plane running software I wrote myself," says Hillis, programmer extraordinaire.

The call-routing program of long-distance phone companies tallies up to about 2 million lines of code. Three faulty lines in those 2 million caused the rash of national telephone system outages in the summer of 1990. And 2 million lines is no longer large. The combat computers aboard the Navy's Seawolf submarine contain 3.6 million lines of code. "NT," the new workstation computer operating system released by Microsoft in 1993, required 4 million lines of code. One-hundred-million-line programs are not far away.

When computer programs swell to billions of lines of code, just keeping them up and "alive" will become a major chore. Too much of the economy and too many people's lives will depend on billion-line programs to let them go down for even an instant. David Ackley thinks that reliability and up-time will become the primary chore of the software itself. "I claim that for a really complex program sheer survival is going to consume more of its resources." Right now only a small portion of a large program is dedicated to maintenance, error correction, and hygiene. "In the future," predicts Ackley, "99 percent of raw computer cycles are going to be spent on the beast watching itself to keep it going. Only that remaining 1 percent is going to be used for user tasks -- telephone switching or whatever. Because the beast can't do the user tasks unless it survives."

As software gets bigger, survival becomes critical yet increasingly difficult. Survival in the everyday world of daily use means flexibility and evolvability. And it demands more work to pull off. A program survives only if it constantly analyzes its status, adjusts its code to new demands, cleanses itself, ceaselessly dissects anomalous circumstances, and always adapts and evolves. Computation must seethe and behave as if it is alive. Ackley calls it "software biology" or "living computation." Engineers, even on 24-hour beepers, can't keep billion-line code alive. Artificial evolution may be the only way to keep software on its toes, looking lively.

Artificial evolution is the end of engineering's hegemony. Evolution will take us beyond our ability to plan. Evolution will craft things we can't. Evolution will make them more flawless than we can. And evolution will maintain them as we can't.

But the price of evolution is the title of this book. Tom Ray explains: "Part of the problem in an evolving system is that we give up some control."

Nobody will understand the evolved aviation software that will fly Danny Hillis. It will be an indecipherable spaghetti of 5 million strands of nonsense -- of which perhaps only 2 million are really needed. But it will work flawlessly.

No human will be able to troubleshoot the living software running Ackley's evolved telephone system. The lines of program are buried in an uncharted web of small machines, in an incomprehensible pattern. But, when it falters, it will heal itself.

No one will control the destination of Tom Ray's soup of critters. They are brilliant in devising tricks, but there is no telling them what trick to work on next. Only evolution can handle the complexities we are creating, but evolution escapes our total command.

At Xerox PARC, Ralph Merkle is engineering very small molecules that can replicate. Because these replicators dwell in the microscopic scale of nanometers (smaller than bacteria) their construction techniques are called nanotechnology. At some point in the very near future the engineering skills of nanotechnology and the engineering skills of biotechnology converge; they are both treating molecules as machines. Think of nanotechnology as bioengineering for dry life. Nanotechnology has the same potential for artificial evolution as biological molecules. Merkle told me, "I don't want nanotechnology to evolve. I want to keep it in a vat, constrained by international law. The most dangerous thing that could happen to nanotechnology is sex. Yes, I think there should be international regulations against sex for nanotechnology. As soon as you have sex, you have evolution, and as soon as you have evolution, you have trouble."

The trouble of evolution is not entirely out of our control; surrendering some control is simply a tradeoff we make when we employ it. The things we are proud of in engineering -- precision, predictability, exactness, and correctness -- are diluted when evolution is introduced.

These have to be diluted because survivability in a world of accidents, unforeseen circumstances, shifting environments -- in short, the real world-demands a fuzzier, looser, more adaptable, less precise stance. Life is not controlled. Vivisystems are not predictable. Living creatures are not exact. "'Correct' will go by the board," Ackley says of complex programs. "'Correct' is a property of small systems. In the presence of great change, 'correct' will be replaced by 'survivability'."

When the phone system is run by adaptable, evolved software, there will be no correct way to run it. Ackley continues: "To say that a system is 'correct' in the future will sound like bureaucratic double-talk. What people are going to judge a system on is the ingenuity of its response, and how well it can respond to the unexpected." We will trade correctness for flexibility and durability. We will trade a clean corpse for messy life. Ackley: "It will be to your advantage to have an out-of-control, but responsive, monster spend 1 percent of itself on your problem, than to have a dedicated little correct ant of a program that hasn't got a clue about what in the world is going on."

A student at one of Stuart Kauffman's lectures once asked him, "How do you evolve for things you don't want? I see how you can get a system to evolve what you want; but how can you be sure it won't create what you don't want?" Good question, kid. We can define what we want narrowly enough to breed for it. But we often don't even know what we don't want. Or if we do, the list of things that are unacceptable is so long as to be impractical. How can we select out disadvantageous side effects?

"You can't." Kauffman replied bluntly.

That's the evolutionary deal. We trade power for control. For control junkies like us, this is a devil's bargain.

Give up control, and we'll artificially evolve new worlds and undreamed-of richness. Let go, and it will blossom.

Have we ever resisted temptation before?