Cheaper than printing it out: buy the paperback book.

Out of Control

Despite my sunny forecast for the network economy, there is much about it that is worrisome. These are the same concerns that accompany other large, decentralized, self-making systems:

  • You can't understand them.

  • You have less control.

  • They don't optimize well.

As companies become disembodied into some Barlowian cyberspace, they take on the character of software. Clean, massless, quick, useful, mobile, and interesting. But also complicated and probably full of bugs no one can find.

If the companies and products of the future become more like software of today, what does that promise? Televisions that crash? Cars that freeze up suddenly? Toasters that bomb?

Large software programs are about the most complex things humans can make right now. Microsoft's new operating system had 4 million lines of code. Naturally Bill Gates claims there will be no bugs in it after the 70,000 beta-test sites are done checking it.

Is it possible for us to manufacture extremely complex things without defects (or even with merely a few defects)? Will network economics help us to create complexity without any bugs, or just complexity with bugs?

Whether or not companies become more like software themselves, it is certain that more and more of what they make depends on more complex software, so the problems of creating complexity without defects becomes essential.

And in an age of simulations, the problem of verifying the truthfulness of a simulation is the same type of problem as testing massive complex software to determine whether it is or is not flawless.

David Parnas, a Canadian computer scientist, developed a set of eight criticisms of President Reagan's "Star Wars" (SDI) initiative. He based his criterion on the inherent instabilities of extremely complex software, which is what SDI essentially was. The most interesting of Parnas's points was that there are two kinds of complex systems: continuous, and discontinuous.

When GM tests a new car on its track field, it puts the car through its paces at different speeds. It will test how it handles a sharp curve going at 50, 60, 70 mph. To no one's surprise, the car's performance varies continuously with the speed. If the car passed the curve test at 50, 60, and 70 mph, then the GM engineers know -- without explicit testing -- that it will also pass at all the intermediate speeds of 55 and 67 mph.

They don't have to worry that at 55 mph the car will sprout wings or go into reverse. How it behaves at 55 will be some interpolated function of what it does at 50 and 60 mph. A car is a continuous system.

Computer software, distributed networks, and most vivisystems are discontinuous systems. In complex adaptive systems you simply can't rely on interpolated functions. You can have software that has been running reliably for years, then suddenly, at some particular set of values (63.25 mph), kaboom!, the system bombs, or something novel emerges.

The discontinuity was always there. All the neighboring values had been tested but this particular exact set of circumstances had not. In retrospect it is obvious why the bug caused the system to crash and perhaps even why one should have looked for it. But in systems with astronomical numbers of possibilities, it is impossible to test every case. Worse, you can't rely on sampling because the system is discontinuous.

A tester can have no confidence that unexamined values in extremely complex systems will perform continuously with examined values. Despite that hurdle there is a movement toward "zero-defect" software design. Naturally it's happening in Japan.

For small programs, zero is 0.000. For extremely large programs, zero is 0.001, and falling. That's the number of defects per thousand lines of code (KLOC), and it is just one crude measure of quality. The methods for attaining zero-defect software borrow heavily from the Japanese engineer Shigeo Shingo's pioneering work on zero-defect manufacturing. Of course, computer scientists claim, "software is different." It duplicates perfectly in production, so the only problem is making the first copy.

In network economics the major expense of new product development stems from designing the manufacturing process and not designing the product. The Japanese have excelled at designing and improving processes; Americans have excelled at designing and improving products. The Japanese view software as a process rather than product. And in the dawning network culture, more and more of what we make -- certainly more and more of our wealth -- is tangled up in symbol processing which resembles code more than corn.

Software reliability guru C. K. Cho admonished the industrialist not to think of software as a product but as a portable factory. You are selling -- or giving -- a factory (the program code) to others who will use it to manufacture an answer when they need one. Your problem is to make a factory that will generate zero-defect answers. The methods of making a factory that produces perfectly reliable widgets can be easily applied to creating a factory that makes perfectly reliable answers.

Ordinarily, software is constructed according to three centralized milestones. It is first designed as one big picture, then coded in detail, then finally, near the end of the project, it is tested as an interacting whole. Zero-defect quality design proceeds by thousands of distributed "inchstones," instead of a few milestones. Software is designed, coded, and tested daily, in a hundred cubicles, as each person works on it.

The zero-defect evangelists have a slogan that summarizes network economics: "Every person within a company has a customer." Usually that customer is the coworker you hand your work off to. And you don't hand off your work until you've done the milestone cycle in miniature -- specifying, coding, and testing what you made as if you were shipping it.

When you ship your work to your customer/coworker, she immediately checks it and lets you know how you did, reporting errors back to you, which you correct. In essence, software is grown from the bottom up in a manner not unlike Rodney Brooks's subsumption architecture. Each inchstone is a small module of code that works for sure, and from which more complex layers are added and tested.

Inchstones alone won't get you zero-defect software. Underlying the zero goal is a key distinction. A defect is an error shipped. An error corrected before shipping in not a defect. Shingo says, "What we absolutely cannot prevent are errors, but we can keep those errors from generating defects." Therefore, the task of zero-defect design is to detect errors early and rectify them early.

But that much is obvious. The real progress comes from identifying the cause of the error early and then eliminating the cause early. If a worker is inserting the wrong bolt, institute a system that prevents the incorrect bolt from being inserted. To err is human; to manage error is system.

The classic Japanese invention for error prevention is a "poka-yoke" system -- making things foolproof. On assembly lines, cleverly simple devices prevent mistakes. A holding tray may have a specific hole for every bolt so that if there are any bolts left the operator knows he missed one. An example of poka-yoke for software production is a spell-checker that doesn't allow the programmer to type a misspelled command or even to enter an illegal (illogical) command. Software developers have an ever widening choice of amazingly sophisticated "automatic program correction" packages that check ongoing programming to prevent typical errors.

State-of-the-art developer tools perform meta-evaluations on a program's logic -- "Hey, that step doesn't make sense!" it says -- eliminating logical errors at the first chance. A software industry trade magazine recently listed almost a hundred error test and removal tools for sale. The most elegant of these programs offer the creator a legitimate alternative, just as a good spell-checker does, to correct the error.

Another poka-yoke of great importance is the modularization of complex software. A 1982 study published in the IEEE Transactions on Software Engineering revealed how the same number of lines of code broken up into smaller subprograms would decrease the number of faults, all other things being equal. A 10,000-line program in one hunk would exhibit 317 faults; when broken into three subprograms, 10,000 lines would total a lesser 265 faults. The decrease per subdivision follows a slight linear function, so fragmenting is not a cure-all, but it is a very reliable trick.

Furthermore, below a certain threshold, a subprogram can be small enough to be absolutely free of defects. IBM's code for their IMS series was written in modules of which three-quarters were entirely defect free. That is, 300 out of 425 modules had zero defects. Over half of the faults were found in only 31 of the modules. The move toward modular software, then, is a move in the direction of reliability.

The hottest frontier right now in software design is the move to object-oriented software. Object-oriented programming (OOP) is relatively decentralized and modular software. The pieces of an OOP retain an integrity as a standalone unit; they can be combined with other OOP pieces into a decomposable hierarchy of instructions. An "object" limits the damage a bug can make. Rather than blowing up the whole program, OOP effectively isolates the function into a manageable unit so that a broken object won't disrupt the whole program; it can be swapped for a new one just like an old brake pad on a car can be swapped for a better one. Vendors can buy and sell libraries of prefabricated "objects" which other software developers can buy and reassemble into large, powerful programs very quickly, instead of writing huge new programs line by line. When it comes time to update the massive OOP, all you have to do is add upgraded or new objects.

Objects in OOP are like Lego blocks, but they also carry a wee bit of intelligence with them. An object can be similar to a file folder icon on a Macintosh screen, but one that knows it's a folder and would respond to a program's query for all file folders to list their contents. An OOP object could also be a tax form, or an employee in a firm's database, or an e-mail message. Objects know what tasks they can and can't do, and they communicate laterally with each other.

Object-oriented programs create a mild distributed intelligence in software. Like other distributed beings, it is resilient to errors, it heals faster (remove the object), and it grows by incrementally assembling working subunits.

The 31 error-filled modules mentioned earlier that were found in IBM's code beautifully illustrate one characteristic of software that can be used to achieve sigma-precision quality. Errors tend to cluster. Zero Defect Software, the bible of the movement says, "The next error you find is far more likely to be found in the module where eleven other errors have already been found, rather than in the module where no errors have been found." Error clustering is so prevalent in software that it is known as the cockroach rule of thumb: where there is one error seen, another twenty-three lurk unnoticed.

Here's the remedy, according to the Zero bible: "Do not spend money on defect-prone code, get rid of it. Coding cost is nearly irrelevant compared to the cost of repairing error-prone modules. If a software unit exceeds an error threshold, throw it out, and have a different developer do the recoding. Discard work in progress that shows a tendency toward errors because early errors predict late errors."

As software programs mushroom in complexity, it becomes impossible to exhaustively test them at the end. Because they are discontinuous systems, they will always harbor odd corners or a fatal response triggered by a one-in-a-million combination of input that eluded detection of both systematic and sample-based testing. And while statistical sampling can tell if there are likely to be faults left, it can't locate them.

The neo-biological approach is to assemble software from working parts, while continuously testing and correcting the software as it grows. One still has the problems of unexpected "emergent behaviors" (bugs) arising from the aggregation of bugless parts. But there is hope that as long as you only need to test at the new emergent level (since the lower orders are already proven) you have a chance -- and you are far ahead of where you'd be if you had to test for emergent bugs along with deeper sub-bugs.

Ted Kaehler invents new kinds of software languages for his living. He was an early pioneer of object-oriented languages, a codeveloper of SmallTalk and HyperCard. He's now working on a "direct manipulation" language for Apple Computers. When I asked him about zero-defect software at Apple he waved it off. "I think it is possible to make zero defects in production software, say if you are writing yet another database program. Anywhere you really understand what you are doing, you can do it without defects."

Ted would never get along in a Japanese software mill. He says, "A good programmer can take anything known, any regularity, and cleverly reduce it in size. In creative programming then, anything completely understood disappears. So you are left writing down what you don't know....So, yeah, you can make zero-defect software, but by writing a program that may be thousands of lines longer than it needs to be."

This is what nature does: it sacrifices elegance for reliability. The neural pathways in nature continue to stun scientists with how non-optimized they are. Researchers investigating the neurons in a crayfish's tail reported astonishment at how clunky and inelegant the circuit was. With a little work they could come up with a more parsimonious design. But the crayfish tail circuit, more redundant than it perhaps needed to be, was error free.

The price of zero-defect software is that it's over-engineered, overbuilt, a bit bloated, and never on the edge of the unknown where Ted and friends hang out. It trades efficiency of execution for efficiencies of production.

I asked Nobel Laureate Herbert Simon how zero-defect philosophy squared with his concept of "satisficing" -- don't aim for optimization, aim for good enough. He laughed and said, "Oh, you can make zero-defect products. The question is, can you do it profitably? If you are interested in profits, then you need to satisfice your zero defects." There's that complexity tradeoff again.

The future of network economics is in devising reliable processes, rather than reliable products. At the same time the nature of this economy means that processes become impossible to optimize. In a distributed, semiliving world, goals can only be satisficed, and then for only a moment. A day later the landscape has changed, and another upstart is shaping the playing field.