Complexity

Bob Colwell, the 2005 recipient of the IEEE Computer Society/ACM Eckert-Mauchly Award, has a thoughtful "At Random" column in the October 2005 issue of IEEE Computer. [IEEE membership required for free access.]

Substitute "software" for "design" and "programming" for "engineering" (which I believe is a perfectly appropriate substitution), and you have an equally thoughtful essay on the effects of complexity in software and how to deal with them.

When nature is the adversary, all that stands between the engineered product and disaster is the product designers’ foresight, wisdom, and skill...

The real art of engineering, its sine qua non, is in evaluating a proposed design from every angle and vantage point to make sure a design will achieve its goals and prove reliable over its intended lifespan.

When a simulation says a design is working, ask whether the simulation is correct and complete. If a formal proof asserts that some aspect of the design is correct, ask whether the proof itself is trustworthy. What makes you so sure the implementation technology will work as needed? What if the product specification itself has holes or blind spots? What if your product’s buyers like it so much that they begin using it in ways you hadn’t intended—can you anticipate that so the product will gracefully accommodate its new uses?

If the designer knows what she’s doing, the design incorporates existing lore—only the desperate or suicidally naive would attempt a product with no familiar or known-trustworthy components—but it can’t be based only on known components.

The nature of engineering is to never design exactly the same thing twice. Every new design pushes the envelope somewhere: performance, cost, reliability, features, capacity. Inevitably, some aspects of the new design will be outside the existing experience base. That’s the part of engineering you don’t learn in school. And assuming two competing design teams are technically proficient and reasonably well led, it’s what these teams decide to do in these unknown areas that will largely determine which design ultimately triumphs.

So how do you handle the wilderness areas of your design, those places beyond your comfort zone and the safety of your tools and direct experience? I think the answer comes down to how well you handle complexity...

You can’t quantify complexity, but you can feel it. In fact, during a leading-edge design, if anything, complexity feels more real than many of the design’s more quantifiable aspects. Your simulation tools can tell you a microprocessor’s projected die size, but there’s still time before the tapeout. Things happen, and as long as you believe that aspect of the design is on track, then for today it’s mainly a theoretical concern.

Performance and power dissipation feel that same way. But complexity is a monster you can hear breathing right outside your cubicle. It whispers your name during planning meetings, but if you’re not paying attention you may not hear it. Later on, you find yourself at the moment of truth in a project, suddenly realizing that things aren’t right and it’s by no means clear if there is a way to salvage them, let alone what that way might be.

There are some hallmarks to complexity that I’ve noticed over the years. I believe design complexity is a function of the
• number of ideas you must hold in your head simultaneously;
• duration of each of those ideas; and
• cross product of those two things, times the severity of interactions between them...

Complexity makes it a little clearer why project success is so sensitive to unknowns. The very fact that these unknowns are, well, unknown, means that they could inject a wide range of behavior into the design. That uncertainty range increases the number of ideas you must simultaneously consider, and it might also increase the predicted time durations needed.

It doesn’t take much uncertainty to make the complexity-derived behavior range too big to mentally handle. And this is perhaps the most insidious issue of all: When faced with a complex situation, you generally know that you aren’t yet in command of all the necessary details. This in itself isn’t alarming; it takes time to understand what hundreds of people are doing or intend to do. But you can’t leave it like that.

One option is to insist that all of the unknowns be researched and quantified so that your spreadsheet can tell you what to do. But this scheme doesn’t work. Your project doesn’t have enough time or idle engineers to do this much new work; besides, not everything you want to know about your project is knowable, much less quantifiable...

Complex designs are more fragile and lead to more surprises (which are always bad). Complexity leads to longer development schedules; it directly causes design errata; it fosters suboptimal tradeoffs between competing goals; it makes follow-on designs much more difficult; and it’s cumulative, with new designs inheriting all of the complexity of the old and with new complications layered on top.

Increased project complexity also shrinks the available pool of engineers who can help when things go awry. Any competent designer can make easy, straightforward choices, but for truly gnarly situations, only wizards will do. And there are never enough wizards.

Stanford’s John Hennessy once said that it’s always possible to design something so complicated that you can never get it right. As a project leader, are you smart enough to mentally absorb the remaining project unknowns on top of what you already know to the extent necessary to make good decisions on any remaining tradeoffs (some of which haven’t even surfaced yet)?

To some extent, everyone on the design team has this same problem. In considering a design’s overall complexity, you’re making the “smart enough” choice on their behalf as well. If, after deep reflection, you believe you have a practical grasp of the project’s complexities, with enough margin to handle the usual surprises downstream, great. Today will be a good day and you won’t have to think about this again until next week, when you must confront the same issue again. But what if you decide that things seem to have gotten a bit too close to the edge? What do you do then? ...

A design’s complexity must serve a project’s major goals. If your design is complicated but coherent, challenging but understandable, you may have struck a good balance between irreducible complexity and the project’s goals.

Strive to avoid creeping complexity, the kind that arises from unintended interactions among multiple unrelated design decisions. Eschew complicated machinery that isn’t clearly and provably necessary to attain one or more of the major project goals. If you aren’t sure you need some logic in your design, keep asking questions until someone either justifies it or agrees to toss it overboard.

Entropy always drags a project in the direction of increasing complexity; things never get simpler on their own.