Resolved by design

There you are, sitting in front of a whiteboard while some obsessive dweeb goes on incessantly about some design issue. You try to ground the discussion and bring the conversation back to the actual work, but this neurotic nag won’t let anything go. “What if this happens?” “We haven’t considered this.” “We don’t know that.” “We can’t start yet.” Analysis paralysis. Meanwhile the clock is ticking and you’ve got to ship.

We all know design is important. We all know quality design is essential to quality products. But darn it, at some point you’ve got to code. At some point you’ve got to ship. The best design in the world is just wallpaper if you can’t get it coded in time, and I don’t hear many managers saying, “Take all the design time you need.”

What is good enough?

Yet the other extreme is just as evil. Jumping in and coding with just a token of forethought on a fleeting whiteboard leads to tremendous rework, tons of bugs, and a fragile framework for future development. Most analysis of defects reveals between 40–50% of them were in the design. Skipping the design step is no solution, but how much analysis is enough? You know when you’re done coding—when is the design sufficient to begin development?

To me, the lack of a well-defined design process is the single most debilitating developer debacle. It’s not like you don’t know it’s important—coding is far easier, faster, and dramatically less error prone with a clear and complete design. No worries, no quandaries, no surprises, and none of the infamous Homer Simpson, “Doh!” But did you learn a design process in college? No, they don’t teach it. Did you learn one when joining Microsoft? Not likely; most groups are pretty ad hoc about design, and many hardly practice design at all. Ah, but your good buddy I. M. is here to help.

Eric Aside I lay out a series of design steps and approaches below that cover all design aspects. However, how do you know which steps you can skip, and how do you know when you’ve completed a step? After all, you want to do just enough design—not too much or too little. The answer is confidence—you want to proceed to implementation with confidence. For each step you ask, “Are we confident about this design aspect?” If so, you can skip it. If not, you work on that aspect until you are confident. Be honest with yourself and your team. You don’t need to understand every detail, but you do need to know what you’re doing.

Design complete

The most important aspect of a design process is completeness. Completeness tells you what is enough. You don’t want to do too much or too little. Software has multiple dimensions. There are internal workings and external interfaces. There are static concepts and dynamic interactions. All of these dimensions must be covered to create a complete design. Additionally, there are different design tools to help you work with the different areas and levels of abstraction. Here’s a table to help make sense of this.

Dimensions of software design

	External	Internal
Static	PM spec API definition	Dev spec Test-Driven Development
Dynamic	Use cases Scenarios and personas	Sequence diagrams State diagrams, flow charts, threat and failure modeling

Watts S. Humphrey showed me this table while he was visiting Microsoft once. My team and I filled it in with common design practices and the movement from quadrant to quadrant.

Most design processes spiral inward with a clockwise flow from the lower-left quadrant of the table, starting with the high-level design steps:

• Scenarios and personas Describes at a high level how customers will interact with the software.
• PM spec Describes the external pieces of the puzzle, which include requirements, dialog boxes, menus, screens, data collection, and other features.
• Dev spec Specifies the class hierarchy and relationships, component stack and relationships, and anything else needed at a high level to describe the structure of the implementation. This spec is also referred to as the design document (a misnomer, given its limited scope) or architecture document.
• State diagrams, flow charts, threat and failure modeling Describe and control complex interactions between the objects and components in the system.

A few comments on these initial, high-level design steps:

• Each step has a limited, well-defined scope. By separating and covering all four design quadrants, you won’t need to rely on lots of words and repetition to fully describe the design. Be brief, reuse prior examples, and rely on simple diagrams or pictures where appropriate.
• Not every step is necessary in every design. Some designs have little external interface. Some designs don’t carry state. Use your own best judgment.
• The external quadrants have a wide audience (all disciplines). The internal quadrants have a purely technical audience.
• Unified Modeling Language (UML) provides ready-to-use diagrams for many of these areas, and there are tools that make creating UML diagrams easy. For example, Visio has a ton of built-in UML diagram types.
• Threat and failure modeling are far easier to get right by reusing component relationship diagrams that you should already have. Use Excel spreadsheets to categorize and rate threats and failures and to specify mitigations.

Eric Aside We now use a threat-modeling tool to do this work. You can find some on MSDN.

Details, details

Just as each step shown in the preceding table drove the next clockwise turn around the design dimensions at a high level, the same process occurs at more detailed levels. You basically spiral into implementation through the following steps, which are closer to the center of the table:

• Use cases Simple descriptions of how actors use the system to perform tasks. Visio even has built-in shapes for use cases if you like pictures.

Eric Aside The term use cases is often used to describe very high-level scenarios, not the simple low-level ones I’m talking about here. Sorry for any confusion.

• API definition When you have the use cases, specifying the application programming interface (API) correctly becomes far easier. This step solidifies the contract between components and objects. Then you can actually start coding the implementation.
• Test-Driven Development TDD provides an ideal framework for methodically and robustly creating a simple, cohesive, and well-factored implementation, complete with a full suite of unit tests.
• Sequence diagrams For particularly complex functions or methods, use sequence diagrams to clarify code constructs.

A few comments on the whole design process I just outlined:

• The process has no unnecessary complexity or added steps. Each step is well defined and leads into the next step with all the information needed.
• As with any process, you may be tempted to skip steps to save time (like skipping straight to TDD without a PM spec, dev spec, or use cases). That’s fine if the outcomes of those steps are trivial, but you invite peril if the outcomes are tricky (and around here outcomes tend to be tricky).
• You know what enough is and when you are done.

Show me what you’re made of

Okay, so I glossed over two bits:

• How do you get the right scenarios and personas?
• How do you bridge the gap between a PM spec built from those scenarios and personas to a dev spec with a class hierarchy and relationships and a component stack and relationships?

There are two ways to get the right scenarios and personas. One way is through direct contact with customers. The second way is top-down, starting with the following even higher-level steps in the design process spiral:

• Market opportunities and personas (external-dynamic)
• Product vision document (external-static)
• Product architecture (internal-static)
• Subsystem interaction diagrams or flow charts (internal-dynamic)

Then continue spiraling into the scenarios and personas as discussed previously.

Eric Aside When you’ve got hundreds of millions of customers, thousands of engineers, and billions of dollars in the balance, I’d advise using a rigorous approach to high-level design. If that upsets you, I’d advise staying away from large, successful projects. When that much money is involved, projects are driven by politics or process.

Mind the gap

Bridging the gap between a PM spec (or higher-level product vision) and a dev spec (or higher-level product architecture) requires mapping functional requirements (like features) to design parameters (like classes or components). There are two straightforward methods for doing this:

• Design patterns Recognize that you’ve solved this problem before, and reuse the old design.
• Axiomatic design Use this method for brand-new designs and even for cleaning up and evolving old designs. The steps are fairly clear-cut:

1. Create a table in Excel or Word, with the rows listing the functional requirements and the columns listing the design parameters. The internal cells should be blank to form a matrix. This is known as the design matrix.
2. Ensure that each functional requirement is written to be orthogonal to the others. This often means breaking up complex requirements into basic pieces. Adding new requirements just means adding new rows (nice).
3. One by one, list design parameters that would satisfy each functional requirement. Often a design parameter (a class or component) helps satisfy more than one requirement. Mark an X in each cell under the design parameter column that helps satisfy the corresponding functional requirement. The design matrix should now have Xs scattered throughout the cells.

Okay, so by now we need an example. A simple one is the design for a faucet. The two orthogonal functional requirements are to control flow and temperature. Consider two alternative designs, each with different design parameters: separate hot and cold water taps, and a single lever with tilt controlling flow and rotation controlling temperature. The two corresponding design matrices look like this:

Two alternative design matrices for a faucet

	Hot tap	Cold tap			Tilt lever	Rotate lever
Flow	X	X		Flow	X
Temperature	X	X		Temperature		X

4. Rearrange the design parameters so that no Xs appear above the diagonal of the design matrix. You may need to rethink how classes or components can better match to a smaller number of requirements to get this to work.

Notice in our example that the separate hot and cold water taps have Xs above the diagonal, while the single-lever design is nicely factored into a diagonal design matrix. The single-lever design produces a far better faucet.

5. The resulting design matrix will document a nicely factored design. If all the Xs are on the diagonal, the design is completely decoupled and will be a cinch to implement. Any Xs below the diagonal indicate dependencies and can help you guide the proper order of development. Any Xs left above the diagonal that you couldn’t remove are nasty, cyclical dependencies that should be handled with great care.

Your recipe for success

So there you are—a step-by-step guide to minimal, yet complete and robust design. It may seem a bit daunting, but really each step is straightforward. By using a methodical approach, you won’t miss anything, you can schedule it, and life won’t seem quite so open-ended when the pressure is on.

This type of complete approach also cures analysis paralysis. Follow every step and you’re done. If people try to add extra steps or bring in extra requirements, you’ve got documentation to halt their advances.

As I mentioned in my last column, A software odyssey—From craft to engineering, dropping bug counts by a factor of a thousand was a necessary step toward meeting the quality that our customers are demanding. The other is discerning the requirements and creating a detailed design that meets them. By using a complete design approach along with an engineering-based implementation, we can exceed our customer’s expectations for quality and value, turn around our industry, and stomp our competitors in the process.