Blessed isolation—Better design

 I received plenty of what I’ll call “feedback” on my last column, “Stop writing specs”. According to my critics, design documentation forces design to occur and differentiates a robust architecture from, well, much of our code. Somehow readers got the impression that I was against documented design. Perhaps it was the column’s title.

That impression is wrong. I am for design; I am against waste. I believe if you work with a small multidisciplinary team in a shared collaborative space, on one feature set at time from beginning to end, then you don’t need to write formal specs. Continuing to write formal specs in such an environment is wasteful and should stop.

But what if your feature set is so big that your feature team has more than eight people? What if your team is spread across floors or even continents? What if you first design groups of features, then implement them all, and then test them all? All these cases call for formal written specs. Without them you can’t keep your team on the same page over time or distance.

Breaking up is hard to do

Architecture. Now I can hear naïve naysayers nitpicking, “Then your whole column was pointless. No real product feature is so small or independent that it can be done by fewer than eight people in isolation.” Architecture. “You’re just like those Agile extremists,” the dogmatic dinosaurs would continue. “They talk a good game, but at Microsoft we work on big products for big customers with big issues.” Architecture.

Is there a way to bridge the gap between big products and small teams? Why yes, there is. It’s called, “Architecture.” The whole point of architecture is to break intractable problems into little manageable ones. Architecture, when done properly, delivers the needed isolation.

Doing it well

Unfortunately, there are countless examples of bad architecture and bad architects out there. How do you do architecture right? Funny you should ask.

While creating robust, reliable, and resilient architectures is hard, the process itself is simple:

1. Collect scenarios and requirements the product architecture must satisfy.
2. Ensure those scenarios and requirements are clear, complete, and independent.
3. Map scenarios and requirements to product components that will implement them.
4. Identify interfaces between the product components.
5. Inspect the components and interfaces for failures and vulnerabilities.
6. Document the components and interfaces.
7. Redesign and refactor as issues and new requirements emerge.

Before I get into the details, which hold all the entertainment value, let’s first talk logistics.

There is no “I” in team

I’ve known architects whose lives are out of control. Because architecture is broad by its very nature, there are loads of people involved. Some architects spend their entire days in meetings with stakeholders, followed by nights and weekends doing the actual architecture work. While the commitment is impressive, it’s not advisable and likely unsustainable.

Of course, it’s no better for an architect to work alone and ignore stakeholders. A far better model is the architecture team.

The architecture team is composed of senior engineers from each feature team and chaired by the product architect. They meet regularly, as often as daily at the beginning of a project, and no less than weekly thereafter.

The architecture team as a group performs all the steps I listed. One member is responsible for documenting the information collected and decisions made. (That role can rotate.) Instead of the architect traveling to stakeholders, all but the most senior stakeholders come to the team. This vastly reduces travel time and randomization for the architect.

Some product lines have architecture teams composed of product architects and chaired by the product line architect. This hierarchy works well to cover the most complex cross-product scenarios and requirements.

Step by step

Okay, we’ve got the steps and the people to do them. Now how about those details:

• Collect scenarios and requirements the product architecture must satisfy. This one should be a no-brainer. The key is getting stakeholders to review these areas with the architecture team as needed. Remember to consider the impact of quality requirements (like security), as well as functional requirements.
• Ensure those scenarios and requirements are clear, complete, and separable. Getting clear and complete requirements puts a burden on the architecture team, but should be clear-cut. (Don’t forget performance.) The separable piece is more subtle. You want separable requirements to avoid circular dependencies. However, often multiple scenarios relate to the same underlying requirement. In that case, the architecture team must break up the scenarios into the shared piece and the independent pieces. The shared piece becomes a separate requirement.
• Map scenarios and requirements to product components that will implement them. This defines the architecture, the layering of the components, and their responsibilities. Ideally each requirement should be implemented by one and only one component. That would provide pure isolation and make implementation a snap.

In real life, many components are often involved in many requirements. The components involved in the largest number of requirements are your biggest dependencies. They need to be the most stable and get completed first. Circular dependencies are to be avoided at all costs. Axiomatic design can be a useful tool for doing this difficult mapping step.

• Identify interfaces between the product components. Once you have the mapping, identifying interfaces is easier. If two components work together to implement a requirement, they need an interface between them; otherwise, they don’t. Generally speaking, only requirement-driven interfaces should be defined at the architecture level.
• Inspect the components and interfaces for failures and vulnerabilities. Quality issues around security, reliability, manageability, and so on are often apparent directly in the architecture. Spotting and resolving them or mitigating them up front can save you countless hours later.
• Document the components and interfaces. Yes, I said it. You should write formal documentation for your architecture. Even if your product group works with small multidisciplinary teams in shared collaborative spaces on one feature set at time from beginning to end, those small teams require isolation to function independently. They get that isolation from clearly defined interfaces and component responsibilities. The architecture is shared across teams, time, and distance, so it must be documented.

Once the component responsibilities and interfaces have been debated, designed, debugged, and documented, the architecture is done. It’s done, but it’s not finished. The architecture doesn’t truly get fleshed out till all the feature teams implement it.

• Redesign and refactor as issues and new requirements emerge. Regardless of how smart or thorough your architecture team is, they will always miss issues and get blind-sided by new requirements. That’s why it’s important for them to keep meeting every week. Problems that arise get brought before the architecture team, which returns to first principles and refactors the design as needed. The architecture team consists of senior engineers from each feature team, so the problems will be visible and well understood.

Eric Aside The C# architecture team, led by Anders Hejlsberg, still meets for two hours three times a week, even though the original spec was completed eleven years ago. While the number of team members has varied, Anders says it ideally consists of six people plus one PM who takes notes and keeps the agenda.

Dogs and cats living together

Once the architecture is documented and the components are isolated, individual feature teams can engage free from conflict. Should an unforeseen problem arise between components, the architecture team can quickly address it.

There’s still plenty of bottom-up design left to do. Because the scope is smaller and the impact confined, there’s ample room to experiment and apply less burdensome Agile design techniques, like Test-Driven Development.

Were you to attempt bottom-up design for the whole product, the simultaneous number of conflicts requiring broad redesign would quickly collapse the project under its own mass hysteria. It’s one thing to rethink a single component behind a stable interface. It’s another to constantly change interfaces and responsibilities across a million-line code base.

By using top-down design just enough to provide isolation and bottom-up design enough to optimize collaboration and quality, you get the best of both worlds. By using architecture teams, you coordinate effort, create growth opportunities for your senior engineers, and achieve blessed isolation. It’s a beautiful, peaceful world within your reach.