Customer dissatisfaction

 You always hurt the one you love. We must really love our customers. We ship buggy code, though that’s not the big problem. We miss our ship dates—not the big problem. We don’t have a clear, broad, and prioritized understanding of customer needs, but that’s not the big problem. We don’t communicate to our customers well and with one voice, but that’s not the big problem. We don’t listen to our customers well and then transfer that information to the right people; but again, that’s not the big problem. No, the big problem is that all too frequently we have no idea when we are tormenting our customers and how badly we’re doing it until it’s too late.

Eric Aside I am severely overstating the case here for dramatic effect. In fact, we do a very good job of listening to our customers and integrating that value into our products. It’s been a huge competitive edge for Microsoft over the years. Regardless, our customers’ expectations have risen as the software market has matured, so to keep our competitive edge we must continue improving. This column discusses the advantages of tracing every code change back to the customers who needed or requested the change.

If we ship bug-free, high-quality code, but it wasn’t what the customer wanted, then customers are dissatisfied. The same is true for shipping undesirable code on time. Even if we do have a clear, broad, and prioritized understanding of customer needs, we still have to ship code that meets those needs or the customer will be dissatisfied. Communicating well and listening well aren’t enough. Nothing counts if we don’t deliver what the customer wanted.

Ignorance is bliss

In fact, good communication and listening actually hurt us. Say we talk to some customers and find out exactly what they want. The customers are pleased that we listened to them, and they know that we know just what they need. Two years later, we deliver a solution that falls short of their expectations. Uh oh. Now the customer is

• Disappointed because the product doesn’t perform as desired.
• Insulted because we wasted their time and raised, then dashed, their hopes.
• Incensed because we broke our commitment to serve them. They may never trust us again.

At least if we had ignored the customer, we could have excused the mistake. “We’re ignorant ninnies” could be our claim. Unfortunately, we did know, we did acknowledge, and we did commit. Even if the commitment wasn’t legal and contractual, it’s a commitment nonetheless, and we broke it.

Too much, too late

Think this doesn’t happen? You are so wrong. We break our commitments all the time. It’s amazing that we still have customers. Our salespeople talk to customers and tell them about our plans. Our consultants visit customers and say they’ll work with product teams to enable certain solutions. Our marketing and product planning people run focus groups and tell customers, “We’re working on it.”

Eric Aside When I say, “We break our commitments all the time,” I mean we fall short of the ideal. We deliver what the customer requested but not what they really wanted or needed. Customers don’t know what they want till they see it. That’s why many Agile methods focus on iterative customer feedback. I use the word “commitment” because it has strong connotations for Microsoft employees. It is too easy to let “little” problems persist in our products, yet it’s those little problems that cause big headaches for customers and I want engineers to feel that.

And we do work on it. Market opportunities drive our product plans and visions. But we don’t close the loop with customers until it’s too late. Heaven forbid that when a customer clicks a button in a working product, we should have time to rethink what we’ve done. Heck, most of the time we’ve already gotten past code complete before customers touch the product.

Eric Aside Let me promote betas and technical previews here, since I failed to mention them in the original column (a significant oversight). Most Microsoft products use betas, but only one or two and often late in the development cycle. However, a number of products are starting to use technical previews and betas early and often—a practice I adore.

In fact, we’re getting really good at closing the loop with customers after we ship. Watson and SQM tell us all about the horrible experiences our customers are having with our shipped products. It’s a phenomenal step forward. We fix the bugs and ship again three months or three years later, and Watson can show us if the annoying problems are gone.

Eric Aside Watson is the internal name for the functionality behind the Send Error Report dialog box you see when an application running on Microsoft Windows crashes. (Always send it; we truly pay attention.) SQM is the internal name for the technology behind customer experience improvement programs for MSN, Office, Windows Vista, and other products, which anonymously aggregate customer usage patterns and experiences. (Please join when you install our software; it lets us know what works and what doesn’t.)

But what about the problems that cause us to break our commitments, throttle our business opportunities, and shred what little trust we still have with our customers? How do we detect those before we ship? How do we prevent those from happening?

Agile delusions

By now the Agile fanatics out there are screaming, “Use Agile methods!” Yeah, well try meeting weekly or monthly with 100 million customers. It’s not as easy as it looks. I’m not saying it’s a bad concept, I’m saying you’re hallucinating.

Sure, you can have the PM or product planner stand in for the 100 million customers, but the chances of them representing all those customers accurately are similar to you winning the lottery. It happens, a lot of people play that game, but you don’t want to build your business or your retirement on those odds.

You need a direct connection back to the customer that closes the loop, like the connection we have with Watson. Any code you write should map back to a specific customer request, market opportunity, business need (like TwC), or customer issue (like a Watson bucket). That way, if a specific question comes up or you want regular feedback on your progress, you know who of the 100 million customers to call.

Eric Aside In fact, we don’t have a direct connection back to the customer with Watson—the information we get is anonymous and aggregated. What we do have is a direct connection to the customer’s problem. Each “Watson bucket” represents and stores a customer issue that thousands, sometimes millions of customers have experienced. So we don’t know who to “call” with a Watson issue, but we can figure out what their problem was. Trustworthy Computing (TwC)—the Microsoft initiative on security, privacy, reliability, and sound business practices—has produced tremendous gains for our customers from Watson data.

Retracing your steps

So how can you tie a specific customer request to a line of code? For bugs, we’ve come close to doing this, but what about feature development? To figure this out, you must retrace your steps:

• Why are you writing that code? What was the requirement or feature?
• Where did that requirement or feature come from? What was the customer scenario?
• Where did that customer scenario come from? What was the market opportunity or customer engagement?
• Who wrote that market opportunity or ran that customer engagement? What is her e-mail alias?

If you can’t trace the work back to a customer, then you’re hopelessly caught up in guesswork about what the customer really wanted. Traceability is the key to any hope of satisfying our customers.

There’s more where that came from

But that’s just the beginning. Like all great pivot points, traceability resolves far more than the immediate need of a relevant customer contact:

• Traceability allows our customers to check on the status of their issues and solutions. Customers can do this somewhat today when they check the status of an error or crash because we now have backward traceability, from servicing to product development. Forward traceability, from product definition to product development, is rewarding for us and for customers.
• Traceability helps us prioritize and make tradeoffs, as well as get the features right. Because traceability can connect us with the business impact of our changes, we can intelligently decide the appropriate number of resources to apply to a feature or change.
• Traceability helps us architect solutions, determine dependencies, and organize projects. This is the most unexpected advantage to me. With traceability, you can know what customer scenarios drove what feature development. So you can know how features are related. This determines the right architecture and dependencies, and with them, the appropriate way to organize the project. Amazing.

Of course, without traceability, the customers have no idea if their needs will be met and when; the product group has no idea what the real business impact will be, and therefore can only guess at tradeoffs; and each group has no idea why they need one feature or another and how they depend on each other. So our lives become an intertwined chaotic catastrophe.

Eric Aside Again, I’m overstating the problem for effect, but I’m not overstating the benefits of traceability. I admit it openly. I’m in love with traceability.

The right tool for the job

So how do you get traceability? Ideally, we’d have a tool that traces scenarios and requirements the same way we track bugs and crashes:

• Salespeople and consultants could use the tool to document customer requirements, scenarios, and commitments.
• Marketing people and product planners could use the tool to submit market opportunities, link them to customer engagements, and define key cross-product scenarios.
• Product planners and PMs could use the tool to consolidate requirements; track duplicates; link related scenarios across products; and draft product-level scenarios, requirements, and feature specs.
• Product groups could use the tool to triage feature requests and track their progress through design and implementation.
• Test teams could relate test cases and bugs to scenarios so that the team could easily see the impact of issues.
• The originators of customer requirements could track progress on their requests, be contacted by product teams to clarify issues or contribute feedback, or contact product teams to update requirements when situations change.

Duct tape and baling wire

Some groups are actually trying to use Product Studio for traceability, but it’s not the full solution yet. Until we have the right tools in place, there are still ways to trace customer requirements and scenarios throughout your entire design:

Eric Aside Product Studio is our internal work-item tracking database. We productized it as part of Microsoft Visual Studio Team System. In the five years since this column was published, most Microsoft divisions have moved to Team Foundation Server (TFS) to track their projects. We are finally close to realizing traceability as I’ve outlined it here.

• When you write a market opportunity document, link to the related customer engagement documents and ensure those documents have contact information. Include your own contact information in the market opportunity document as well.
• When you create a high-level scenario, link to the related market opportunities and customer engagements, and again include your contact information.
• When you write a feature spec, requirements list, or product scenario, link to the related high-level scenarios, requirements, market opportunities, and customer engagements. Be specific about which documents relate to which features—don’t just create a laundry list.
• When you create design documents, link to the specs and other supporting documents. Again, don’t create one long list of references—link to specific information as much as possible. You may even want to pull out specific contacts.
• When you are making tradeoffs or reviewing work, trace all the way back through the links to the people who know. Contact the true source.

Customer satisfaction

Today, we run our business like a child’s game of “telephone.” Each person tells the next what he thought the customer said she wanted. Each step along the way twists and distorts the message. By the time we ship, the customer doesn’t even recognize what she asked for. (This may remind you of the classic cartoon in which the customer wants a tire swing hung from a tree, but instead gets a tree on stilts with a hole in the middle.)

As the product changes and develops, you need to check back with customers to ensure that you’re making the right decisions. Equally important is the ability for customers to check with you whenever their requirements or scenarios change. Without a closed loop, this just isn’t possible.

Traceability closes the loop, but you have to be aware of what to do and remain diligent. Any breakdown along the way risks breaking a commitment. But getting it right means getting the customer just what he needed every time. And that is a reward well worth the effort.