Software geeks know that registers fetch data roughly 10 times faster than the L2 cache, 100 times faster than main memory, and more than a million times faster than hard drives. Smart software engineers work hard to keep all the data for their inner loops in registers or at worst the L2 cache. They keep the rest of their time-critical data in main memory, and they only go to the hard drive when absolutely necessary.
Continually fetching data from main memory or the hard drive while in the inner loop of your most time-critical functions would be madness. I mean, why would anyone in the computer industry be so breathtakingly boneheaded as to deliberately distance data for the most common critical functions? Wouldn’t you want to shake such a person and scream, “What the heck are you thinking?!?”
What is the inner loop of software development? It’s a developer writing code. Placing developers’ keyboards and screens on a different floor from their desks would be absurd. What is the next most inner loop of software development? It’s collaborating with your cross-discipline feature team (Scrum team or feature crew). Yet cross-discipline feature teams are commonly separated by long hallways and entire floors. It makes me want to shake the managers and executives who arranged such seating and scream, “What the heck are you thinking?!?”
It was the best of times
“Yeah, but I sat in a shared space once, and I couldn’t get anything done.” You are absolutely correct.
The worst place for productivity is a shared space amongst people working on unrelated feature areas. The chatter around you is noise—irrelevant, distracting, concentration-killing noise. Sitting in a closed office would be a big improvement—you’re isolated from your feature team, but at least it’s quiet.
The best place for productivity is a shared space amongst people working on the same feature area (typically three to 12 people). The talk around you is pertinent and essential—it’s the relevant information you need to avoid issues, save time, and arrive at the best design and code. That’s why after I created team rooms my development leads (who had individual closed offices) voluntarily chose to move into the team spaces—that’s where all the action was.
Eric Aside
If you have developers in a team room who need quiet to get into a flow, have them wear headphones. Establish rules around interrupting people who are wearing headphones, and have focus rooms where people can make calls, do interviews, and otherwise make noise unrelated to feature work.
As for the ideal team room setup, I let the feature teams configure their own space. It gives the teams more buy-in and lets them arrange the room to their preferences. The only essential is to restrict the space to feature team members. Even one outsider is enough to be a distraction and introduce noise instead of value.
Eight days a week
What difference does colocation of cross-discipline feature teams make to the inner loop of software development? I’ve come across many studies that indicate the benefit is substantial, but none from Microsoft. So when my group was scheduled for an office defrag a couple of years ago, I started collecting data for the six teams reporting to me. I compared the per-week average of feature work completed for the few months before the move (when people worked in offices or cubicles arranged by discipline or office availability) to the average completed for the few months after the move (when cross-discipline teams worked together in team spaces).
One of the six teams was stabilizing for a release and thus doing very little feature work. Even so, that team completed 10 percent more feature work per week on average than before the move. Three of the teams were 20–26 percent more productive than before. One team was 55 percent more productive, and the sixth team was 62 percent more productive.
So let’s recap my little experiment. The five cross-discipline teams of five to 10 people each that were actively working on feature development gained a minimum of 20 percent productivity just by sitting together. In other words, they gained an extra day of a development each week. One team got three extra days per week—and still had two days off for weekends. The teams have maintained those output levels ever since.
Eric Aside
How did one team gain three extra days per week? The members broke down their work into smaller chunks, assigned work to team members on demand instead of in advance, and used the team space to stop and gather together to solve serious design issues whenever they came up.
Tradition…tradition!
Given the overwhelming advantage of locating cross-discipline feature teams together, why would any manager or executive be so boneheaded as to sit them apart? These are software managers and executives—people with intimate knowledge of computers, caches, and latency penalties for fetches to disk. Why would they deliberately place the data sources for the inner loop of product development so far from each other? Rather than claim these managers are imbeciles, I’d claim their reasoning is simply misguided by obsolete practice and out-of-place priorities.
The lean approach to software development—specifying, implementing, and validating a feature before taking on the next feature—is only a decade old at Microsoft. That may sound like a long time, but it’s recent enough to have missed when many upper-level managers were still actively developing software. Some parts of Microsoft still use a pure waterfall model instead of feature crews, improvement teams, Scrum, or Kanban.
Managers and executives without lean experience may think, “Developers mostly work with other developers—therefore, they must sit together to be productive.” That’s true for pure waterfall teams, but team members using any modern and efficient form of software development spend most of their time engaging with their feature team colleagues, independent of discipline.
Managers and executives may also think that engineers should sit with their discipline group for career growth. That’s a great reason, but career mentorship and guidance is part of the outer loop of software development. The inner loop is developing the product—feature teams. Feature team colocation takes priority. Feature teams go in the registers and L2 cache of the team room. Discipline teams can be in main memory down the hall or on a different floor.
You’re so far away from me
If having feature team members separated by a hallway or floor slows you down like pulling data from main memory, then splitting your feature team across cities and time zones is like fetching that same time-critical data from the hard drive. What a disaster! Managers who organize feature teams with members across continents really should be forced to use US mail for all of their communications.
Distributed development is great, and it can be done well, as I describe in So far away: Distributed development. You just shouldn’t split the inner loop of software development across locations. Separate the work across multiple feature teams, and then locate those feature teams together in their local team rooms.
I can’t do this anymore
“Yeah, but what happens when the features or feature teams change? Now I’ve got to move people constantly!” No, you don’t. The best thing to do is keep feature teams together. Like any team, they get to know each other and work better together over time.
Of course, sometimes people come and go, and sometimes you need particular people on particular features. In those cases, keep most of the feature team intact, but alter a couple of the team members as needed. A healthy team will gladly absorb the new viewpoints and talents.
Eric Aside
Over the years, I’ve gradually turned over the membership of multiple feature teams an individual or two at a time. People are happy to join teams that work well together—the team’s positive culture is contagious. My supervisors have always remarked on how my teams maintain their productivity, even when changing key members.
Happy together
People aren’t computers with CPUs, caches, and hard drives. However, people’s productivity does depend on fast and easy communication with the folks they work with most. Microsoft managers and executives have long known this, which is why after every reorg, there’s an equal and aligned office reshuffle. However, those office reshuffles often focus on discipline and reporting structure, rather than on placing people closest to those they work with most—their feature team.
Yes, your discipline team helps you grow, but your feature team helps you ship. Mentoring is important, but it’s not the minute-by-minute central activity of your day. That activity is doing your individual work and working with your feature team.
Easy, cross-team communication dramatically increases productivity. I’ve personally seen it add the equivalent of one to three extra days per week—days we all could really use. There’s no excuse in this modern age of devices and services not get the most out of your time and teammates. Be together to ship together—make your team room the place where magic happens.
Excellent post Mr. Wright! In the last few weeks (months) I have been in the “war room” (aka team room) with my team and the level of information flowing through the room is fantastic. Now that we are back to our offices, only a couple of days, I can see a shift in the communication patterns that is less collaborative and more informative. Instead of a developer saying, “This is what I am thinking about doing will it impact you?” I see more of, “This is what I have done please let me know if you have an issue.” The former being a much more productive way to work in a team. I am looking forward to working in a team room again.
Having worked in both modes, I can say that the benefit "depends on a variety of other factors" and who you asked :). It's not slam-dunk goodness. When we were developing Visual Studio Functional Test SKU, we were building something new from grounds up, but the work was really integrating a lot of core services. The conversation really really helped. I can vouch for significant double digit throughput gains in productivity.
When I moved to the .NET core runtime area, I saw the exact reverse. In that case the moving parts were less and all the "dev" work was core algorithmic. Bug fixes dealt with complex memory dumps, races and poking into dis-assembly. The co-locaiton hurt a lot. Anyone stopping by or even other conversation would reset the mental frame. Headphones do not work!!! Remember it's not cubicle but open space, so people moving around is distraction as well. After I moved to an office things were oh so much better.
So I always say that instead of blindly embracing, you need to ask yoruself, what kind of work your team does and your team's mix. In most cases I see managers and PMs consider co-location always as good. However, if your majority of team comprises of individual contributors working on deep dive stuff, this might turn out to be a bigger issue. So instead of co-location and handing out head-phones, the reverse of giving them office and then having 15-mins standup be more productive.
There is a tendency within the industry to make development less "headquarters-centric" and more distributed. My current org has scrum meetings not just across floors but across offices a couple of thousand miles apart. I think it could be very interesting to use these analogies (transistor level <-> human level) for analyzing the "really distributed" model (completely different locations, time zones, etc) and see whether they hold true or if there are mitigations or other factors that could drastically impact the performance differences. I believe there are distributed development teams that do perform at a high level, although I'm not sure if there are studies on how these same teams perform when their members get together.
Team rooms inflict heavy multitasking for all. "The talk around you" won't let you concentrate on the task most of the time, that's all. I just don't see how this can be more productive, at least for devs during feature work milestones.
PM and QA probably would be more productive in such environment though – so that's where the savings might be coming from.
My team hates it, and it is likely we'll have retention issues. Asking a teammate a question can happen instanly if needed via IM. If a team member does not want to be interrupted IM can be marked busy. Getting constantly interrupted during the day causes many people to work off-hours and at home to get anything done.Maybe it seems like more is getting done, but the reality is different. If the software we have written in the last 10 years is any indication it is worse, not better, and we spend more time in meetings and wasting time communicating simple status then ever before. Don''t get me started on TFS and Scrum – notice what the stock price has done since people started using those 'innovations'. They are great for small services, web pages, and trivial software, in the real world they are a fine way to look busy without doing any significant work. Look at Azure if you want to see a scrum disaster.
I've worked in open spaces, cubicle farms and individual offices. I vastly prefer individual offices and can't fathom why any manager intentionally inflicts open spaces on their workers. I don't feel it necessary to elaborate on all the negatives of open spaces. And no – headphones are NOT a solution.
I went to an elementary school that was using an "Open Concept" floor plan. From Grades 1-6, I could hear the buzz of other classrooms. If you're exposed to that kind of environment every day, you learn how to filter out the noise that isn't aimed at you, and to focus on the elements that do. It's a skill, and it can be learned.
I sit in a single office, in an org that, due to history is likely to always want to be single office. And I wish I could have a team room. I thrive with more human contact and direct communication.
It's funny how, just because "that's how we've always done it at Microsoft", anything different from "our way" is questioned. To me, single offices are the "Kool-Aid". Don't assume what works for you works for everybody.