The success of Agile’s concept of self-organizing teams can be traced back to Austrian economics

The best architectures, requirements, and designs emerge from self-organizing teams.

This is one of the twelve principles behind the Agile Manifesto, and is a key step in changing the culture in an organization as it moves to Agile development and other DevOps principles. This is not a new concept however. Although it has been studied on its own in academia, the underlying psychology of it has been thoroughly detailed in the Austrian school of Economics.

A primer on Austrian Economics

If you are unfamiliar with the Austrian school, then you should know that this is a  school of thought, and not a brick-and-mortar school on some real estate in Austria. The adjective “Austria” refers more to the common origin of many of the early thinkers and founders of this school of thought. This includes the economists Ludwig von Mises, Friederich A. Hayek, and Carl Menger. These thinkers, in turn, derived their opinions, and findings based on the writings of previous economists and philosophers such as Adam Smith, John Locke, and David Ricardo. You may not of heard of many of these names before, but you may have heard of their students: Milton Friedman, Arthur Laffer, Charles and David Koch, Ronald Reagan, and many many others. This school of thought is one that is often shunned by more famous economists like Paul Krugman who follow the Keyensian school of thought, after John Maynard Keynes. The Keyensian school of thought is strong in central planning, and is the school that is most modeled after by the US Federal Reserve and other central banking institutions. Now that you know the general players, lets discuss one of the key tenants in the Austrian School that is so often overlooked by the Keynesian school

Its the knowledge problem

Hayek in particular described the local knowledge problem as the observation that the data required for rational economic planning are distributed among individual actors, and thus unavoidably exist outside the knowledge of a central authority. When the knowledge is allowed to be acted on in a distributed manner (rather than centrally planned), we observe the individual actors working within their own self interest, voluntarily, and using their own unique perspective of the world at hand. This is what he called spontaneous order.

Think of it as this: a group of 100 people, in all their years of living, with different skills in different subjects of varying degrees, will have a superior ability to make aggregate decisions than would a single person deciding minutia for the 100 people. No matter how intelligent the one person is, they are no match for the collective intelligence of the 100, making decisions for themselves, based on their own knowledge. The “manager” cannot have the same quantity of local knowledge as the sum of all the individuals he manages.

Professor Lynne Kiesling of Northwestern University describes the knowledge problem in excellent detail:

Applying these lessons during an Agile transformation and to software development

So, now that we have discussed the concept of the local knowledge problem from an economic view point, let’s discuss it from a software engineering view point, which is really a subset of an economic view point.

Perhaps you have 100 people on your team. To be a software developer, you must have at least a certain minimum level of cognitive and reasoning ability. Most of these people have college degrees (perhaps you have a few who do not), which carries at least 4 years of knowledge from different institutions. They also have life experience and specific industry experience, which is unique to every person. The manager or director, CIO, CTO, or VP of IT who manages them, no matter how educated, or skilled, will never have the same collective knowledge as the sum of those 100 people. As such, it is counter-intuitive to micromanage these developers. Rather, it is more important they all share the same interest and goal: the success of the project. For that is a self-interest for each person as they are paid and rewarded based on their individual merit (or should be in any case).

Thus, the manager would be wise to enable the teams to self-organize, and to decide which work they each are best suited to work on. This requires not just a rubber-stamp approval and lofty speeches. Rather it requires that the manager encourage and enable them to do this. Now, in some organizations, one of the road blocks to true self organizing teams is hyper specialization in skills. For example, you may only have one person who understands a server scripting language, another who is the only one who writes Java, and another who is the only one who knows and understands HTML5, and yet another who is the only COBOL developer. This is how silos develop. We see this most often as distributed vs mainframe, or Java vs .NET teams. The manager must enable and facilitate the teams to to cross-train, and learn from each other. Collaboration between individuals, across skill sets, and across roles is critical. This very issue was the subject of research conducted by Nils Brede Moe, Torgeir Dingsøyr, and Tore Dybå SINTEF ICT, of Norway (nilsm|torgeird|tored)@sintef.no, in their paper Understanding Self-organizing Teams in Agile Software Development, which found that one of the critical barriers for self organizing teams is silos of skill sets. I have personally witnessed this with a large consulting services organization who deployed an army of developers on a project that we had been consulted to help, and none of these developers had more than a single competency in any one skill. This made for a very unproductive, slow, bureaucratic project, as well as a frustrated customer.

There are many methods to help alleviate these silos. This includes:

• Lunch and learns, where new ideas, or skills are proposed by members of the team
• Ancillary classroom training, where a person with one skill is sent to a public class to learn a new, but complimentary skill (i.e. a web designer who knows Photoshop, goes to an HTML5 bootcamp, or a COBOL developer goes to learn eclipse based development and interactive debugging of COBOL using IBM Developer for z).
• Rewarding the teams for certifications beyond their skill sets
• Having team members spend 4 hours per week on exploration type activities on the project (i.e. How could containers help my deployment strategy? Would node.js work better than our current JSP based development?)

Next, make your team accountable. As you gradually have the team self-organize, make them accountable for what roles and responsibilities they take on. For example, when moving from rigid requirements or even Use Case based requirements to User Story based requirements, have the team member responsible for completing a story, be the one to estimate its tasks. After all the most accurate estimate is going to be from the person doing the work, rather than by an architect who has a different skill set than the one tasked with completing the story.

As you move towards time-boxed iterations, ensure that you are doing regular retrospectives. This may seem trivial, but this is vital to your team’s success. Other branches of your organization do this outside of software development: the marketing team does this with SWOT analysis, the accounting team uses audits, the sales team uses weekly, monthly, and quarterly quotas. If you are not learning from your mistakes, you will continue to make them. The retrospective is a critical asset, and by harnessing the knowledge from ALL of your team, the team is better able to deliver value in the next iteration. If only the manager writes down the lessons that he or she learned, there is a titanic loss of information. Rather, enable your team to help steer, collaborate, and provide feedback. Collaborative lifecycle management tools can help facilitate this collaboration such that everyone can see the project plan, can see the defects, the tasks, the requirements, and the project plan. Their knowledge can help refine and improve the project plan such that they can self-assign tasks and requirements to themselves, and can provide their own estimates on time.

References

• The Pretence of Knowledge 1974 lecture at NobelPrize.org, F.A. Hayek
• Understanding Self-organizing Teams in Agile Software Development, Nils Brede Moe, Torgeir Dingsøyr, Tore Dybå SINTEF ICT, Norway (nilsm|torgeird|tored)@sintef.no
• Spontaneous Order,  John Stossel, the Foundation for Economic Freedom