The Bureaucratization of Agile
Why Bureaucratic Software Environments Aren’t Agile
Introduction
Back in 2001, a small group of programmers launched a revolution in software engineering by publishing the Manifesto for Agile Software Development (commonly called the Agile Manifesto) [1] and, shortly thereafter, The Twelve Principles of Agile Software [2].
There are many lessons to be drawn from the Manifesto and its Principles. One of them is that we should favor lightweight processes that reduce managerial drag and trust motivated workers to decide how best to do their work.
With this lesson in mind, we should be moving toward ever simpler and lighter ways of structuring our workforces so they can respond quickly to change and become more effective at their tasks.
Instead, we have an ever-expanding number of complicated frameworks that require elaborate administrative roles, checkpoints, best practices, reviews, and other hallmarks of the distant Waterfall era. When you examine these frameworks and see the sheer number of process-guardian roles, it’s natural to wonder if one goal of the frameworks is to smother an organization in bureaucracy.
Unsurprisingly, any bureaucracy tasked with reorganizing itself will likely create a derivative bureaucracy. After all, as the saying goes, everything looks like a nail when all you have is a hammer. If bureaucracy is the tool that a system has, then solutions look like different forms of bureaucracy. This mentality may be why frameworks with layers of administrators are popular in bureaucratic systems. We’re only human, and it’s natural that we should stick to what we know.
The Manifesto and its Principles heralded a brighter future that still seems distant all these decades later. Indeed, about sixty percent of Agile teams report difficulties with command-and-control management hierarchies above them [3]. From these issues comes a question:
What has gone wrong?
The Original Intent of Agile
The original intent of Agile was to move organizations from heavy and unresponsive Waterfall systems to lightweight, quickly responding ones. Instead of requiring months or years to deploy software, we would deliver quickly and pivot based on learning from our frequent deployments. Software changes would no longer require glacially paced approval from change-request boards. Instead, we would respond to change and deliver greater customer value over shorter time horizons. Everyone would win.
There was only one downside, but it was significant for some people. Gone would be the belief that we were predictable and consistent. In its place would be the acceptance, however reluctantly, that uncertainty is an inherent and unavoidable aspect of our work [4]. This belief was probably a bridge too far for many schooled in Waterfall or more predictable industries. If those same people were in positions of managerial authority, as was often the case, their natural response would be to desire something offering the illusory comfort of certainty. From there, it was a short step to the proliferation of bureaucratic frameworks.
Why do Bureaucracies Form?
Let’s begin with first principles: Why do bureaucracies form in our workplaces? There must be some overarching reason that many of us work in bureaucratic systems. There are many such reasons, and we’ll cover some of them next.
The Dynamic Duo: Max Weber and Frederick Taylor
In the late nineteenth century, German Sociologist Max Weber laid the foundation of modern bureaucracies. The details are beyond the scope of this paper, but Weber’s ideas can be summarized in a few key points.
- Organizational control is achieved through rules and regulations.
- Work is subdivided into specialized skills, and workers perform their specialization according to strict processes.
- There are hierarchies of management layers, with each layer responsible for workers in its domain and answering to the layer above them in a strict chain-of-command fashion.
These ideas probably look familiar. They’re essentially the blueprint for how we structure our workplaces today. Weber believed that following these principles would provide the most effective form of managing large organizations. [5]
A contemporary of Weber was Frederick Taylor, who devised his Scientific Management approach to structuring work. Today, we know it as Taylorism. Like Weber’s methods, Taylorism depends on enlightened managers deciding how work will be done and who will do it. It can be summarized as follows. [6]
- There is only one “right way” to do a job, as determined through scientific principles that management alone understands. Workers shouldn’t be trusted to apply their own rules of thumb for doing their work.
- Managers should decide who is best for given jobs and not allow workers to decide for themselves.
- Managers should do the planning and training, and workers should simply perform their assigned tasks.
As with Weber, Taylorism may look distressingly familiar in today’s software organizations, even in those that claim to be Agile. At their core, both approaches take a dim view of workers. They assume layers of administrators must oversee workers who can’t be trusted to do their work without constant oversight.
Organizations have arranged themselves according to Weber and Taylor for over a century, something we see in today’s bureaucratic software structures. It should be evident that such structures are entirely antithetical to modern Agile philosophy. Indeed, one of the more profound Agile Principles states this:
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. [2]
We certainly don’t trust workers to get the job done if we micromanage them because of low regard for their skills and integrity. Agile methods make an essential point of allowing teams to self-organize. It’s hard to see how that’s possible if workers are swaddled in bureaucratic oversight.
At this point, it’s worth asking a question that may give some pause for thought:
Should we base our modern, highly variable software environments on management ideas anchored in the nineteenth century and designed for repetitive manufacturing?
Low Social Trust
Environments of low social trust depress morale, initiative, and the belief that “we’re all in this together.” The result is a self-reinforcing spiral of low trust and poor outcomes, generating even more distrust. [7] It typically descends into management “holding workers accountable” with punitive measures reinforcing the idea that no one can be trusted to do a good job without an overseer’s authority always at hand.
Low social trust is a direct function of the belief that workers lack initiative and aim to work as little as possible while still receiving a paycheck. Given this, it’s no surprise that the response is management layers whose primary function is to certify that the workers below them are busily engaged and productive. Whether that productivity produces useful software or not seems to be of less importance.
In short, low social trust results in a bureaucratic structure intended to certify work. These structures broadcast their lack of trust, further undermining trust and morale. The unfortunate result is that the system causes more of the problem it’s intended to address.
Theory X
In the 1950s, Douglas McGregor developed the concept of two types of work classifications, Theory X and Theory Y. [8] Simplifying, Theory X holds the profoundly negative view that most people dislike work, will seek to do as little as possible, and must be closely managed to assess their productivity. Also, workers have little initiative to solve problems independently and must be told what to do by their managers. A “carrots-and-sticks” approach keeps workers in line and certifies that work is satisfactorily done. Finally, managers see themselves as more capable and motivated than their workers and must closely oversee their work.
The parallels to Theory X are immediately apparent in Taylorism and its environment of low social trust. In such a system, the only way to achieve outcomes is to create a bureaucratic structure to oversee all aspects of the work.
While it’s doubtful that this philosophy was valid even during the repetitive manufacturing era of Taylorism, it isn’t helpful in today’s software engineering world populated by motivated and creative knowledge workers. It’s remarkable that we still see evidence of this philosophy in some workplaces today, but it explains the abundance of bureaucracy in them.
In Search of Predictability
Among other causes, bureaucracy results from believing that software engineering is predictable and plannable. This belief may be justified in traditional endeavors such as building a bridge or designing a kitchen. Still, it holds little value in the empirical, non-deterministic, and iterative model of building software [4]. If we incorrectly believe software development is predictable, we will inevitably create plans and bureaucratic management structures to control them.
These structures aim to erase variation by making the work output consistent. This approach is often misunderstood because software engineering lies not in the Complicated Domain of the Cynefin model but almost exclusively in the Complex domain. [9]
Briefly, the Cynefin model is a decision-making guide that delineates complexity domains and the unique decisions required for each one. In the Complicated domain, it’s possible with skill and care to plan and predict the outcome of work. This approach is nearly impossible in the Complex domain, requiring iterative rounds of probing, sensing, and responding as we steer toward a solution that can’t be foreseen at the outset.
We’ll never understand why our software efforts aren’t predictable until we grasp the Cynefin model and its implications. Failing that and being frustrated, we often double down on bureaucratic structures as the answer to controlling uncertainty. While believing we achieve economic gains by doing so, we ironically miss out on the gains from exploiting variability instead of trying to squelch it. [10]
An anecdote:
A colleague once said that during an Agile guide assignment, he asked a manager if the team could run experiments to discover better ways of working. Not grasping the contradictions in their forthcoming reply, the manager responded, “Only if I know what the outcome will be beforehand.”
It goes without saying that experiments are designed to test hypotheses and thereby reach conclusions. The outcomes can’t be known in advance. Anything that fits that definition is, by nature, a mandate, not an experiment, and results in spending money for zero informational value. Unfortunately, this occurs when bureaucracies throttle experimentation, discovery, and innovation in the pursuit of predictability.
The Problems Bureaucracy Causes
For some, bureaucracy might seem to address how to manage poorly regarded workers in a low-social-trust environment. But such a philosophy causes myriad other problems we don’t always see.
For those who believe in bureaucratic solutions, it pays to heed a warning from none other than the father of modern bureaucratic philosophy, Max Weber. He expressed his concern that leaving bureaucracy unchecked would strangle its host.
“…the entrepreneur [is] needed as a counterweight to the bureaucrats inside … organizations, or these will take over and stifle them.”
Max Weber
So far, we have only hinted at some of the problems caused by bureaucracy. Let’s now enumerate them in detail.
Where is the Customer?
In bureaucratic environments, we don’t believe workers can be trusted to be self-organizing or motivated. Consequently, we create frequent checkpoints to verify that work is being completed. Each checkpoint must then be guarded by a sentry whose job is to police the work being done. In deeply hierarchical organizations, there even will be sentries whose job is policing the sentries below them.
Notice what’s missing here. While an abundance of people check boxes confirming work is being done, no one validates whether we’re actually building useful software. The complicated org charts of bureaucratized Agile include many roles, but they omit a direct role for the most important person, the one vouching for the usefulness of our software:
The customer.
Even though Agile frameworks contain areas devoted to “customer centricity,” there is no single placeholder strictly for the customer. If we’re genuinely customer-centric, org charts would resemble a hub-and-spoke model, with the customer as the hub and spokes radiating outward toward the workers. In bureaucracies and the frameworks they employ, we see instead a large hierarchy of administrators reigning over workers and the customers challenging to find.
Layers of Indirection
There’s an old programming joke that goes like this:
“We can solve any problem by introducing an extra level of indirection.” [11]
While layers of indirection might be a meaningful way to structure our code, it’s a questionable way to structure our teams. If in doubt, look no further than the obvious rejoinder to the above quote: “…except for the problem of too many levels of indirection.” [11]
Layers of indirection between the programmers and end users cause several ills.
- The software user’s voice must be passed between one or more people before reaching those writing the code. Each step along the way dilutes and distorts the original message. Like the old game of telephone, the message may have changed so much that it’s lost value by the time it reaches its intended recipients.
- Each layer of indirection creates one or more queues where information must wait before its intended target receives it. Queues are costly, wasteful, and typically invisible and unmanaged. [10]
- When code, user stories, and other information don’t pass directly to their target, they inevitably become inventory that waits to be picked up and used. Inventory is a classic sign of waste in Lean circles. [12]
If more layers of indirection would solve software engineering challenges, then a sufficiently large number of layers would eliminate all of our problems. Such a conclusion is clearly absurd, yet its underlying assumption often goes unquestioned, leading us to build layers of bureaucracy that we believe will improve outcomes.
The more layers of bureaucratic indirection we have, the more we waste time and money and unnecessarily increase risk. Unfortunately, there is no line item on traditional budgets for this waste. As a result, it typically goes unnoticed and uncontrolled, with little understanding of the benefits we could achieve by reducing it.
Levying a Tax on Customer Interaction
Let’s consider bureaucratization from an economic perspective. Suppose we assume that there’s a transactional relationship between customers and the developers who create products that customers want. In a free exchange of goods — my money for your product — all of the money spent would flow directly to those making the product as a reward for their labor. The more I value the product, the more I’m willing to pay. The market between us is perfectly efficient.
However, we don’t typically construct our organizations in this fashion. Instead, the layers of indirection between the parties act as a tax levied on the exchange. The more layers we have, the higher the tax. At some point, the market becomes so inefficient that a substantial portion of the exchange is lost to tax, and customers overpay for the goods they receive. In effect, we insert intermediaries into the transaction whose presence adds cost.
In one sense, bureaucratic hierarchies resemble the centrally planned economies of the Soviet era. A handful of managers deciding how a large number of workers will do their job mimics the apparatchiks deciding how much green tomatoes should cost versus red tomatoes.
What if the layers of administrative bureaucracy were removed? Instead, the money spent on them could be redirected to those who create products, enabling us to build and sell even more products. Alternatively, we might be able to lower the products’ cost without affecting margins, making the products more competitive in the marketplace.
One problem with bureaucratization is that these transactional costs are seldom made visible. As a result, we rarely consider its economic impact.
Bureaucracy Is Self-Sustaining
If there’s a universal characteristic of bureaucracies, it’s that they’re self-sustaining once established. Max Weber himself saw their nature as inescapably perpetual:
“The individual bureaucrat is, above all, forge to the common interest of all the functionaries in the perpetuation of the apparatus and the persistence of its rationally organized domination.”
Max Weber
Once a bureaucracy is established, however small and seemingly inconsequential it may first appear, it soon becomes a living organism seeking to increase its reach and duration. Whatever positive values it might provide, these are sometimes subordinate to maintaining its existence.
We can try trimming the weight of a bureaucratized system by cutting its budget. However, attempting to do so will often result in the system responding with the Washington Monument syndrome, where the most valuable services are the first to be threatened with extinction. [13] In software environments, the response to bureaucracy reduction might be, “Well, I guess we’ll need to shut down the servers we depend on.”
In short, attempting to shrink a bureaucratized system can cause it to shift and react in ways that negate the attempt. It becomes a frustrating cycle of repeatedly perturbing a system and watching it revert each time to its former equilibrium.
Optimizing the Five Percent
It’s distressingly common for bureaucratized systems to focus on a minute subset of an organization’s problems. Generally, these efforts are directed at improving the workers’ performance via objectives, performance reviews, and rankings. Unfortunately, this focus is often misplaced because the workers form only a fraction of the overall problems. The celebrated management guru, W. Edwards Deming, made this abundantly clear with a statement known as the 95/5 rule.
“95% of variation in the performance of a system is caused by the system itself; only 5% is caused by the people.”
W. Edwards Deming
Yet we build entire bureaucratic edifices to manage the 5% (workers) and blithely ignore the 95% that causes most of our problems. Why do we do this? Possibly because we tend to manage what we see most easily, namely, the workers. Also, our blindness to the more significant system problem is quite likely because we are so immersed in the system that we can’t see outside it. It’s the metaphorical air we breathe without realizing it’s all around us.
Morale and Initiative
One of the more pernicious aspects of bureaucratized systems is their negative effect on morale and initiative. Workers, hamstrung by restrictive processes and approval procedures, are gradually worn down by bureaucratic red tape. Their morale and initiative suffer, imperiling the organization’s future. Many workers eventually succumb to learned indifference and simply strive to “keep their heads down and their mouths shut.” This outcome is hardly a recipe for long-term corporate success in the face of hungry competitors.
When left unchecked, bureaucracy eventually drives out innovation, independent thought, and the joy of solving problems. Everything is standardized to serve the rule-makers, whose power is too often absolute. It’s easy to convince ourselves that this approach makes work more predictable and straightforward to measure. Whether that’s true is doubtful, but one thing isn’t: Our work is ultimately less satisfying and less open to the innovation that drives market advances and customer delight. Consider this:
“There’s compelling evidence that bureaucracy creates a significant drag on productivity and organizational resilience and innovation.” [14]
The power to solve problems should belong to those closest to the work. In bureaucracies, power frequently lies with those who know less about the work and whose primary function is enforcing rules. The workers closest to the work see the urgent problems that need solving. Managers above them are too far removed to see the gritty details, often directing their charges to solve the less-important issues that the managers observe.
These less-important problems frequently revolve around evaluating individual performance and compliance with rules. It’s hard to see how this focus adds customer value. Indeed, it often subtracts from it. To wit:
“A myriad of studies have documented the time lost to low-value management processes, from budgeting to the performance review. On the basis of this evidence, it’s reasonable to assume that as much as 50% of all internal compliance activity is of questionable value.” [14]
The focus on performance and compliance means that we’re ultimately settling for mediocrity by removing the bothersome outliers, those who see the world in a different light but are the necessary ingredient of innovation. When we carefully fit everyone into boxes where they’re measured and evaluated, we banish the unreasonable people whose outsiders’ views bring invention, experimentation, and the leaps of progress that open new markets. The Irish playwright George Bernard Shaw, quite bothersome himself, aptly described such people (in gender-specific terms) in a delightful syllogism:
“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.”
George Bernard Shaw
One goal of bureaucracy is the understandable desire to prevent disastrous errors that might destroy an organization. However, in our zeal to avoid these errors, we extinguish the twinned flame of morale and initiative that drives a business’s success. We arrive at this extinguished state because we typically see only the obvious and immediate consequences of errors, not bureaucracy’s slow strangulation of success accumulating in small increments.
An anecdote:
Many years ago, I worked in a large company with a long history of Waterfall development. It was deeply invested in transitioning to a bureaucratized Agile framework when I arrived. One day, a brilliant colleague with a remarkable grasp of numerical algorithms mentioned that he was considering adding a complicated, algorithmic feature to our main product. This feature would distinguish it from our competitors and increase sales. I enthusiastically replied that it was an excellent idea. Having only recently arrived at the company, I was unaware of its bureaucratization and unprepared for the response to my colleague’s idea.
Given that we were managers, one of the tasks my colleague and I were required to perform was to generate regular status reports on how work was progressing in our departments. Although I thought they were wasteful, I meekly accepted that it was just part of the job that had to be done. My colleague, much to his credit and courage, thought that his time was better spent on the new feature and opted to forego his reports.
The bureaucracy reacted with alarm tinged with fury. My colleague received many tongue lashings from the managerial layer above him, but undaunted and more steadfast than me, he continued his work on the feature. Despite my concerns that he was jeopardizing his position, he dismissed the reprimands as nothing of consequence, just the “sound and fury” of a foolish system.
Shortly thereafter, my colleague completed his feature and released it to customers. They immediately showered praise on it, propelling our product forward in the market. The CEO was so pleased that he awarded my colleague a bonus and a special write-up in the company news about the importance of innovation.
I don’t know if the CEO ever realized the bureaucratic slings and arrows my colleague had suffered by simply trying to do something beneficial. I also don’t know what he would have done had he learned of them. Would he have waded into the bureaucratic thicket, brandishing a sword and cutting red tape? If he had, would the thicket have quickly grown back?
Our managerial overseers were doubtlessly well-intentioned, but I don’t imagine they realized the negative side-effects of their insistence on rigid adherence to established protocol. Had they known beforehand what value lay in the new feature, they certainly would have cleared a path for its creation. But therein lies the problem. We can’t know in advance what innovations will be useful. We only know after the fact. We must keep the path clear at all times and avoid smothering the innovative spirit with bureaucracy. Otherwise, we risk inadvertently destroying future benefits.
What to Do About This
So far, we’ve discussed why bureaucracies form and the problems they cause. The obvious next topic is what to do about them.
Reform is challenging because, as discussed above, bureaucratized systems resist change. Nonetheless, some things might help us achieve a more Agile approach. Let’s examine them now.
High Social Trust
Practicing high social trust in an organization is a virtuous circle. Environments with high trust produce a sense of the common good and sacrifice for the betterment of all. This trust creates cooperative work patterns and successful outcomes, rewarding everyone with even more trust in their coworkers.
High social trust environments allow us to flatten the managerial hierarchy. There are plenty of examples of enterprise-class businesses that have remarkably few managers. [14] Because workers are free to discover and solve problems, there is little need for multiple management layers.
Let’s restate the crucial Agile Principle:
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. [2]
In addition to trust, this Agile principle has one fundamental idea at its heart: Autonomy. The Principle suggests that we should free motivated workers from bureaucratic oversight and give them the autonomy to do their jobs as they see fit. It means stepping back and extending trust, knowing our workforce is talented and capable. In doing so, we’ll discover that freed workers are remarkable engines of progress that don’t require expensive policing and measuring layers above them.
Theory Y
Closely tied to social trust, Theory Y states that workers are intrinsically motivated to do their best work and derive great satisfaction in doing so. Additional rewards aren’t required to achieve better outcomes. [8]
With this theory, we choose to see workers as highly engaged and requiring little, if any, supervision. They can determine what needs to be done and then do it. There’s no need for bureaucratic layers above them to direct their activities and certify that the work is done.
Theory Y is a profoundly different outlook on workforces than in bureaucratized systems. Adopting this outlook signals our trust in motivated workers, freeing them to be their best and excel at their work.
On-Site Customer
Most software organizations use proxies that represent the customer’s interest. These proxies are charged with understanding the customer’s needs and translating those to the development team. While this is undoubtedly better than omitting the customer role, it is far less effective than having the customer itself on site.
Consider the recommendation of the Extreme Programming (XP) approach, which advocates for direct interaction with the customer via the rule, “The Customer is Always Available.” Specifically, XP states the following:
“All phases of an XP project require communication with the customer, preferably face to face, on site. It’s best to simply assign one or more customers to the development team.” [15]
When we bring the customer in-house to work directly with developers, we remove the layers of indirection between those who consume the product and those who produce it. Instead of layers of bureaucracy separating consumers from producers, we use direct, face-to-face communication that minimizes ambiguity and the delays caused by sending messages through proxies.
Innovate, Innovate, Innovate
In the early life of a business, we face one of two options:
- Innovate, move quickly, and minimize overhead to discover an exploitable market niche.
- Fail at the above and go out of business.
If a company succeeds at the first option, safeguarding the revenue-generating core of the business is all too easy. Then, the rise of bureaucracy begins. Given enough time, unchecked bureaucratization strangles innovation. The company eventually goes under when markets shift, and new competitors arrive.
This situation can be avoided by understanding that any niche will gradually disappear. Thus, we must keep innovating because long-term survival depends on continually finding new niches. In short, innovation led to the success that established the business. It’s innovation that will keep it alive and bureaucracy that gets in the way. Put simply, it can be described as:
Innovate, innovate, innovate. The money will follow.
Innovate not only on the products but on how we do our work. The latter is critical if we want to become a learning organization.
Become a Learning Organization
There are many ways to define what it means to be Agile. There are equally many offerings that promise Agile transformations by following a series of predetermined steps, a dubious promise founded on a dubious premise.
One of the more effective means of striving toward agility is simply becoming a learning organization. This tactic requires understanding that being an Agile organization means a commitment to learning, something few organizations genuinely recognize. While straightforward in theory, this is often exceedingly tricky in practice because it requires changing deeply-held beliefs about the nature of a workplace.
Learning organizations have cultures that focus on continuously acquiring and disseminating knowledge throughout. They prioritize personal and professional improvement, pursuing knowledge through experimentation, psychological safety, an openness to new and sometimes oppositional ideas, and leaving time for reflection. [16] They test the validity of their knowledge through productive debate and entertaining dissenting viewpoints. They seek to learn from everyone instead of knowledge being handed down from above on stone tablets.
Contrasted to bureaucratic, performance-based organizations prioritizing worker productivity, learning organizations make knowledge acquisition their north star. They understand that the path to long-term success is through learning, not rules and compliance.
Beyond Projects
Traditional software projects require estimates, plans, and budgets. They also have start dates, milestones, end dates, and expensive managers overseeing them. At the end of the project, we consider the work done, and the project is deemed officially over.
This approach is grounded in how work was done in the Waterfall era, casting doubt on its underlying philosophy. It’s natural to wonder, what if we could do away with most of that and instead focus on self-organizing around the most important things?
Enter the idea of Beyond Projects.
Beyond Projects addresses the concerns listed above. Most notably, it handles the mismatch between the theory of software projects being “done” and the reality that they’re never done because they’re constantly modified, repaired, and improved. They’re only “done” when they’re no longer in use. With a project-based mindset, we too easily lose focus on what should be our goal: Delivering useful software to paying customers. What we typically focus on instead is adherence to project plans and budgets. [17]
A significant contradiction with project-based development is the rise of continuous delivery. If we can respond quickly to customer feedback by delivering software every few minutes, what’s the point of long-range project plans, requirements, and budgets? Continuous delivery drives the marginal cost of deployments toward zero, enabling tiny batch sizes and reducing queues and inventory. Projects traditionally assume the opposite, that software will be planned in advance and released in large batches. While projects can be restructured to incorporate continuous delivery, it’s difficult to justify their associated bureaucratic costs when we can deploy our software continuously.
Using the Beyond Projects approach, we no longer accrue a large batch of work items, push them onto workers, and deliver them in one enormous deployment. We instead allow work items to arrive at any time, typically in response to customer feedback, and then pull the most valuable ones from a limited list of these items. We size the items, slicing them as required so they’re small enough to be continuously delivered in small batches. Even better, we can let the workers discover the most critical work that will delight the customer and give them the freedom to pursue it.
Without formal projects, we have little need for the bureaucratized processes of traditional project management. Instead, we’re “responding to change over following a plan.” [1]
Open Allocation
Here’s a heretical thought:
Why do we need people to keep workers motivated and productive?
What if we followed the Agile Principle of hiring motivated people, giving them the tools they need, and trusting them to get the job done? [2] We might find that the managerial impediments intended to detect unmotivated workers are unnecessary. We could save money and obtain better outcomes instead of wasting money on questionable oversight mechanisms.
Welcome to the concept of Open Allocation.
In Open Allocation, the workers propose ideas, and it’s their job to recruit other workers to join them. Promising ideas naturally attract workers and gather momentum. Weak ideas soon wither. [18] This approach is based upon the Open Space concepts of a marketplace of ideas, the freedom to choose those most interesting to us, and the ability to change our minds at any moment and go somewhere else where we can add value. [19] In short, we use market signals to determine which ideas are most viable and which are best extinguished.
Traditionally, managers are viewed as those with the insight to determine how to allocate people, essentially saying, “We know what’s best for you and the company, so you’ll work where we decide.” Instead of management deciding who will work where, Open Allocation lets workers choose from a constantly changing menu of offerings. In effect, we tell them, “You’re good at what you do. That’s why we hired you. We trust you to choose where you can add the most value.”
This model is used in other contexts as well. It’s the underlying mechanism for Open Source software. Products are initiated, and developers can choose which ones to work on. The products that prove popular attract more workers. Those that don’t will stagnate. It’s also a model that mimics the venture capital approach, where ideas compete for users and, if successful, they’re awarded rounds of capital. Most ideas won’t survive, but those that do provide ample reward.
Open Allocation has the benefit of reducing the incentives that lead to bureaucratization. There’s little need for expensive, hierarchical layers when we’re free to propose and choose the work that interests us. We’re also less likely to incur the costs of wasteful undertakings kept alive by managerial politics. Such undertakings quickly dissolve because they don’t attract workers. This model requires no one to keep workers motivated and productive. The workers do that themselves, meaning less bureaucracy and more independence.
Closing Thoughts
Our industry frequently sees companies attempting an “Agile transformation.” Often, this includes hiring expensive consultants to guide the transformation, and in the end, we are pronounced “an Agile organization,” albeit one that’s heavily bureaucratized. However, to be Agile requires us to transform our mindset. Rather than believing that enlightened managers must oversee workers, we must change our beliefs to see our workforce as motivated, self-organizing, and requiring little oversight. Most so-called transformations yield disappointing results because we too often fail to adjust our mindsets.
Being unable or unwilling to transform our mindsets is why so many Agile transformations fail to make organizations truly Agile. Instead, they simply exchange one form of bureaucracy for another.
Instead of an Agile transformation, the success of which is typically hard to define, we should focus on making work easier. When we do this, we will likely find that bureaucratic impediments make our jobs unnecessarily hard. We might also discover we are more profitable and can better innovate to solve customer problems when those impediments are swept aside.
Better yet, we should avoid seeing an Agile transformation as an end goal to achieve so that we are “now Agile” and can rest on our accomplishments. Transformations imply a static finish line. Instead, like the proverbial horse with a carrot forever dangling before it, we should consider it a journey. We should accept that Agility is a never-ending voyage asking us perpetually to rethink our approach.
“To practice a discipline is to be a lifelong learner. You never ‘arrive.’ The more you learn, the more acutely aware you become of your ignorance.”
Peter Senge
Such a mindset brings us closer to the original spirit of the Agile Manifesto and Principles, delivering us from bureaucratization. In doing so, we might reach that most difficult of states: Being truly Agile.
References
[1] “Manifesto for Agile Software Development”
https://agilemanifesto.org/
[2] “Principles Behind the Agile Manifesto”
https://agilemanifesto.org/principles.html
[3] Rigby, Darrell, 2018, “Bureaucracy Can Drain Your Company’s Energy. Agile Can Restore It.” Harvard Business Review.
https://hbr.org/2018/05/bureaucracy-can-drain-your-companys-energy-agile-can-restore-it
[4] Meadows, J., 2020, “Accepting Uncertainty: The Problem of Predictions in Software Engineering”
https://jmlascala71.medium.com/accepting-uncertainty-the-problem-of-predictions-in-software-engineering-26dbcd120b90
[5] Swedburg, Richard, Ageval, Olla, 2005, “The Max Weber Dictionary: Key Words and Central Concepts.” Stanford University Press, pp. 18–21.
[6] Taylor, Frederick Winslow, 1911, “The Principles of Scientific Management”
https://archive.org/details/principlesofscie00taylrich/page/n5/mode/2up
[7] Zak, Paul J., 2017, “The Neuroscience of Trust”, Harvard Business Review.
https://hbr.org/2017/01/the-neuroscience-of-trust
[8] Wikipedia, “Theory X and Theory Y”
https://en.wikipedia.org/wiki/Theory_X_and_Theory_Y
[9] Snowden, David, and Boone, Mary, 2007, “A Leader’s Framework for Decision Making”, Harvard Business Review
https://hbr.org/2007/11/a-leaders-framework-for-decision-making
[10] Reinertsen, Donald G., “The Principles of Product Development Flow: Second Generation” Lean Product Development. Celeritas Publishing, 2009.
[11] Wikipedia, “Fundamental theorem of software engineering”
https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering
[12] Lean Manufacturing Tools, Waste of Inventory; causes, symptoms, examples, solutions
https://leanmanufacturingtools.org/106/waste-of-inventory-causes-symptoms-examples-solutions/
[13] Wikipedia, “Washington Monument syndrome”
https://en.wikipedia.org/wiki/Washington_Monument_syndrome
[14] Hammel, Gary, and Zanini, Michele, 2016, “Excess Management s Costing the U.S. $3 Trillion Per Year.” Harvard Business Review.
https://hbr.org/2016/09/excess-management-is-costing-the-us-3-trillion-per-year
[15] “Extreme Programming: A Gentle Introduction.”
http://www.extremeprogramming.org/
[16] Garvin, David, Edmondson, Amy, and Gino, Francesca, 2008, “Is Yours a Learning Organization?”, Harvard Business Review
https://hbr.org/2008/03/is-yours-a-learning-organization
[17] Kelly, Allan, 2014, “The Problem with Projects and some ideas on what to do about it,” InfoQ
https://www.infoq.com/articles/kelly-beyond-projects/
[18] Wikipedia, “Open Allocation”
https://en.wikipedia.org/wiki/Open_allocation
[19] Owen, Harrison, “A Brief User’s Guide to Open Space Technology,” Open Space World.
https://openspaceworld.org/wp2/hho/papers/brief-users-guide-open-space-technology/
