A couple of years ago I was working with a Web development company who were looking at adopting Scrum. As I often do, I started be comparing Scrum with a traditional ‘Waterfall’ model to highlight the former’s features. However, I quickly discovered that the delivery manager had never heard of waterfall. She, and the company, were quite young and new to the world of software development. Their problem was not that they were stuck with heavy weight traditional delivery processes and needed a more agile framework to help them deliver (which is more often the case for most companies I deal with). Instead, this company had grown rapidly in a few years and were looking for a product management framework to help them control their delivery more effectively. Scrum proved to be very effective for them.
It struck me that often the perception of ‘Agile’ is that it is about being less controlled. However, this is only because of that heavyweight process background many have come from (the Agile Manifesto was of course a reaction to such an environment). In reality, Agile approaches, like Scrum, are about being very controlled, and rightly so. There is a lot at stake in delivering complex software in complex environments, and the greater the complexity the more control is needed. I enjoy rock climbing in my spare time. Climbing, like software development, starts with a challenge, which you attempt to solve, by applying whatever skill/physical dexterity you have. Like software development, there are also a number of ways of tackling that challenge, ranging from a high risk to a highly controlled approach. Often folks coming from a traditional context see Agile approaches as being in the ‘less control’, more risk camp, but it is important to understand that Agile is actually all about reducing the risk of failure through highly controlled software delivery.
For climbing, the simplest approach to the challenge is to just start at the bottom of a mountain and work to the top, without any kit or planning. This can be a highly risky approach. The risk is maintained throughout the climb and the impact, e.g. death from falling, increases the higher you go. This kind of ‘free climbing’ is very Agile in the sense that you can vary your route as much as needed to get where you want to go. The ability to change direction, forge new paths and having the flexibility to solve the challenge in different ways, is very important in the delivery of software development. It is the central agile concept. However, an uncontrolled, see-what-we-get-when-we-finish, ‘free climbing’ approach to software development would not only be highly risky, but also deeply unpalatable to the business.
A ‘free climbing’, uncontrolled approach to software development may have worked (to some degree) in the early days of the industry in the 1960s, but it was quickly seen as inadequate. Winston Royce’s ‘Waterfall’ approach, from his 1970 IEEE presentation, was one reaction to his lack of control and has had a lasting impact on the industry ever since. The Waterfall approach is an attempt to control software delivery by predicting up front what needs to be done and then rigidly sticking to that plan until completion.
Using the climb analogy again, Waterfall is akin to the ‘Top-roping’ method. This is something I did a lot when my kids were small. I would investigate a suitable crag and spend about an hour setting up a belay point at the top, for a safety rope to be attached to. We would then start the climb from the bottom. They would then climb tied on to one end of the safety rope, while I held the other end. The idea being to reduce the risk of injury. However, although it was true if they slipped they didn’t hurt themselves (because the rope stopped their fall), there was a high risk of failure to solve the challenge, i.e. that they didn’t get to the top, because the climb was just too hard. As they were forced to go one way by the top-rope setup, if there was a problem, there was little option but to abandon the climb and repeat the process elsewhere, i.e. take the ropes down and set up a new belay. Such is the way with a predictive-sequential software delivery model. In an attempt to control risk, a heavyweight process is applied that demands the absence of change and runs a high risk nugatory upfront effort in planning how to solve the challenge, only to find out that the challenge is unsolvable when you finally start implementing the solution.
Agile then, is to software delivery, what ‘trad climbing’ is to rock climbing. A way of minimising risk by applying a great deal of control, in a flexible way. In trad climbing, you do some upfront preparation, but don’t waste time setting up a belay dictating the end point. Two climbers start at the bottom of the crag. The lead climber climbs without a rope above them, but at various intervals clips the rope onto various protection points in the rock. If the leader falls, there is some risk of injury, but they only fall as far as the last protection point (not all the way to the bottom). The two climbers work together to climb in a low risk, highly controlled way to the top. Although the intention is to solve the climbing route they set out to do, there is flexibility to take different routes. For example, moving to left or right during the climb to avoid a section that is too difficult.
To me this is a perfect picture of Agile software development: collaborative working, reducing risk through incremental progress, an emphasis on learning by doing rather than failure through incorrect predictions upfront, using tools to support the delivery not constrain innovation, a focus on solving the challenge not following the process for the sake of it, and, above all, creating maximum flexibility through applying a high-level of team-centred process control.