Bridging the Gap Between Business and Software Development

14th February 2015 Pete George no responses Agile

Last week, I delivered a Scrum training course to two quite different clients. One was a government department and the other a large international private sector company, but the anticipated challenge both groups voiced was to how to successfully communicate a shared understanding of business value between the business and development areas. This is a common issue and I thought I’d jot down a few notes on some of the things discussed, which might help folks struggling in the same way.

Firstly, the source of the problem is that both organisations wanted to avoid the typical command-and-control/big-plan-up-front approach of ‘traditional’ software delivery. That is, in traditional (‘Waterfall’) projects, the problem the business wants solving is first solved and then the solution is handed over to the development folks to deliver. There is no real concern about how to bridge the gap between business and development, because the gap is filled with a large set of requirements that explain to the developers exactly what to build (in theory anyway).

These organisations were looking to move to a Scrum/Agile software development approach, recognising that getting the business (i.e. the people who want the problem solving) together with development (i.e. the people who know how to solve the problem) is actually a very beneficial idea. That’s where it gets a bit more tricky. How do you do that well?

The standard Scrum approach is to appoint a Product Owner to work alongside the developers as a member of the Scrum team. The PO is the single representative of all the stakeholders, someone who understands the business needs and has the authority to make decisions on business priorities. As far as the Scrum development team are concerned, they don’t need to know anything else about higher level business needs, because the PO provides their oracle of what is valuable in the form of a prioritised Backlog. The Backlog is essentially the business ‘problem’ broken down into a list of sub-problems to be solved by the developers. Which is fine if you have folks like that, but, as is often the case, individuals like that can be scarce.

How do you get good Product Owners? The first question I think you need to consider is whether the PO needs to be a domain specialist. For example, in the case of the public sector group, internal customers of the products performed the role. This will depend on the complexity of the product being delivered, management structures and the relationship to other stakeholders. For example, if there are few other stakeholders available to the Scrum team to discuss requirements with on a regular basis, then it may be particularly beneficial for the PO to have a strong domain knowledge.

Alternatively, the PO could be someone skilled/experienced in gathering and communicating business needs, e.g. a business analyst. In this case, the PO would be expected to gather the required information on behalf of the team from all the stakeholders and ultimately use their feedback to produce the prioritised Backlog for the team. As an alternative to a BA, a developer could fulfill this role (as is often the case), but the danger is that the steer for the team is on ‘building the product right’ over ‘building the right product’.

Whichever approach to PO is chosen, the issue of authority is also a challenge. The PO needs to be able to make decisions quickly in order to provide the team with the necessary feedback they need to move forward. Unfortunately, the situation for many organisations is that the decision makers are often too busy to work closely with Scrum teams. You either get POs who are not really a committed member of the Scrum team, or you get POs who are available but not able to make decisions without referring to their seniors. Either way, the delivery of the product is disrupted. One compromise is to share the PO role between a senior manager and more hands-on user, but that is not ideal.

What is certainly needed, is for the PO to be empowered to be able to make the necessary decisions. This could be a bit of a culture shock for many organisations and particularly for the PO, so good education in Scrum is essential. A lot of organisations see the need to invest in training Scrum Masters and even training development teams, but, in my experience, training for the PO is often neglected. I suspect that is out of ignorance generally at a higher management level of the importance of this role and perhaps, in addition to PO training, there is a need for better understanding of Scrum/Agile by business managers, sponsors et al. (Ken Schwaber and Jeff Sutherland, Scrum co-creators, put out a good book to address this kind of audience, in 2012: Software in 30 Days: How Agile Managers Beat the Odds, Delight Their Customers, And Leave Competitors In The Dust).

Assuming the Product Owner is empowered, educated in Scrum, and has the necessary skills and experience for the task, the Scrum team should be able to deliver the ‘sub-problems’ on the Backlog effectively and the business will get exactly what they want.

However, here is one last thing to ponder. Scrum has been used successfully used for over twenty years and all that I have said above is perfectly valid in getting the most value out of Scrum framework, but, although the aim is for the team to understand the business problem because it is communicated to them via PO/Backlog, in reality the problem has already been largely solved by the time it hits the Backlog. There are various higher layers of the problem above the level of the Backlog items. It may be that a Scrum team with a good PO can understand these higher business requirements, but, in more recent years, there has been a growing realisation that this may not be enough and that greater collaboration at these higher levels is needed. Various techniques and process patterns have developed in this area, generally described under the umbrella title of Behavior-Driven Development, which are well worth further investigation.