adopt

Just recently I reviewed the 9^th Agile survey (read about it here) and already the 10^th version is out (you can access the survey here), so I thought I provide an update. Given the results are not that different from the previous year, I will write this post a bit different. I will review some of the key results and then will provide in a second section some critical thoughts on the survey.

Part 1 – Review of the Results

Our delivery world is complex (multi-modal, distributed and outsourced)

Looking at the results about company experience, 53% say that less than half of the teams in the organisation uses Agile, only 9% say that everyone is doing Agile. So clearly we live in multi-modal world. 82% are working with distributed teams, which is larger than I expected but reflects the largely distributed nature of IT projects these days. And nearly 70% use outsourcing for their development projects. This complex environment is normal. I find it interesting how many clients tell me there are “special” because of these characteristics, they clearly are not.

What are typical benefits of Agile?

The report highlights the following three to be the top answers (which have been the same last year):
– Ability to manage changing priorities
– Team productivity
– Project visibility

This sits well with me as the usual suspects that we have to debunk are not in this list, e.g. faster time to market (only on rank 6) and cheaper delivery (not in the list at all).

How do you measure the success of Agile?

The answers to this are also stable:
– On-time delivery
– Product Quality
– Customer Satisfaction
Good answers, I think this reflects well what success looks like. I guess these are somehow measurable and provide some success measure for Agile.

How do you scale Agile?

We see a significant jump for SAFe here from 19% to 27% and I am not surprised. It certainly has a lot of momentum in the industry and provides practical guidance that I leverage in my daily work. Other frameworks are really not present (DAD, LESS,…) and have less than 10%.

Of the top 5 tips for Scaling Agile at least two are in my top lessons learned too: Consistent process & practices and Implementation of a common tool across teams. I agree with the other 3 tips as well: Executive Sponsorship, getting help from someone with experience like an Agile consultant and creating an internal support team.

What tools do you use?

Really not much movement here. The previous favourites remain on top: Jira, VersionOne, and TFS.

What makes Agile fail?

Culture has been called out as main reason this year. And it is hard to argue with this. That however is difficult to address directly. The other information is easier to address: lack of experience is the second main reason Agile fails, this means that we should make sure experienced Scrum Masters and coaches support new projects. Lack of management support and lack of support for the cultural shift are next, something we need to work on as an industry. And then a very much addressable one still gets 38% of the votes: Inconsistent agile practices and processes. This we need to fix, I don’t understand how so many people still allow Agile projects to fail because they think Agile is a freeform exercise. At the end of the day there is a project/outcome to be delivered and too many coaches still shy away from that accountability. If you work with Agile coaches make sure they have skin in the game so that they balance learning with delivery responsibility.

Overall I like the results in the report and it is consistent with the previous year which is good to see.

Part 2 – A more critical look at the survey

I like the results of the survey and got some interesting insights, but how valid is the survey? Do I understand the mechanics behind it well enough to use the data?

Let’s look at a few pieces of information that should make us think about the data before blindly quoting it:

• First of all – who is the sample audience? Clearly the people responding to this have somehow heard about Agile as it is unlikely that others respond to an Agile survey. The people who respond are self-nominating, so they might be more on the extreme ends and feel they have something to contribute. It is not a random sample of people working in IT. This then influence the data to some degree I would think.
• 1% said that Agile adoption has failed in their organisation. That number seems very low and might be because those people where it has failed are not responding, or it could be because the definition of failing is actually quite difficult to understand. Does failing mean you are completely going back to Waterfall or because you had mostly bad results from Agile. Not sure what my definition would be, so perhaps the lack of definition causes people to not use this value in the survey.
• Agile techniques used – Now here I really start to struggle with the values and figuring out how this information should help me. Do only people respond to this who use Agile or everyone (even those using Waterfall)? Daily Stand-ups and prioritised backlog seem pretty fundamental to me and other than in rare cases I would expect Agile projects to use these. The numbers seem low on that basis. But wait it gets much worse, only 54% do iteration reviews/showcases, only 45% have devs and testers in the same team. I would argue that if you don’t have those two you are not really Agile – I defined Agile in another blog post as “ONE team delivery an INCREMENT of a product (at least product tested) within ONE iteration”. The problem here is that one cannot relate the info back to the methodology used and hence it becomes difficult to derive insight from this data.
• Success measures – Velocity is the largest one. I was hoping we got over the velocity race, but clearly it’s still on. The success measures are very contextual and I would caution anyone to use this without thinking. Take for example “Planned vs. Actual stories” – it could be very positive to deliver more stories than planned or even less stories (as long as the most value is delivered). Context is unfortunately king for metrics, so don’t use the information from this survey blindly.