Category Archives: Agile

Thoughts on the 10th State of Agile Survey

adopt

10th Agile reportJust recently I reviewed the 9th Agile survey (read about it here) and already the 10th version is out (you can access the survey here), so I thought I provide an update. Given the results are not that different from the previous year, I will write this post a bit different. I will review some of the key results and then will provide in a second section some critical thoughts on the survey.

Part 1 – Review of the Results

Our delivery world is complex (multi-modal, distributed and outsourced)

Looking at the results about company experience, 53% say that less than half of the teams in the organisation uses Agile, only 9% say that everyone is doing Agile. So clearly we live in multi-modal world. 82% are working with distributed teams, which is larger than I expected but reflects the largely distributed nature of IT projects these days. And nearly 70% use outsourcing for their development projects. This complex environment is normal. I find it interesting how many clients tell me there are “special” because of these characteristics, they clearly are not.

What are typical benefits of Agile?

The report highlights the following three to be the top answers (which have been the same last year):
– Ability to manage changing priorities
– Team productivity
– Project visibility

This sits well with me as the usual suspects that we have to debunk are not in this list, e.g. faster time to market (only on rank 6) and cheaper delivery (not in the list at all).

How do you measure the success of Agile?

The answers to this are also stable:
– On-time delivery
– Product Quality
– Customer Satisfaction
Good answers, I think this reflects well what success looks like. I guess these are somehow measurable and provide some success measure for Agile.

How do you scale Agile?

We see a significant jump for SAFe here from 19% to 27% and I am not surprised. It certainly has a lot of momentum in the industry and provides practical guidance that I leverage in my daily work. Other frameworks are really not present (DAD, LESS,…) and have less than 10%.

Of the top 5 tips for Scaling Agile at least two are in my top lessons learned too: Consistent process & practices and Implementation of a common tool across teams. I agree with the other 3 tips as well: Executive Sponsorship, getting help from someone with experience like an Agile consultant and creating an internal support team.

What tools do you use?

Really not much movement here. The previous favourites remain on top: Jira, VersionOne, and TFS.

What makes Agile fail?

Culture has been called out as main reason this year. And it is hard to argue with this. That however is difficult to address directly. The other information is easier to address: lack of experience is the second main reason Agile fails, this means that we should make sure experienced Scrum Masters and coaches support new projects. Lack of management support and lack of support for the cultural shift are next, something we need to work on as an industry. And then a very much addressable one still gets 38% of the votes: Inconsistent agile practices and processes. This we need to fix, I don’t understand how so many people still allow Agile projects to fail because they think Agile is a freeform exercise. At the end of the day there is a project/outcome to be delivered and too many coaches still shy away from that accountability. If you work with Agile coaches make sure they have skin in the game so that they balance learning with delivery responsibility.

Overall I like the results in the report and it is consistent with the previous year which is good to see.

Part 2 – A more critical look at the survey

I like the results of the survey and got some interesting insights, but how valid is the survey? Do I understand the mechanics behind it well enough to use the data?

Let’s look at a few pieces of information that should make us think about the data before blindly quoting it:

  • First of all – who is the sample audience? Clearly the people responding to this have somehow heard about Agile as it is unlikely that others respond to an Agile survey. The people who respond are self-nominating, so they might be more on the extreme ends and feel they have something to contribute. It is not a random sample of people working in IT. This then influence the data to some degree I would think.
  • 1% said that Agile adoption has failed in their organisation. That number seems very low and might be because those people where it has failed are not responding, or it could be because the definition of failing is actually quite difficult to understand. Does failing mean you are completely going back to Waterfall or because you had mostly bad results from Agile. Not sure what my definition would be, so perhaps the lack of definition causes people to not use this value in the survey.
  • Agile techniques used – Now here I really start to struggle with the values and figuring out how this information should help me. Do only people respond to this who use Agile or everyone (even those using Waterfall)? Daily Stand-ups and prioritised backlog seem pretty fundamental to me and other than in rare cases I would expect Agile projects to use these. The numbers seem low on that basis. But wait it gets much worse, only 54% do iteration reviews/showcases, only 45% have devs and testers in the same team. I would argue that if you don’t have those two you are not really Agile – I defined Agile in another blog post as “ONE team delivery an INCREMENT of a product (at least product tested) within ONE iteration”. The problem here is that one cannot relate the info back to the methodology used and hence it becomes difficult to derive insight from this data.
  • Success measures – Velocity is the largest one. I was hoping we got over the velocity race, but clearly it’s still on. The success measures are very contextual and I would caution anyone to use this without thinking. Take for example “Planned vs. Actual stories” – it could be very positive to deliver more stories than planned or even less stories (as long as the most value is delivered). Context is unfortunately king for metrics, so don’t use the information from this survey blindly.

Thoughts on the 9th State of Agile Survey

State of Agile 9The 9th Annual State of Agile report had some confirming and some surprising results. If you have not read it, it is worthwhile having a look here. And yes I know that number 10 is soon coming out, but hey there is still value in the 9th one to look at. Besides the usual statistics around how much Agile people are doing ( key number for me was only 5% working for fully traditional organisations), it provided some interesting answers to some of the more common questions that I hear and the answers are often not surprising but there was the odd surprise. Let’s jump in:

What are typical benefits of Agile?

The report highlights the following three to be the top answers (which have been stable over the last few years):
– Ability to manage changing priorities
– Team productivity
– Project visibility

This sits well with me as the usual suspects that we have to debunk are not in this list, e.g. faster time to market (only on rank 7) and cheaper delivery (not in the list at all).

How do you measure the success of Agile?

Uhhh, tricky one this one. I have heard the question many times and honestly have struggled to give an answer that satisfies senior stakeholders. So what did the report say:
– On-time delivery
– Product Quality
– Customer Satisfaction

Good answers, I think this reflects well what success looks like. Interesting that some of the items mentioned under top benefits are showing up much lower here: Managing priorities obviously speaks to the product quality and customer satisfaction. But team productivity (29% measure it) and visibility (30% measure it) are much lower on the list. An open question for me is how people would measure productivity in the first place (see my other blog on this).

How do you scale Agile?

As much as I come across SAFe it is only used by 19% of the respondents with all the other ones even lower (DAD, LESS). The largest proportion just uses Scrum of Scrums or some custom created method.

Of the top 5 tips for Scaling Agile at least two are in my top lessons learned too: Consistent process & practices and Implementation of a common tool across teams. I agree with the other 3 tips as well: Executive Sponsorship, having a coach and creating an internal support team.

What tools and practices do you use?

Wow – I was surprised, but perhaps I should not have been that Excel and Project are the most used tools…seriously, are we not better than that??? Oh well on the real Agile tools, Jira and VersionOne takes the cake, with TFS close behind. IBM is much much lower. This represents my position as well, JIRA is certainly the one most used and few people complain about it, especially when integrated with Confluence.

There is also information on the practices uses and I was shocked to see that only 69% use retrospectives and 48% have a dedicated product owner. Overall the adoption rates of the practices feels very low, perhaps there is some fundamental flaw in the data if it considers people who run Waterfall projects but use a select few Agile practices…hmmm….

What makes Agile fail?

Good information here as well, lack of experience is the main reason Agile fails, this means that we should make sure experienced Scrum Masters and coaches support new projects. Lack of management support and a non-aligned company culture are the other two main reasons Agile fails. Those are a bit more difficult to tackle but are important to be aware of as you set off on an Agile project.

Overall I like the results in the report and it certainly helps to see some market data validating points that I keep making with my clients.

Have we Agilists misused the military as example of Command & Control

Washington_Crossing_the_Delaware_by_Emanuel_Leutze,_MMA-NYC,_1851 (1024x656)If you are like me you believed since early days in your life, that the military is the example for command and control. I personally have never experienced the military by myself, but frequently I heard phrases like “We are not as strict with our command and control as the military”. Only recently after hearing from Don Reinertsen and Mark Horstman about their experiences in the military did I come to question my understanding.

“No plan survives the contact with the enemy” – from Helmuth von Moltke the Elder

So in my head I had this organisation where everything is well planned and if anything I would have associated it with the “waterfall” mentality more than an “agile” mentality. But let’s look a bit closer. In any real combat situation the enemy will behave differently to what you expect and it is very unreasonable to account for all possible details in the field. The above quote from von Moltke the Elder demonstrates that. So clearly the planning in detail and then just executing the plan approach will not work in such cases. So how does the military then operate?

The military makes sure that the soldiers understand what the goal of the mission is. Planning is being done on a high level (which mountain to take or what strategic points to take) which then breaks down into more detailed plans and not just for one scenarios. When practicing some variables get changed so that the soldiers learn to improvise and replan as more information about the situation becomes available. Does this sound familiar? It sounds exactly like the behaviour of an Agile team (with the difference that Agile teams don’t get the chance to practice their projects many times before doing it for real).

What this allows is a high-speed of decision while still adhering to a high level plan. It is possible because the organisation is aligned on the goals and high level plan. Imagine soldiers always had to wait in the field until the “project plan” is updated before they can proceed with changes to the plan. That would take way too long, so they are empowered to change as required within certain well-known parameters. By pushing decisions down to the lowest level the speed of decisions improves. And with clear parameters of what they can decide and what not, the risk of these decision is adjustable. When the lower level decisions aggregate to changes to the overall plan, then there are people at this level who can make those decisions as well (the product owners and release train engineers I guess).

I certainly think differently about the military now after hearing stories and examples that show how inherently agile they have to be. It makes for a good organisational example of combining high level plans and goals with agility and how to achieve positive results.

Here is a slide from Don’s talk with a few additional points:

military

I am no expert in the military so I am looking forward to your thoughts and I will surely learn from the discussion.

A personal DevOps Journey or A Never-Ending Journey to Mastery

I spent the last few days at a technical workshop where I spoke about Agile and DevOps and while preparing my talks I did a bit of reflection. What I realised is that my story towards my current level of understanding might be a good illustration of the challenges and solutions we have come up with so far. Of course everyone’s story differs, but this is mine and sharing it with the community might help some people who are on the journey with me.

As a picture speaks more than a thousand words, here is the visual I will use to describe my journey.
(Note: The Y-axis shows success or as I like to call it the “chance of getting home on time”, the X-axis is the timeline of my career)

Personal Journey

The Waterfall Phase – a.k.a. the Recipe Book

When I joined the workforce from university and after doing some research into compilers, self-driving cars and other fascinating topics that I was allowed to explore in the IBM research labs, I was immediately thrown into project work. And of course as was custom I went to corporate training and learned about our waterfall method and the associated process and templates. I was amazed, project work seemed so simple. I got the methodology, processes and templates and all I have to do was following them. I set out to master this methodology and initial success followed the better I got at it. I had discovered the “recipe book” for success that described exactly how everyone should behave. Clearly I was off to a successful career.

The Agile Phase – a.k.a. A Better Recipe Book

All was well, until I took on a project for which someone else created a project plan that saw the project completed in 12 weeks’ time. I inherited the project plan and Gantt chart and was off to deliver this project. Very quickly it turned out that the requirements were very unclear and that even the customer didn’t know everything that we needed to know to build a successful solution. The initial 4 weeks went by and true to form I communicated 33% completion according to the timeline even though we clearly didn’t make as much progress as we should. Walking out of the status meeting I realised that this could not end well. I setup a more informal catch-up with my stakeholders and told them about the challenge. They agreed and understood the challenge ahead and asked me what to do. Coincidence came to my rescue. On my team we had staffed a contractor who had worked with Agile before and after a series of coffees (and beers for that matter) he had me convinced to try this new methodology. As a German I lived very much up to the stereotype as I found it very hard to let go of my beloved Gantt charts and project plans and the detailed status of percentage complete that I had received from my team every week. Very quickly we got into a rhythm with our stakeholders and delivered increments of the solution every two week. I slowly let go of some of the learned behaviour as waterfall project manager and slowly became a scrum master. The results were incredible, the team culture changed, the client was happier and even though we delivered the complete solution nowhere close to the 12 weeks (in fact it was closer to 12 months), I was convinced that I found a much better “recipe book” than I had before. Clearly if everyone followed this recipe book, project delivery would be much more successful.

The DevOps Phase – a.k.a. the Rediscovery of Tools

And then a bit later another engagement came my way. The client wanted to get faster to market and we had all kind of quality and expectation setting issues. So clearly the Agile “recipe book” would help again. And yes, our first projects were a resounding success and we quickly grew our Agile capability and more and more teams and projects adopted Agile. It however quickly became clear that we could not reduce the time to market as much as we liked and often the Agile “recipe book” created a kind of cargo cult – people stood up in the morning and used post-its and consider themselves successful Agile practitioners. Focusing on the time to market challenge I put a team in place to create the right Agile tooling to support the Agile process through an Agile Lifecycle Management system and introduced DevOps practices (well back then we didn’t call it DevOps yet). The intention was clear, as an engineer I thought we could solve the problem with tools and force people to follow our “recipe book”. Early results were great, we saved a lot of manual effort, tool adoption was going up, and we could derive status from our ALM. In short, my world was fine. I went off to do something different. Then a while later I came back to this project and to my surprise the solution that I put in place earlier had deteriorated. Many of the great things I put in place had disappeared or had changed. I wanted to understand what happened and spent some time investigating. It turned out that the people involved made small decisions along the way that slowly slowly lost sight of the intention of the tooling solution and the methodology we used. No big changes, just a death by a thousand cuts. So how am I going to fix this one…

The Lean Phase – a.k.a. Finally I Understand (or Think I do for Now)

Something that I should have known all along became clearer and clearer to me: Methodology and tools will not change your organisation. They can support it but culture is the important ingredient that was missing. As Drucker says: “Culture eats strategy for breakfast”. It is so very true. But how do you change culture… I am certainly still on this journey and cultural change management is clearly the next frontier for myself. I have quickly learned that I need to teach people the principles behind Agile and DevOps, which includes a elements of Lean, Systems Thinking, Constraint Theory, Product Development Flow and Lean Startup thinking. But how do I really change the culture of an organisation, how do I avoid the old saying that “to change people, you sometimes have to change (read replace) people”. As an engineer I am pretty good with the process, tools and methodology side, but the real challenge seems to lie in the organisational change management and organisational process design. And I wonder whether this is really the last frontier and or will there be a next challenge right after I have mastered this one…

The good news is that many of us are on this journey together, and I am confident that on the back of the great results we achieved with tools and methodology alone, truly great things lie ahead of us still as we master the cultural transformation towards becoming DevOps organisations.

The most Controversial Concept in Agile Delivery – Estimating in Story Points

This blog post is another one of those that I should have written a while ago as the topic of story point based estimation keeps coming up again and again. To really understand why story point based estimation is important for Agile delivery, I think I need to explain the idea behind it.

The purpose of estimates is to get a good idea of how much work needs to be done to achieve a certain outcome. To do that, the estimate should be accurate and reasonably precise. This is where the crux of the problem is: precision. If I’d asked you how long it takes to fly from Sydney to Los Angeles, you would not respond with an estimate that includes minutes and seconds because you know that it is ineffective as it is not precise. The more precise we get in estimates, the more we pretend to be able to do something that we cannot do: work at that level of precision. The other downside of precision is that each level of precision requires more work to be put in the estimation process. I have done many IT projects and can tell you that my estimates for each individual task is off by as much as +/- 100% easily, but in aggregate my estimates are pretty good.

Let’s explore the difference between accuracy and precision a bit further:

accurate vs precise

It should be clear that we care more about accuracy than we care about precision and that is exactly what story points do for me. I am spending just the necessary amount of time estimating to be reasonably accurate without trying to become too precise. The usual Fibonacci sequence (1,2,3,5,8,…) helps to avoid false precision as well. Now, to be honest we could call it 1,2,3,5,8 days and be done with it as that would probably achieve the same outcome as story points. The problem is that for some reason we are a lot more tempted to use the other in between numbers when we talk about days. We are also tempted to equate days of effort with schedule, and most of us can attest that a day of effort is hardly ever done in a day of schedule as we get distracted, multi-task or attend meetings. The story point concept provides us with a nice abstraction that prevents these mental shortcuts and keeps us focused on the relative nature of the estimate.

The other thing that should be obvious is that a day of effort for one person is not the same as a day of effort for another person. More experienced people need less time than more junior people, so any estimate in hours or days is flawed unless you know who will do it. Story points do not suffer from this problem as they are relative to other stories and independent to the person performing the tasks associated with it.

The other nice thing with Agile estimation is that it usually is a lot closer to the often recommended Delphi technique, which asks multiple independent experts to estimate tasks and then aggregate it. Planning poker is a pretty close approximation of the Delphi technique and is therefore much more accurate than estimates done by individuals.

But why do we need a point system at all, why do we not just do relative sizing in t-shirt sizes or something similar. As I have explored in another blogpost (link), teams need a goal line whenever there is a certain outcome to be achieved. The easiest way to do so is by tracking progress on a numerical scale (see Agile reporting post). And if you work in a larger organisation you probably want to have some common currency to be able to measure the throughput (see productivity blog) and be able to swap stories between teams. Here I will go with the guidance that SAFe provides, start with a point being roughly a full day of work and estimate everything else relative to that. On a regular basis bring members of the team together to estimate an example set of stories and use this process to recalibrate the relative understanding of size.

So what if things change? One thing that people are always concerned about is scope creep or inaccurate estimates. For a story by itself I don’t have strong opinions on whether or not you update the size once you realise there is more or less work than expected. However, if you use larger buckets for your initial estimates (e.g. a feature that should roughly take you 100 pts), then I think it is important to measure how many points the stories of that feature actually add up to – if that is different to 100 pts in this case you have some real scope change that will impact your timelines.

To close off, I will provide a few helpful links to other comments/blogs about story points which you can use to learn more about this topic:

http://www.scruminc.com/story-points-why-are-they-better-than/

http://collaboration.csc.ncsu.edu/laurie/Papers/ESEM11_SCRUM_Experience_CameraReady.pdf

http://www.mountaingoatsoftware.com/blog/seeing-how-well-a-teams-story-points-align-from-one-to-eight

https://www.scrum.org/Forums/aft/564

http://www.mlcarey321.com/2013/08/normalized-story-points.html

http://blogs.versionone.com/agile_management/2013/10/14/scalable-agile-estimation-and-normalization-of-story-points-introduction-and-overview-of-the-blog-series-part-1-of-5/

Sure ways to fail #2 – Not knowing what the goal is

“If you don’t know where you going, any road will take you there” – The Cheshire Cat in Alice in Wonderland

The_Cheshire_Cat_by_JTwilight97This quote from Alice in Wonderland is very insightful and I have quoted it many times when talking about strategies and plans. In this case I want to use this as an illustration for a way an Agile project will surely fail. If you don’t know what the goal line is for a specific release or sprint, you have no means to understand how you are travelling and whether you are on the right track. I have seen many teams post great velocities in their first couple of sprints, but when asked whether they can achieve the release goal they have no idea. This is painful and leads to a lot of anxiety for the team and stakeholders.

How does this happen? I believe that this is due to the fact that in Agile we don’t want to spend too much time planning and estimating too much to the future and hence start kicking off projects really quickly. We do this with just the first one or two sprints worth of stories ready for implementation and then we keep grooming the backlog. This is great, but if you choose to do so, I think you need to spend a little bit of extra time to give the team a final goal. You don’t need to have all the stories for the release, after all we want to be flexible in Agile, but you should have some idea of the things you need to deliver. This can be stories, themes, epics, features or whatever you choose. At this stage you should do a quick estimation to provide the overall scope for the release and a goal line that the team can use in their burn-up graphs (read more on reporting here). You can then track changes to the goal line if epics require more stories than expected or if any new scope is introduced. This allows the team to have meaningful discussions with the product owner and the stakeholders about required changes to the release. If you don’t have such a goal line, you could get a shock surprise towards the end of your release and surprises are not something we want with Agile delivery.

The one caveat is if you work in a project that truly does not know what the scope is or the scope is undefined. What you can do in those cases is either work in a Kanban style or with an assumed target velocity. If you work in Kanban you only ever have to worry about the first few items in your backlog and set expectations with your stakeholders on how long they will take and then do the same for the next set of stories on a regular cadence. This requires a lot of trust between the team, product owner and stakeholders. Alternatively you can set yourself a certain target velocity and just work towards that and fill the velocity with stories that are getting groomed on an ongoing basis.

Agile Reporting at the enterprise level (Part 2) – Measuring Productivity

productivityThose of you who know me personally, know that nothing can get me on my soapbox quicker than a discussion on measuring productivity. Just over the last week I have been asked three times how to measure this in Agile. I was surprised to notice that I had not yet put my thoughts on paper (well in a blog post). This is well overdue so here I share my thoughts.

Let’s start with the most obvious: Productivity measures output and not outcome. The business cares about outcomes first and outputs second, after all there is no point creating Betamax cassettes more productively than a competitor if everyone buys VHS. Understandably it is difficult to measure the outcome of software delivery so we end up talking about productivity. Having swallowed this pill and being unable to give all but anecdotal guidance on how to measure outcomes, let’s look at productivity measurements.

How not to do it! The worst possible way that I can think of is to do it literally based on output. Think of widgets or java classes or lines of code. If you measure this output you are at best not measuring something meaningful and at worst encouraging bad behaviour. Teams that focus on creating an elegant and easy to maintain solution with reusable components will look less productive than the ones just copying things or creating new components all the time. This is bad. And think of the introduction of technology patterns like stylesheets, all of a sudden for a redesign you only have to update a stylesheet and not all 100 web pages. On paper this would look like a huge productivity loss, updating 1 stylessheet over updating 100 pages in a similar timeframe. Innovative productivity improvements will not get accurately reflected by this kind of measure and teams will not look for innovative ways as much given they are measured on something different . Arguably function points are similar, but I have never dealt with them so I will reserve judgement on this until I have firsthand experience.

How to make it even worse! Yes, widget or line of code based measurements are bad, but it can get even worse. If we have done measurements on this we do not incentivise teams to look for reuse or componentisation of code, and we are also in danger of destroying their sense of teamwork by measuring what each team member contributes. “How many lines of code have you written today?” I have worked with many teams where the best coder writes very little code and that is because he is helping everyone else around him. The team is more productive by him doing this than by him writing lots of code himself. He multiplies his strength rather than linearly growing the team’s productivity by doing more himself.

Okay, you might say that this is all well and good, but what should we do? We clearly need some kind of measurement. I completely agree. Here is what I have used and I think this is a decent starting point:

Measure three different things:

  • Delivered Functionality – You can do this by either measuring how many user stories or story points you deliver. If you are not working in Agile, you can use requirements or use cases or scenarios. Anything that actually relates to what the user gets from the system. This is closest to measuring outcome and hence the most appropriate measure. Of course these items come in all different sizes and you’d be hard pressed to strictly compare two data points but the trending should be helpful. If you did some normalisations of story points (another great topic for a soapbox) then that will give some comparability.
  • Waste – While it is hard to measure productivity and outcomes, it is quite easy to measure the opposite: waste! Of course you should contextually decide which elements of waste you measure and I would be careful with composites unless you can translate this to money (e.g. all the waste adds to 3MUSD, not, we have a waste index of 3.6). Composites of such diverse elements such as defects, manual steps, process delays and handovers are difficult to understand. If you cannot translate these to dollars, just choose 2 or 3 main waste factors and measure them. Once they are good find the next one to measure and track.
  • Cycle time – This is the metric that I would consider above all others to be meaningful. How long does it take to get a good idea implemented in production? You should have the broadest definition that you can measure and then break it down into the sub-components to understand where your bottlenecks are and optimise those. Many of these will be impacted by the levels of automation you have implemented and the level of lean process optimisation you have done.

This is by no means perfect. You can game these metrics just like many others and sometimes external factors influence the measurement, but I strongly believe that if you improve on these three measures you will be more productive.

There is one more thing to mention as a caveat. You need to measure exhaustively and in an automated fashion. The more you rely on just a subset of work and the more you manually track activities the less accurate these measures will be. This also means that you need to measure things that don’t lead to functionality being delivered, like paying down technical debt, analysing new requests for functionality that does not implement or defect triage. There is plenty of opportunity to optimise in this space – Paying technical debt down quicker, validating feature requests quicker, reducing feedback cycles to reduce triage times of defects.

For other posts of the Agile reporting series look here: Agile Reporting at the enterprise level – where to look? (Part 1 – Status reporting)

Here is a related TED talk about productivity and the impact of too many rules and metrics by Yves Morieux from BCG