Author Archives: Mirco Hering

Impressions from the DevOps Enterprise Summit 2014

8 Replies

I spent the last 3 days at the DevOps Enterprise summit here in San Francisco and wanted to share my thoughts with those that couldn’t come over here. Overall it was a great conference, especially if you consider that this was the first time this conference was organised. A few glitches, but that just made it more likeable. And I am sure it will be even better next year. And I have to admit that I hope not hear about horses and unicorns for a few days…

So what did I take away from the conference? Here are a few of the themes that were pretty common through the 3 days:

  • DevOps Teams – While there were certainly exceptions (most notably from Barclays), most organisations that spoke seemed to have a dedicated shared services DevOps team to focus on the tooling, governance and support of their DevOps platform. This is certainly my preferred approach as well and it was good to see that many organisation have made positive experiences with this. But it was also good to hear positive stories from organisations that have chosen a more federated approaches and to learn how they approached it successfully.
  • DevOpsSec – While obvious in hindsight, the frequent mentioning of information security as a critical element in the DevOps journey really brought this home for me. And as some of the speakers highlighted the ability to automated the compliance to regulations and policies is so powerful, that information security can actually be your ally in the DevOps journey and not a blocker. Great change of perspective for me personally.
  • Balance of Culture and Technical Practices – Not surprisingly a lot was being said about culture change and also about technical practices for DevOps. I think this balance is important and is good for us to keep in mind in our day-to-day as we sometimes get to focussed on only one side of the equation.
  • Internal Conferences – So many companies use internal conferences to spread the word and share experiences across the organisation. This is fantastic to hear and I am glad they are able to get the support for it as it can be hard to make a quantifiable business case for those as I know from experience.
  • Servers are cattle not pets – A lot was being said about the importance of having environments that are commodity and consistent, so that you can replace servers easily and reliably. Quite a few of the tool vendors were from a server monitoring, configuration management drift detection space as well. This clearly deserves more focus going forward.
  • Tooling – A completely non scientific impression is that certain tools are much more prolific than others in the DevOps toolkit, examples are: Jenkins, Atlassian tools, Git.
  • Measuring everything – Not really a new thought, but interesting to see how many of the organisations had good data to support their story. So important to get this right and use it to drill down on bottlenecks and cost sources.
  • Scaled Agile Framework – SAFe got a lot of positive mentioning by the speakers, seems to be widely adopted at large enterprises.
  • A few smaller takeaways:
    • Impact Score of Releases – I like the idea of measuring the impact of releases by measuring the sum of (number of Defects x severity). Brilliant.
    • Inverse Taylor Manoeuvre – Such a good name for self-enabled teams
    • Inverse Conway Manoeuvre –  A great name for addressing the architectural challenges that many of us face with existing architecture
    • Release notes as blog – Such a good idea to not send notes around but rather use a blog to document all release changes
    • Sprint Plan review meeting – A meeting after the sprint plan to get all relevant stakeholder across the plan (like Ops, InfoSec, Business). Great idea to test.
  • Favourite Quotes:
    • “Branches are evil”
    • “There is a right way to develop software (and DevOps is it)”
    • “We geeks don’t just like SkyNey – we want to build it”
    • “Cease dependence on mass inspection to achieve quality”
    • “Just talking nicely to each other does not delivery software”
    • “Time does not make software better”

A few things could be improved going forward in my opinion:

  • Many of the Enterprise Scale organisations were talking about their Web presence or Digital space, and only a few talked about the DevOps transformation for their Systems of Record. A better balance would be nice, to not only hear the positive stories and learn from the really difficult cases
  • One aspect that was only mentioned as a sidenote and by 2 or 3 speakers is the reality of working with many different vendors and systems integrators. How do you enable this multi-party setup for DevOps practices? Having been on both sides of that story, perhaps I should share my experiences next year…
  • The sessions were pretty much back-to-back and there was little time for Q&A and to ask informal questions. Perhaps a short break in between sessions or a more formal way to socialise with the speaker right after the session would be good. I have seen this very successful at other conferences.

And last but not least a shout-out to some of the outstanding speakers from the conference, if you get a chance check-out the recordings later in the week when they are available on the conference website at http://devopsenterprisesummit.com.
– Gary Gruver
– Em Campbell-Pretty
– Jason Cox
– Mark Schwarz
– Owen Gardner
– Carmen DeArdo
– to highlight just a few, there were many more that are worth listening to if you have the time

Agile Reporting at the enterprise level – where to look? (Part 1 – Status reporting)

4 Replies

In this week’s post I will talk about a topic close to my heart. How do you manage Agile projects if you have some scale. And especially how do you report on progress and use the lessons from your projects effectively across the enterprise? Let’s start by looking at what we want to get out of the reporting.

What should a good report show you:

  1. How has the scope changed over time?
  2. How much scope have we successfully completed?
  3. How is the team progressing towards the goal over time?
  4. How are we doing with cost?
  5. How is this team progressing compared to other teams?
  6. Which teams should I help to get them back on track? 

So here is the first example of a “traditional” Agile release burn-up that gives us information for points 1-3:
burnupSo how do you read this report? The first line shows how the target scope developed over time. In this case you can see that the number of story points required moved a bit up and down as stories were delivered. This is pretty normal for an agile project, nothing to worry here. We have an ideal line to show us what the “ideal” amount of accepted story points would be, of course no team ever delivers according to this line. If they do, I would ask a few questions as it just looks too good to be true – perhaps we bend the truth a bit in those cases to not explain the difference…

The actual burn-up and the predictions that the team tracks let me to the conclusion that this will be close, but that they can make it. This is a team I will keep an eye on going forward to see whether they catch-up as predicted. As the overall trend of total scope is also slightly downwards, there is a chance that the trend continues to a number even below 911 story points, which would be good for this team. 

Challenges with Agile reporting (points 4 & 5) 

If you are like me and are working in an enterprise environment it is likely that your teams use different iteration schedules, don’t have an aligned story point structure and are of different size. If you look at the release burn-up graphs, you can hardly compare them and by looking at many across the enterprise you don’t necessarily are able to compare their progress. Here is where my favourite report comes in – the relative burn-up that answers point 5 from our list of requirements.

relative burnupThe beauty is that you can show all kinds of things on this graph. In the example shown here, we track the progress of accepted scope against the budget (point 4) that we have used (and yes we are falling behind, looks like we need more budget to deliver our scope). Of course you can do the same for scope vs. schedule. And because these reports are relative to 100% of scope, budget or schedule you can now compare your status reports across teams. We are all aware that status is not linear and that the ideal line is mostly to be ignored, but if after the release has finished you look back at the teams that were successful and those that had challenges you can look for patterns. What percentage of scope did successful teams delivery after 50% of their schedule? How much budget did they have left after they completed 90% of the scope? Through this insight you can develop a portfolio view that I describe later.

One of the biggest challenges: How accurate is the data

Before I do that I want to share one more challenge: Real-Time data. It is near impossible in my experience to get good reporting if you have to manually collate the data and/or the report is purely used for outside stakeholders. To make your reporting effective you should explore ways to make the report automatic based on data that your team already provides rather than making it a separate activity. And to avoid the challenge of stale data, make it part of your stand-up to look at the report; this will help keeping people motivated to update their data. Ideally in any status meetings you have you and your stakeholders look at the tool you use and not extracts, excel sheets or PowerPoint slides; but I have not seen many organisations doing this successfully. 

And now to the promised portfolio view (point 6):
How to keep track of many projects or teams in your organisation? In the manual for Agile it says that you should get your status by attending stand-ups. How do you do that if you have 20 teams working at the same time? I think we all understand that some kind of reporting will be required and as long as it is helpful and used for the right purpose (to provide help where required, not to call teams out where they struggle) we all want that kind of visibility. If you have the relative reporting in place that I mentioned above, you can now use this to get a good view of your teams and you can chart your learning from the past into the report. It could look like this (this is still conceptual as I have not yet implemented it on my program):

portfolio

If you use the relative burn-ups you can now draw each of the teams on portfolio map like the above. Depending on how far they are through their scheduled iterations and how much scope has been accepted you position them on this map. From your experience with successful and challenged teams you know which area is green, amber and red (this will differ from organisation to organisation). But now you can quickly see for which teams it is worthwhile to look at them a bit closer. In the above example the size of the team (size of bubble) and the planned release date (colour) are also shown. As enterprise coach I would talk to the team represented by the large red bubble at 50% of schedule and the small red bubble at 85% of their schedule. At this point you will have to understand the context of these teams to see whether they need help or not. This diagram makes watermelon status (appearing green from the outside but really being red) less likely as it is purely based on the agile burn-up information and your experiences in the organisation. I will let you know how well this works once I have implemented it successfully.

Now this is one aspect of reporting done (status), I will talk about how to report on delivery health in the next post, where we will talk about a delivery health score card and cumulative flow diagrams. As always reach out if you have thoughts or perhaps even good examples.

Sure Ways to Fail #1 – No Time for Retrospectives

Leave a reply

8226451812_88007f08df_zThis is the first post in a series of posts about anti-patterns that I have seen in my travels. I thought it is helpful for people to understand what is common in projects that fail so that they can learn from others’ mistakes and failures. Over time I will post more and more patterns and I will also post signs of success in a parallel series of blog posts. While these posts are numbered, they will not strictly be in priority order and will rather follow the same pattern as my other posts, which are timely to discussions I have had close to the point of writing the post. Perhaps later I prioritise them, but for now let’s get them on paper.

The number 1 reason I see in teams that fail is that they don’t have effective retrospectives. This can have many reasons, but the most common one I see is that the team does not have time because delivery is taking all their time. They are already working overtime and now they are asked to spend an hour on a retrospective. Clearly that can wait until next iteration. You might ask: how the next iteration will get any better if there is no time to look at what went wrong this time and to improve it. And you are correct. It is likely that the next iteration will end up consuming all the time and there again wont be time for a retro. In my opinion it is not often the case that there is really no time, it’s just a lack of understanding of the importance of the retro and perhaps the experience that previous retro wasn’t effective. One sure way to get teams out of the habit of running retros is when there is no action out of the retro, the team is bringing up all kinds of challenges they had and everyone is happy to pitch in and perhaps actions are also discussed on how to improve things, but then nothing happens. Sometimes the iteration is already committed and there is no room to action anything above and beyond the committed scope. Sometimes the ScrumMaster is not willing to pursue organisational challenges that require escalation and rather waits it out. Sometimes it is simply that no owner was assigned. Whatever it is, if after a couple of retros the team is still highlighting the same challenges and nothing is being done about it, they will question the value and then find it harder and harder to find the time to even do a retro. So perhaps it is not a time problem after all.
I personally like this cartoon, which pretty much sums it all up:
http://listcrown.com/wp-content/uploads/2013/10/busy.jpg

So how do you avoid this sure way to fail:

8002453131_7fd9489dfd_zHold regular retrospective – The first remedy is to hold retrospectives regularly and to make sure the team participates. One thing I learned is that there is not one format that works for everyone. You will have to experiment with the team until you find the right duration and activity to get maximum participation. And over time you should mix it up. To put post-it notes under two columns for good and bad can get pretty boring and take away from the value that the retrospectives add to the team.

Follow-up with action – The best retrospective does not help if it is just a session for feedback and then people leave and the system cards and ideas remain in the room. You need to make sure that at least 1-2 actions are taken from each retrospective session and that these actions are visible to the team (ideally on their Kanban wall). If you are the Scrum Master and there have been actions for the organisation that have come up in the retrospective and that the team thinks are important, then you need to make sure that you are accountable for doing something with it. I have attended retrospectives with teams over time and the motivation of the team is proportional to the level of action that comes out of a retrospective.

Bubble up what needs to bubble up – While many actions from the retrospective are internal to the team, there are items that are of interest to the whole organisation and where teams can learn from each other. One risk of the very team focuses ceremonies of Agile is that teams learn but not the organisation. It is therefore that the team and the Scrum Master and any coaches involved make sure that insights from retrospectives are being bubbled up to the organisational level to allow for that exchange of lessons learned and that actions can be taken for things that impact many different teams.

Pictures:
Crossroads: Success or Failure by Chris Potter from www.stockmonkeys.com
License: Creative Commons
Retro-spective is here .. by Paul Downey from https://www.flickr.com/photos/psd/
License: Creative Commons

Continuous Everything in DevOps…What is the difference between CI, CD,CD,…?

9 Replies

Okay so training was more work than expected, hence I will now slowly make my way through the backlog of topics. We will start with some the different techniques being used in DevOps. I will move the definitions to my definitions page as well, as I will refer to them again and again over time I am sure.

Continuous Integration (the practice)
This is probably the most widely known in this list of practices. It is about compiling/building/packaging your software on a continuous basis. With every check-in a system triggers the compilation process, runs the unit test, runs any static analysis tools you use and any other quality related checks that you can automate. I would also add the automated deployment into one environment so that you know that the system can be deployed. It usually means that you have all code merged into the mainline or trunk before triggering this process off. Working from the mainline can be challenging and often concepts like feature toggles are being used to enable the differentiation between features that are ready for consumption and features that are still in progress. This leads to variants where you run continuous integration on specific code branches only, which is not ideal, but better than not having continuous integration at all.

Continuous Integration (the principle)
I like to talk about Continuous Integration in a broader sense that aims at integrating the whole system/solution as often and as early as possible. To me Continuous Integration means that I want to integrate my whole system, while I could have a Continuous Integration server running on individual modules of the system. This also means I want to run integration tests early on and deploy my system into an environment. It also means “integrating” test data early with system to test as close as possible to the final integration. Really to me it means test as far left as possible and don’t leave integration until Integration Test at the end of the delivery life-cycle.

Continuous Delivery vs. Continuous Deployment
What could be more confusing than having do different practices that are called CD: Continuous Delivery and Continuous Deployment? What is the difference between CD and CD. Have a look at the summary picture:CDvsCd

As you can see the main practices are the same and the difference is mainly in where to apply them. In Continuous Delivery you aim to have the full SDLC automated up until the last environment  before production, so that you are ready at any time to deploy automatically to production. In Continuous Deployment you go one step further, you actually automatically deploy to production. The difference is really just whether or not there is an automatic or manual trigger. Of course this kind of practice requires really good tooling across the whole delivery supply chain: everything that was already mentioned under continuous integration, but you will have to have more sophisticated test tooling that allows you to test all the different aspects of the system (performance, operational readiness, etc.). And to be honest I think there will often be cases where you require some human inspection for usability or other non-automatable aspects, but the goal is to minimise this as much as possible.

Continuous Testing
Last but not least Continuous Testing. To me this means that during the delivery of a system you keep running test batteries. You don’t wait until later phases of delivery to execute testing but rather you keep running tests on the latest software build and hence you have real-time status of the quality of your software and if you use Test-Driven-Development you have real-time status of progress. This is not terribly different to the others mentioned before but I like the term because it reflects the diffusion of testing from a distinct phase to an ongoing, continuous activity.

I hope this post was helpful for those of you who were a bit confused with the terms. Reach out with your thoughts.

Technical Debt or the Tyranny of the Short Term

Leave a reply

This week you will see more content than the usual once per week posting. I am training technology architecture and as such I will include a daily post as basis for discussions in the course. Post the discussion in class I will update the post with a summary of the discussion. I hope you find this an interesting concept as I am experimenting with the blog medium. On to the first topic of four: Technical Debt.

Much has been said about technical debt and how hard it is to pay it down. In the last week I had a few discussions about this and I thought I put my thoughts down on paper (or really the keyboard).

What is technical debt?
To get everyone on the same page let’s define what technical debt actually means. Technical debt is best explained by what it causes. Similar to debt, it causes interest over time. And as you all know interest is basically paying money for which you don’t really get anything in return. You are paying for an earlier decision (e.g. a purchase you made when you could not yet afford it) and if you don’t start paying down the debt then you will be able to afford less and less with the same amount of money as the interest takes over.
DebtIn IT what happens is that you set out to implement a new solution and of course you try to deliver the best solution possible. Over time decision points come up where you could implement something that costs a bit more but would provide better maintainability later on, like automated unit testing or separating out a function that you might reuse later rather than keeping it within a different function. You now need to decide whether to invest the extra time/money for this non functional aspect or to focus purely on the functionality that your business stakeholder requires. Every time you choose the short-term solution you increase your technical debt as next time you want to change something or use the functionality that you could have split out, you now require an additional effort. Of course there are many more ways to incur technical debt than just the lack of automation or modularisation, but these serve as examples.

Why is it so hard to avoid technical debt?
The crux of the matter is that by making all the right decisions (according to some criteria), you can still incur an increasing amount of technical debt. Imagine you are working on an application and you have exactly one project and you don’t know whether there will be any other projects after you. Should you still make the investment in good automation and modularisation practices? What if you know there are other projects, but you don’t know whether it will impact the same areas of the code or would use the same automation? …
You can see its a slippery slope.
tech debtLook at the graph on the left. It shows the total cost of change over time, initially it is cheaper to just implement the functionality without investing in good practices, but then over time the cost of changes increases as the technical debt makes it more costly to make changes. At some stage the total cost of change means each change is now more expensive than if you had implemented all the good practices from the beginning, but now you have to pay down all that debt and it is costly to jump back to the other cost line. You also see that even with great practices the cost of change generally increases a bit over time, although there are people arguing that great modularisation and reuse can actually reduce the total cost of change over time as you recombine existing services to create new ones, but that is for another post.

What does it take to pay it down?
The challenge with paying down technical debt is that it usually takes a lot of time and while you can accelerate it through a dedicated program, the only long-term solution is to leave the software in a better state every time you make a change. Otherwise you run a “paydown” project to reduce technical debt but then increase it with each subsequent functional project until you do the next “paydown” project. If you do it little by little you will have a much more sustainable model and the cultural shift that is required to do this will be beneficial for your organisation in any case. If your Agile implementation can help by making the technical debt more visible and by visually tracking each time you pay a bit of debt down, then you are onto a model that get you to the total cost of change curve that you aspire to. And my personal view is that you need to make sure that the PMO organisation and the project managers are clear about their responsibility in all this.  They should be evaluated not only by the on-time and on-budget delivery of functionality but also by how much they have done with their projects to pay down technical debt, otherwise PMs are too often likely to choose the short-term answer to achieve on-time, on-budget delivery at the cost of technical debt for the next project or in not so kind terms by “kicking the can a bit down the road”.

How can you measure it?
Here I am still without a really good answer…theoretically I think it is the sum of the additional cost that each change costs at the moment minus what it would have costed if you had all the good practices in place. But that is really hard to calculate. Alternatives that i have seen are to create a backlog of all the things that you should have done and to add to it every time you make a short-term decision. The size of this backlog is you technical debt in this case. Not yet great answers, but i keep looking. Please reach out if you know of a better way.

Picture: 3D Shackled Debt by Chris Potter from http://www.stockmonkeys.com
License: Creative Commons

The Software Factory Model Analogy – Appropriate or Not?

3 Replies

I felt compelled after some recent discussions to provide another blog post about the analogy I have been using in the title of this blog: The Software Delivery Factory model.

Let’s talk about the traditional way of thinking about Factories:
We start from the Wikipedia definition of factory system characteristics

  • Unskilled Labor – Now while labor arbitrage has certainly been a factor in the move towards a software factory model, I think we all agree that we are unlikely able to move away from at least a mix of experience levels and that we cannot sustain good software delivery without the right skills. In this model people are usually referred to as resources, but that’s for another post later. Inappropriate analogy!
  • Economies of Scale – By bringing together everyone involved in the delivery process and by centralising some of the functions like PMO we do see some economies of scale. Appropriate analogy!
  • Location – In the past this has been about factories being close to infrastructure like rivers, roads and railways, these days it is to be close to the right talent. This continues to be important as you can see in the move to India and China to get closer to large talent pools there, and also in Silicon valley where a lot of top talent is located these days. Appropriate analogy!
  • Centralisation – In a factory means for production were brought together which individuals were not able to afford (e.g. an assembly line or weaving machine). In software delivery we see heaps of small competitors taking on the big guys with sometimes more advanced open source technology. We also see a lot of distributed teams across the globe who work from different office or even home. Inappropriate analogy!
  • Standardisation and Uniformity – How often do we produce the same piece of software many times over. Not really that often. There are some cases where the same pattern is required for example for pricing changes, but more often than not each project requires a unique solution and is contextual to the client and technology used. Inappropriate analogy!
  • Guarantee of Supply – In a factory the work flows relatively smoothly and there are few hiccups if any in the production process. Looking at data from the chaos report and looking at my own experience, the smoothness of flow in software delivery is an illusion. And to be honest if I see a burnup or burndown graph that is smooth I suspect some gaming is going on. Inappropriate analogy!

So in summary the vote goes to it not being an appropriate analogy 4:2. It conjures up images of people sitting in their cubicles actioning work packages,

  • one person translating a requirement into a design handing it over to
  • the next person writing a bit of code
  • then to the next one testing it
  • and in all this no one talks to each other, it’s all done mechanical like people on an assembly line

In bad times software delivery in a factory model can feel a bit like Charlie Chaplin in Modern Times

Some of my colleagues talked to me about a new factory model, so let’s talk about the characteristics of this alternative model that people point out to me:

  • Orchestration of complex production process – software delivery today does require the delivery of many individual components very similar to the complex production process that for example is required to build a Boeing Dreamliner. Most systems are built of many components who are developed by many different team sometimes even across many locations and organisations thanks to the offshoring and outsourcing models. This examples of a modern factory does apply to software delivery. Appropriate analogy!
  • Automated Tool chains – If you look at modern factories from Toyota or BMW, you see very few works and a lot of automaton chains. This is very similar to this little video on CD. In that regard I agree that software delivery should be like these modern factories. Appropriate analogy!

I guess a modern BMW factory is the right analogy for this model:

Overall we end up with 4:4 votes on this list. In my head the image of a factory is not that of empowerment and of people working together to achieve great unique outcomes, its one of mass production and that just doesn’t work well with my understanding of software delivery. I guess I will keep the name of my blog as it is and just look forward to many more interesting discussions about this topic.

Here are some thoughts from others on Software Delivery Factory models (and yes of course it is more likely I come across things that confirm rather than oppose my view – please call out references to opposing views and I will post them here):

The Abilene Paradox – Can Agreement be dangerous?

2 Replies

Those of you, who have shared a meal, German beer or single malt with me, know that I am passionate about psychology (especially social psychology and cognitive bias). Given how often this relates to my work and the topics in this blog, I think it is time to do my first post about psychology. It’s about the Abilene Paradox (a shout out is required here to the social psychology coursera course from Scott Plous – where I learned about it).

What is the Abilene Paradox?
The Abilene Paradox is when a group reaches consensus to do something that nobody in the group actually wants to do. Everyone ends up doing something they think everyone else wanted, but nobody did. Let me explain this with an example from my personal life  and you can read the original name-giving example here.  So let’s see whether you can relate to this:
wrong wayYou had a pretty busy week and on your way home on friday you think: “This week I hadn’t had much time for my wife, so I will make an effort and we go out together even though I rather stay at home on the couch and watch TV together”. You walk in the door and say: “Honey, what do you wanna do tonight”. She responds: “You must be tired, so perhaps we stay in. It’s OK you know” and internally she thinks: “I had a rough week as well, so we should stay home and have some quiet time”. You then respond to her: “We could go out for dinner?” and to be nice she responds “Ok, that sounds good”. You both head out for dinner and sit in the restaurant and after an average meal on the way home she says to you: “Honey, i wish we would have just stayed home.” You look at her and even though you both agreed on what to do, you ended up doing what neither of you wanted to do in the first place. Now does this sound familiar to any of you?

Example from work
How does this relate to work you might ask. Quite often when looking for solutions people don’t put forward the solution they think is best but rather a solution they either think others expect or (and I have been guilty of this as well) an idea they think is “out-there” just to find out what the reaction is. Now imagine everyone else in the room thinks the same way or just doesn’t want to disagree…all of a sudden you are on your way to Abilene.
It gets worse when you are in an environment where there is a bit of distrust. Imagine you think your boss wants you to run a certain project that you think is doomed. You provide status that is not a lie, but in the grey area to make it look okay and sound upbeat about it because you think your boss wants it. Now he might encourage you to continue working on it even though he has his doubts. Neither of you call it out and you spend days, weeks, months on something neither of you think can be successful.

What to do about it
First of all speak up. It is really hard to get out of this kind of group dynamic if there is no-one who has the courage to speak up. I have heard the expression: “Going to the balcony” in this context and it always reminds of the old Waldorf and Statler from the Muppets. It means look at the situation mentally from a few feet away. More than once in my life have I said: “Guys, let’s take a step back. Is this really what we want to do? Who here really thinks this is the best option?” In the past I always wondered what happened to us, now I know we were on our way to Abilene…
There are many ways to try to avoid this, and a lot will depend on the context. You can try a secret vote to find out the real deal, you can ask someone to actively propose an opposing view or you can use fist to five  and look out for 3s (if the majority has a 3, you might want to dig a bit deeper).

I hope you enjoyed this first trip into psychology on my blog, there will be more to come. Stay tuned until next week.

Picture: Why doesn’t anyone read the signs? by Neil R
taken from https://www.flickr.com/photos/islespunkfan/
under Creative Commons license

 

Agile, DevOps and Design Thinking – How do they relate?

4 Replies

This week I had several discussions in which I had to explain how I see the relationship between Agile, DevOps and Design Thinking. Of course there is no clear differentiation and for that matter definition of these terms (or if there is then certainly not everyone is referring to it in the same way).

Let me start by defining these terms in my context:

Agile (in the context of Agile software delivery)
A set of adaptive methods to deliver software based solutions based on the Agile Manifesto. It is an umbrella term for delivery methods like SCRUM and Kanban and for engineering methods like XP. From a cultural perspective these methods are meant to bring the customer or business stakeholder closer to the IT organisation through closer collaboration and by making delivery less black box and more white box.

DevOps
I personally like the following definition:
“Using governance and automation techniques to optimise collaboration across development and operations to enable faster, more predictable and more frequent deployments to market”.
From a cultural perspective this is bringing down the barriers between the development and operations parts of the organisation to achieve the right balance between stability or reliability and the required changes to deliver to the end-customer.

Design Thinking
This is the process of identifying and defining what a solution should look like by emphasizing with the subjects in question, by creatively solving the problem at hand and by analytically testing whether or not the solution is feasible from a technical perspective and in the problem context (viable for the customer and the business).

Now that we have the definitions out-of-the-way – which I will also use to start my definition page here on the blog – lets look at how all this relates to each other. I will start with a picture that I started to use a while ago:
House of AgileFor me this explains the relationship quite nicely, but not comprehensively. It shows how the three ‘pillars’ that I described above work together to hold up the IT operating model and how it is based in the cloud. Clearly there is more to it than this picture shows and what is missing are the overlaps and differences. This all sounds very esoteric, so lets dig a bit deeper.

So let’s look at the first two pillars: Design Thinking & Agile.
If you pick up the original book by Ken Schwaber it pretty much starts to describe what happens when the Product Owner comes to the team with a list of prioritized items to implement. When I first read the book I had the same reaction that probably many of you had – “If Agile tells me ‘How’ to implement something, how do I find out ‘What’ to implement?”
Design Thinking can be an answer to this question, there are some groups that are really good at this like d.School or Fjord. In my travels it has often been a slightly less elaborate activity like a 1-2 week Discovery phase where IT and business come together to define the solutions, but ideally you use Design Thinking to come up with great solutions. In my MBA I had the pleasure to work an expert in this field and to go through a design thinking workshop and it certainly provided a very different perspective on the problem at hand. In a later post I will describe another aspect of this phase which can be called Design Slicing – the ability to define logical slices that provide value by itself.

Let’s move to the second set of pillars: Agile and DevOps.
Here is becomes more complicated. The previous comparison we could simplify to Design Thinking = What, Agile = How, for Agile and DevOps we wont be able to make such a clear differentiation. Really good Agile adoptions focus on the cultural change required, the methodology changes that come with SCRUM and Kanban among others and focus on the technical practices like the ones that XP describes. DevOps in a similar vein talks about cultural change and technical practices. Here is now where I take a pragmatic approach, for me the methodology aspects are sitting within the Agile space and seeing Post-It notes, burn-up graphs and stand-ups are indications that someone is adopting Agile – whether successful or not requires more than Post-Its by the way. If someone is changing the way software is being coded and deployed and the change is much less visible in the offices (perhaps you can see green and red build lights) then we are talking about DevOps. This differentiation also allows me to break with the conventional wisdom that DevOps and Agile go together. They are definitely better together (like a good meal and a good wine – great individually, even better together), but you can get value from one without the other – just not as much as you could from both of them working together. If I force myself to simplify the difference between the pillars, I would say Agile = Bringing business and IT together supported by methodology, DevOps = Bringing development and operations together supported by technical practices.

I have not spoken about the IT Operating Model and the Cloud.
Let me spend a few words on this as well, by starting with the IT operating model. One of the things I notice when I speak to clients is that no matter how well their Agile and/or DevOps adoptions go that there is a lurking problem that requires addressing as well and that is the IT operating model. Again this is a term that can mean many things to many people. I will highlight a few aspects of what I mean. If you really change the way you deliver software based solutions by using Agile, DevOps and Design Thinking you will likely run into challenges with your existing IT operating model in the following ways:
Funding – Your funding and budgeting process might not allow you to progressively learn and adapt but rather requires locking things down early and measure against that plan.
Workforce management – How can you change from assembling people for projects and disbanding the team at completion to standing teams that work flows towards. And how should these teams look like – teams representing value flows, a central DevOps team or federated accountability, release trains a la SAFe,…
Incentives and Commercial constructs – How do you make sure that all your employees, contractors and delivery partners/systems integrator share your goals and can support the new way of working?
Roles and Responsibilities – How do you need to change role descriptions to make the new way of working stick?
All these are aspects that are not necessarily covered in your Agile methodology or DevOps practices, but that require thinking about and adjusting. And I like to consider this a change of the IT operating model.

And of course we should talk about the cloud – there is lots to say here, but lets leave it with one sentence for now: To achieve the ultimate flexibility and speed to market many aspire to you will have to make effective use of the Cloud (private, public or mix).

Longest post so far, so I will say goodbye for now. Post your comments – I am looking forward to a controversial discussion of the above.

Are we Agilists in danger of making the perfect the enemy of the good?

Leave a reply

perfectOver the last year or so I have had lots of good robust discussions with other Agile coaches and one thing started to worry me. I heard “But that is not Agile” or “But that is not REAL DevOps” more and more frequently. While I agree that we should always strive for better and better performance, the absolutes seem to me counterproductive. Two topics close to my heart seem to cause this kind of reaction more often than others.

 

SAFe – The Scaled Agile Framework
There is lots of discussion on the internet about SAFe and why it is or why it is not really that Agile. Most organisations that I talk to are nowhere close to the maturity that you assume when you see the SAFe framework. I am sure that there are companies who are further down their Agile path and think that SAFe is very restrictive and old-school. I have to say that most people I talk to would be extremely happy if they could achieve the Agility that SAFe can provide. And it is a framework after all, a bit like a scaffold that you can use to move forward from the old waterfall ways into a more Agile enterprise without throwing everything out. And yes once you think that SAFe is not challenging your organisations and you see opportunities to become even more effective go ahead – push yourself further. For now I quite happy to use the SAFe framework within large organisations to help me speak a language my clients understand and to push them a bit further on their journey. I will have to admit that probably have not spent enough time with the other scaled agile ideas to judge them all – and perhaps writing this blog can be a motivation to do that. For now SAFe is my go-to framework of choice and even those who argue it’s not really Agile, I think even those would agree that many organisations would be more Agile if they had implemented SAFe than they are now – and that’s good enough: For now!

DevOps
So many organisations and projects I encounter do not have the right technical practices in place that allows them to deliver solutions effectively. Practices like Configuration Management, Automation of Build, Deployment and Test, and Environment Management I think belong under the big headline of DevOps. So when I then talk about DevOps practices with peers at conferences and describe that in large organisations I often recommend a DevOps team to start with, I hear “But that is not what DevOps is about”. The so often quoted cultural barriers between Operations and Development in large organisations makes it simply impossible in my view to embedd an operations guys in each development team. And to be honest there are often many more development teams than operations folks who could be embedded. So why wouldn’t I then create a team with representation from both sides to begin with and to get the best guys into that team to solve the difficult technical problems. After all that what Google has done with their Engineering Tools team. I think that is a valid step and yes perhaps afterwards we push this further, but for now most organisations that I have been working with can gain a lot from good practices being implemented through a DevOps team. Having a DevOps team does not mean we don’t want to change the culture, it just means we want to do this one step at a time.

Picture: Perfect by Bruce Berrien
taken from https://www.flickr.com/photos/bruceberrien/with/384207390/
under Creative Commons license

If software delivery is not a factory – what is it?

3 Replies

After posting my first blog I had a discussion with a friend of mine who is an Agile coach. He asked the obvious question: “Mirco, if you think it is not a factory anymore, what is it then?”.

Studio_AaltoNow that caused me to pause, but after a moment I remembered a discussion panel at a conference I attended. The topic was “Software Delivery Metrics” and the whole room was looking for answers on how to measure the effectiveness of software delivery teams or the even more elusive productivity. The discussion went around and around in circles and while I was hoping to get some good answers there was none to be had. People were talking about business outcomes and customer and employee satisfaction – which I agree with but are difficult to measure and to determine cause and effect. Before giving up I threw the following analogy in the room: “So what you are saying is that software delivery is like marketing? When you run a campaign and everyone is happy with the campaign and the product sells more than you consider yourself successful with your marketing campaign. But you don’t know whether your campaign was successful or whether your competitor did something stupid, and you don’t know whether there would have been a more productive way to deliver the campaign” – so perhaps software delivery is not a factory anymore, it is a studio (like a marketing or design studio).

Now this analogy might not be perfect either and I am sure some marketing buffs out there will tell me that marketing is very measurable. Especially if you consider the abilities that came with A/B testing in online marketing, but hey its closer to the truth than the factory model is.

I will close with a quote from John Wanamaker:

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half”

– I leave it to you to see what this means for software delivery.

Picture for this post:
Studio Aalto, by Pepechibiryu, available from: http://en.wikipedia.org/wiki/Alvar_Aalto#mediaviewer/File:Studio_Aalto.jpg
License: CC BY-SA 3.0