Category Archives: Uncategorized

What computer games can teach us about maturity models – Choose your own DevOps Adventure

2 Replies

To use maturity models or not is an eternal question that Agile and DevOps coaches struggle with. We all know that maturity models have some weaknesses, they can easily be gamed if they are used to incentivise and/or punish people, they are very prone to the Dunning-Kruger effect and often they are vague. Of course on the flipside, maturity models allow you to position yourself, your team or your company across a set of increasingly good practices and striving for the next level could be the required motivation to push ahead and implement the next improvement.

Clearly there are different reasons behind different kinds of maturity models. For a self-assessment and to set a roadmap, a traditional maturity model like the Accenture DevOps maturity model is what it takes to get these done. There are many others available on the internet, so feel free to choose the one you like best.

At one of my recent clients we performed many maturity assessments across a wide variety of teams, technologies and applications. Of course such large scope means that we did not spend a lot of time with each team to assess the maturity and not surprisingly the result was that we got very different levels of response. We heard things like “Of course we do Continuous Integration, we have Jenkins installed and it runs every Saturday”, had this team not mentioned the second part of the sentence we would have probably ticked the Continuous Integration box on the maturity sheet.

A few months later we were back in the same situation and needed to find a way to help teams self-assess their maturity in an environment where a lot of DevOps concepts are not well known and different vendors and client teams are involved which means the actual maturity rating becomes somewhat political. I was worrying about this for a while and then one night while playing on my PC, inspiration hit me – I remembered the good old Civilisation game and the technology tree:

techtree_original

Now if I could come up with a technology tree just like this for DevOps I might be able to use this with teams to document the practices they have in place and what it takes to get the next practice enabled. Enter the DevOps technology dependency tree (sample below):

CD Technical Dependencies Tree

In this tree for each leaf we created a definition and related metrics and now each team could go off and use this tree to chart where they are and how they progress. This way each team chooses their own DevOps adventure. We also marked capabilities that the company needed to provide so that each team could leverage common practices that are strategically aligned (like a common test automation framework or deployment framework). This tree has been hugely successful at this specific client and we continue to update it whenever we find a better representation and believe new practices should be represented.

Who would have thought that playing hours of computer games would come in handy one day…

DevOps in Scaled Agile Models – Which one is best?

3 Replies

DevOps in ScaledAgileI have already written about the importance of DevOps practices (or for that matter Agile technical practices) for Agile adoption and I don’t think there are many people arguing for the contrary. Ultimately, you want those two things to go hand in hand to maximise the outcome for your organisation. In this post I want to have a closer look at popular scaling frameworks to see whether these models explicitly or implicitly include DevOps. One could of course argue that the Agile models should really focus on just the Agile methodology and associated processes and practices. However, given that often the technical side is the inhibitor of achieving the benefits of Agile, I think DevOps should be reflected in these models to remind everyone that Software is being created first and foremost by developers.

Let’s look at a few of the more well known models:

SAFE (Scaled Agile Framework) – This one is probably the easiest as it has DevOps being called out in the big picture. I would however consider two aspects of SAFe as relevant for the wider discussion, the DevOps team and the System Team. While the DevOps team talks about the aspects that have to do with deployment into production and the automation of the process, the System Team focuses more on the development side activities like Continuous Integration and Test automation. For me there is a problem here as it feels a lot like the DevOps team is the Operations team and the System Team is the Build team. I’d rather have them in one System/DevOps team with joint responsibilities. If you consider both of them as just concepts in the model and you have them working closely together then I feel you start getting somewhere. This is how I do this on my projects.

DAD (Disciplined Agile Delivery) – In DAD, DevOps is weaved into the fabric of the methodology but not as nicely spelled out as I would like. DAD is a lot more focused on the processes (perhaps an inheritance from RUP as both are influenced/created by IBM folks). There is however a blog post by “creator” Scott Ambler that draws all the elements together. I still feel that a bit more focus on the technical aspects of delivery in the construction phase would have been better. That being said, there are few good references if you go down to the detailed level. The Integrator role has an explicit responsibility to integrate all aspects of the solution and the Produce a Potentially Consumable Solution and Improve Quality processes call out many technical practices related to DevOps.

LESS (Large Scale Scrum) – In LESS DevOps is not explicitly called out, but is well covered under Technical Excellence. Here it talks about all the important practices and principles and the descriptions for each of them is really good. LESS has a lot less focus on telling you exactly how to put these practices in place, so it will be up to you to define which team or who in your team should be responsible for this (or in true Agile fashion perhaps it is everyone…).

In conclusion, I have to say that I like the idea of combining the explicit structure of SAFE with the principles and ideas of LESS to create my own meta-framework. I will certainly use both as reference going forward.

What do you think? Is it important to reflect DevOps techniques in a Scaled Agile Model? And if so, which one is your favourite representation?

8 DevOps Principles that will Improve Your Speed to Market

Leave a reply

I recently got asked about the principles that I follow for DevOps adoptions, so I thought I’d write down my list of principles and what they mean to me:

  • Test Early & Often – This principle is probably the simplest one. Test as early as you can and as often as you can. In my ideal world we find more and more defects closer to the developer by a) running tests more frequently enabled by test automation, b) providing integration points (real or mocked) as early as possible and c) minimising the variables between environments. With all this in place the proportion of defects outside of the development cycle should reduce significantly.
  • Improve Continuously – This principle is the most important and hardest to do. No implementation of DevOps practices is ever complete. You will always learn new things about your technology and solution and without being vigilant about it, deterioration will set in. Of course the corollary to this is that you need to measure what you care about, otherwise all improvements will be unfocused and you will be unable to measure the improvements. You should measure metrics that best represent the areas of concern for you, like cycle time, automation level, defect rates etc.
  • Automate Everything – Easier said than done, this is the one most people associate with DevOps. It is the automation of all processes required to deliver software to production or any other environment. For me this goes further than that, it means automating your status reporting, integrating the tools in your ecosystem and getting everyone involved to focus on things that computers cannot (yet) do.
  • Cohesive Teams – Too often I have been in projects where the silo mentality was the biggest hindrance of progress, be it between delivery partner and client, between development and test teams or between development and operations. Not working together and not having aligned goals/measures of success is going to make the technical problems look like child’s play. Focus on getting the teams aligned early on in your process so that everyone is moving in the same direction.
  • Deliver Small Increments – Complexity and interdependence are the things that make your cycle time longer than required. The more complex and interdependent a piece of software is, the more difficult it is to test and to identify the root cause of any problem. Look for ways to make the chunks of functionality smaller. This will be difficult in the beginning but the more mature you get, the easier this becomes. It will be a multiplying effect that reduces your time to market further.
  • Outside-In Development – One of the most fascinating pieces of research I have seen is by Walker Royce about the success of different delivery approaches. He shows that delays are often caused by integration defects that are identified too late. They are identified late because that’s the first time you can test time in an integrated environment. Once you find them you might have to change the inner logic of your systems to accommodate the required changes to the interface. Now imagine doing it the other way around, you test your interfaces first and once they are stable you build out the inner workings. This reduces the need of rework significantly. This principle holds for integration between systems or for modules within the same application and there are many tools and patterns that support outside-in development.
  • Experiment Frequently – Last but not least you need to experiment. Try different things, use new tools and patterns and keep learning. It’s the same for your business, by using DevOps you can safely try new things out with your customers (think of A/B testing), but you should do the same internally for your development organisation. Be curious!

If you follow these 8 principles I am sure you are on the right track to improve your speed to market.

Working with SIs in a DevOps/Agile delivery model

1 Reply

In this blog post (another of my DevOps SI series), I want to explore the contractual aspects of working with SI (System Integrator). At the highest level there are three models that I have come across:

  • Fixed Price contract – Unfortunately these are not very flexible and usually require a level of detail to define the outcome, which is counterproductive for Agile engagements. It does however incentivise the SI to use best practices to reduce the cost of delivery.
  • Time and Material – This is the most flexible model that easily accommodates any scope changes. The challenge for this one is that the SI does not have an incentive to increase the level of automation because each manual step adds to the revenue the SI makes.
  • Gain Share – This is a great model if your SI is willing to share the gains and risks of your business model. While this is the ideal model, often it is not easy to agree on the contribution that the specific application makes to the overall business.

So what is my preferred model? Well let me start by saying that the contract itself will only ever be one aspect to consider, the overall partnership culture will likely make a bigger impact than the contract itself. I have worked with many different models and have made them work even when they were a hindrance for the Agile delivery approach. However, if I had to define a model that I consider most suitable (Ceteris Paribus – all other things being equal), I would agree on a time and materials contract to keep the flexibility. I would make it mandatory to do joint planning sessions so that both staff movements and release schedule are done in true partnership (it does not help if the SI has staffing issues the client is not aware of or the client makes any ramp-ups/ramp-downs the SI’s problem). I would agree on two scorecards that I would use to evaluate the partnership. One is a Delivery Scorecard, which shows the performance of delivery, things like: are we on track, have we delivered to our promises, is our delivery predictable. The second is an Operational Scorecard , which shows: the quality of delivery, the automation levels in place, the cycle times for key processes in the SDLC.

With these elements I feel that you can have a very fruitful partnership that truly brings together the best of both worlds.

How to support Multi-Speed IT with DevOps and Agile

5 Replies

These days a lot of organisations talk about Multi-Speed IT, so I thought I’d share my thoughts on this. I think the concept has been around for a while but now there is a nice label to associate this idea. Let’s start by looking at why Multi-Speed IT is important. The idea is best illustrated by a picture of two interlocking gears of different sizes and by using a simple example to explain the concept.

The smaller gear moves much faster than the larger one, but where the two gears interlock they gearsremain aligned to not stop the motion. But what does this mean in reality? Think about a banking app on your mobile. Your bank might update the app on a weekly basis with new functionality like reporting and/or improved user interface. That is a reasonable fast release cycle. The mainframe system that sits in the background and provides the mobile app with your account balance and transaction details does not have to change at the same speed. In fact it might only have to provide a new service for the mobile app once every quarter. Nonetheless the changes between those two systems need to align when new functionality is rolled out. However, it doesn’t mean both systems need to release at the same speed. In general, the customer facing systems are the fast applications (Systems of Engagement, Digital) and the slower ones are the Systems of Record or backend systems. The release cycles should take this into consideration.

So how do you get ready for the Multi-Speed IT Delivery Model?

  • Release Strategy (Agile) – Identify functionality that requires changes in multiple systems and ones that can be done in isolation. If you follow an Agile approach, you can align every n-th release for releasing functionality that is aligned while the releases in between can deliver isolated changes for the fast moving applications.
  • Application Architecture – Use versioned interface agreements so that you can decouple the gears (read applications) temporarily. This means you can release a new version of a backend system or a front-end system without impacting current functionality of the other. Once the other system catches up, new functionality becomes available across the system. This allows you to keep to your individual release schedule, which in turn means delivery is a lot less complex and interdependent. In the picture I used above, think of this as the clutch that temporarily disengages the gears.
  • Technical Practices and Tools (DevOps) – If the application architecture decoupling is the clutch, then the technical practices and tools are the grease. This is where DevOps comes into the picture. The whole idea of the Multi-Speed IT is to make the delivery of functionality less interdependent. On the flip side, you need to spend more effort on getting the right practices and tools in place to support this. For example you want to make sure that you can quickly test the different interface versions with automated testing, you need to have good version control to make sure you have in place the right components for each application, you also want to make sure you can manage your codeline very well through abstractions and branching where required. And the basics of configuration management, packaging and deployment will become even more important as you want to reduce the number of variables you have to deal with in your environments. You better remove those variables introduced through manual steps by having these processes completely automated.
  • Testing strategies – Given that you are now dealing with multiple versions of components being in the environment at the same time, you have to rethink your testing strategies. The rules of combinatorics make it very clear that it only takes a few different variables before it becomes unmanageable to test all permutations. So we need to think about different testing strategies that focus on valid permutations and risk profiles. After all, functionality that is not yet live requires less testing than the ones that will go live next.

The above points cover the technical aspects but to get there you will also have to solve some of the organisational challenges. Let me just highlight 3 of them here:

  • Partnership with delivery partners – It will be important to choose your partners wisely. Perhaps it helps to think of your partner ecosystem in three categories: Innovators (the ones who work with you in innovative spaces and with new technologies), Workhorses (the ones who support your core business applications that continue to change) and Commodities (the ones who run legacy applications that don’t require much new functionality and attention). It should be clear that you need to treat them differently in regards to contracts and incentives. I will blog later about the best way to incentivise your workhorses, the area that I see most challenges in.
  • Application Portfolio Management – Of course to find the right partner you first need to understand what your needs are. Look across your application portfolio and determine where your applications sit across the following dimensions: Importance to business, exposure to customers, frequency of change, and volume of change. Based on this you can find the right partner to optimise the outcome for each application.
  • Governance – Last but not least governance in very important. In a multi-speed IT world you will need flexible governance. One size fits all will not be good enough. You will need light-weight system driven governance for your high-speed applications and you can probably afford a more powerpoint/excel driven manual governance for your slower changing applications. If you can run status reports of live systems (like Jira, RTC or TFS) for your fast applications you are another step closer to mastering the multi-speed IT world.

The need for an Onshore Technical Apprenticeship model to counter the falling ladder

2 Replies

This post is a little bit different to my other posts, it highlights a challenge that most of us in the western world is facing, now that offshoring and nearshoring is just a reality of life. I think companies need to think about an onshore technical IT apprenticeship program that allows them to build more technical skills onshore. And it does not matter whether you are a consulting company, an IT shop or a manufacturing business, software is an important part of your business and you better keep building the skills required to run it.

When I started working for my current employer over 10 years ago everyone did some kind of technical work as a new joiner, I personally did some Cobol programming, many of my friends worked with Java. Then as we got more and more senior we had a good understanding of what it takes to develop a solution.

Nowadays if you are trying to staff a technical onshore role, it is actually not easy to find people and often you look to colleagues from offshore centres to help out. A lot of project and program management is still being done onshore and many of the current leaders still come from my generation of “Been there, done that”.
Unfortunately in many IT teams onshore it is today more important to learn exactly 3 technical skills: Excel, Powerpoint and MS Project…

ladderA bit of background:
The problem I describe is what I call the “Falling Ladder” or which in a TED talk was called “sinking skill ladder” (around 10:50 into this video http://www.ted.com/talks/nirmalya_kumar_india_s_invisible_entrepreneurs/transcript?language=en#t-698501) – once you have given all the on the ground technical work to delivery centers, you will soon realise that the people onshore do not have the knowledge to design solutions really well, so you start handing over design as well. Soon you don’t have skills to really project manage anymore as all you can do is manage the spreadsheet, so you hand over project management as well. And then soon all there is onshore is the demand creation either through sales externally or internally. And would you trust a Technical Architect who has never seen code? Okay – this is a slight exaggeration, but you get the gist.
I had a friend who recently left the IT company he was working for and after 2 months of job search we caught up, here is what he said: “It is actually quite hard to find a job if you don’t have any skills beyond excel and MS project, the market is looking for people who understand the technology you are dealing with”. You end up with two choices you get some of the younger guns for the newest set of technologies or someone much older who still has the technical skills of well established technologies, but is that sustainable?

So what should you do:
I think you need to find a way to create the next generation of technical leaders. Technology and specifically software will remain an important part of your business. It is unrealistic to expect to all IT work in-house and onshore, but to really drive technology to the extent that you need to, it is important to retain technical talent. Create some roles in your team that are onshore even if it is unpopular. The investment you make here is small and will pay back easily if you can keep good technical people around and grow them as the next technology leaders in your company.

Picture: The Fall by Celine Nadeau
License: Creative Commons

DevOps for Systems of Record – DevOps Summit Melbourne 21st of November

Leave a reply

Update – The DevOps summit has been postponed until early next year.

This time I just want to provide a quick teaser about my upcoming talk at the DevOps summit in Melbourne about DevOps for Systems of Record. I will highlight in this talk my experiences from two case studies, one with Mainframe and one with Siebel and share lessons learned.

IBM_Blue_Gene_P_supercomputerThe talk will highlight four different principles of dealing with Systems of Records in your DevOps adoption:
Principle 1 – Overcoming Culture
Principle 2 – Tenacity over Vendor Advice
Principle 3 – A word about Maturity Models
Principle 4 – DevOps changes everything

After the talk I will provide a more detailed post about the topic, but for now you will have to come over to the Melbourne Convention and Exhibition Centre to hear me speak about it. Check this link for details about the summit: http://www.devops-summit.org/Melbourne

Sure Ways to Fail #1 – No Time for Retrospectives

Leave a reply

8226451812_88007f08df_zThis is the first post in a series of posts about anti-patterns that I have seen in my travels. I thought it is helpful for people to understand what is common in projects that fail so that they can learn from others’ mistakes and failures. Over time I will post more and more patterns and I will also post signs of success in a parallel series of blog posts. While these posts are numbered, they will not strictly be in priority order and will rather follow the same pattern as my other posts, which are timely to discussions I have had close to the point of writing the post. Perhaps later I prioritise them, but for now let’s get them on paper.

The number 1 reason I see in teams that fail is that they don’t have effective retrospectives. This can have many reasons, but the most common one I see is that the team does not have time because delivery is taking all their time. They are already working overtime and now they are asked to spend an hour on a retrospective. Clearly that can wait until next iteration. You might ask: how the next iteration will get any better if there is no time to look at what went wrong this time and to improve it. And you are correct. It is likely that the next iteration will end up consuming all the time and there again wont be time for a retro. In my opinion it is not often the case that there is really no time, it’s just a lack of understanding of the importance of the retro and perhaps the experience that previous retro wasn’t effective. One sure way to get teams out of the habit of running retros is when there is no action out of the retro, the team is bringing up all kinds of challenges they had and everyone is happy to pitch in and perhaps actions are also discussed on how to improve things, but then nothing happens. Sometimes the iteration is already committed and there is no room to action anything above and beyond the committed scope. Sometimes the ScrumMaster is not willing to pursue organisational challenges that require escalation and rather waits it out. Sometimes it is simply that no owner was assigned. Whatever it is, if after a couple of retros the team is still highlighting the same challenges and nothing is being done about it, they will question the value and then find it harder and harder to find the time to even do a retro. So perhaps it is not a time problem after all.
I personally like this cartoon, which pretty much sums it all up:
http://listcrown.com/wp-content/uploads/2013/10/busy.jpg

So how do you avoid this sure way to fail:

8002453131_7fd9489dfd_zHold regular retrospective – The first remedy is to hold retrospectives regularly and to make sure the team participates. One thing I learned is that there is not one format that works for everyone. You will have to experiment with the team until you find the right duration and activity to get maximum participation. And over time you should mix it up. To put post-it notes under two columns for good and bad can get pretty boring and take away from the value that the retrospectives add to the team.

Follow-up with action – The best retrospective does not help if it is just a session for feedback and then people leave and the system cards and ideas remain in the room. You need to make sure that at least 1-2 actions are taken from each retrospective session and that these actions are visible to the team (ideally on their Kanban wall). If you are the Scrum Master and there have been actions for the organisation that have come up in the retrospective and that the team thinks are important, then you need to make sure that you are accountable for doing something with it. I have attended retrospectives with teams over time and the motivation of the team is proportional to the level of action that comes out of a retrospective.

Bubble up what needs to bubble up – While many actions from the retrospective are internal to the team, there are items that are of interest to the whole organisation and where teams can learn from each other. One risk of the very team focuses ceremonies of Agile is that teams learn but not the organisation. It is therefore that the team and the Scrum Master and any coaches involved make sure that insights from retrospectives are being bubbled up to the organisational level to allow for that exchange of lessons learned and that actions can be taken for things that impact many different teams.

Pictures:
Crossroads: Success or Failure by Chris Potter from www.stockmonkeys.com
License: Creative Commons
Retro-spective is here .. by Paul Downey from https://www.flickr.com/photos/psd/
License: Creative Commons

Welcome to Not A Factory Anymore

1 Reply

Hi everyone,

this is my first blog post. Yes I have finally made the leap into the blogosphere. Over time I will blog here about software delivery, agile, DevOps, Continuous Delivery and anything else that grabs my attention.

But first things first: Why ‘Not a factory anymore’?

I think many of us have heard over time the analogy of Software Delivery with a factory. In my mind that analogy is not accurate anymore and sends the wrong image. Software is not being produced, it is being created, it’s a creative process.

On my blog I will share with you my thoughts on everything that I think shows why the factory model is not appropriate and giving this blog that name leaves no doubt in the reader’s mind, where my allegiance lies in regards to how software based solutions should be delivered.

Looking forward to engage with the community via this blog and to see you around at conferences and meetups