The Dunning Kruger effect (a.k.a. Illusionary Superiority)

If you attended any of my conference talks or spoke to me about maturity models, you would have heard me mention Dunning Kruger. Yet to this day I have not written a blog post about it. It’s well overdue as it is the single best explanation for the behaviours I see in our industry and the source of many of my frustrations. Let’s take a good hard look at ourselves in the mirror.

What is the Dunning-Kruger effect?

The Dunning Kruger effect says that people who are not that knowledgeable about a certain area actually believe that they are quite good at it. Perhaps we start with a few examples:

• 90% of drivers consider themselves to be above average – Daniel Gilbert Stumbling on happiness
• 94% of college teachers consider themselves to be above average – Daniel Gilbert Stumbling on happiness
• In a survey of faculty at the University of Nebraska, 68% rated themselves in the top 25% for teaching ability. – Wikipedia
• In a similar survey, 87% of MBA students at Stanford University rated their academic performance as above the median. – Wikipedia
• How do you think people would rate you as a leader?” It turns out that 74% of the respondents think they’re either above average or the best leader their people have ever had. – SmartBrief on Leadership
• Or consider Lake Wobegon where all children are above average 😉

Perhaps it is best explained by the anecdote that gave it the name, which inspired the original research from Dunning and Kruger:

A man decided rob a bank and did his research to prepare the big heist. He then entered the bank robbed the bank of it’s money and made his way home happy to see everything going smoothly. When he arrived at home the police was already waiting for him and arrested. The man was shocked. In his research he had found out that lemon juice can be used as invisible ink, and so he covered himself in lemon juice to be invisible to the people and cameras due to the invisible ink…

Is Dunning Kruger really a thing?

You might wonder whether Dunning-Kruger is really a thing and why it is important for you. Let me tell where I came across it most often. I do a lot of assessment work to understand where organisations are in their DevOps journey and there is one aspect that keeps baffling me: Continuous Integration. Here is a pretty common dialog that I experience with someone from the development of an organisation, let’s call him Adam:

Mirco: Do you practice Continuous Integration?

Adam: (with a little hesitation) Yes we do.

Mirco: (thinking I could stop here and tick the box, but am curious) How do you know that you are doing CI?

Adam: We have CI server and are using Jenkins.

Mirco: How often does the CI server build your software?

Adam: We run it once a week fully automatically.

Mirco: How often are your developers checking in code?

Adam: Multiple times a day.

Mirco: Does your CI server run cover static code analysis and automated unit tests?

Adam: No.

(…)

I don’t believe that people have the intention of lying in this scenario, I think Dunning-Kruger is working its magic here. I was baffled by seeing this pattern again and again. Once I learned about Dunning-Kruger things fell into place.

Once you know it, you will see it everywhere in the DevOps and Agile world. People believing they are done when they are just at the beginning. People attending conference talks and rather than understanding the difference picking up only the things they have in common to validate their self-evaluation.

This also means that self-assessments are potentially dangerous and that maturity models can be misleading. It might also make you question the validity of some of the surveys you see out there. Well, it might make you doubt yourself? Am I really doing this right or am I falling into Dunning-Kruger?

I think the most important takeaway is that we need to talk more. We need to try to understand the context and the situation much better before we do an evaluation. We need to look for others to take an external look at our situation and compare it as objectively as possible. We need to provide more education for our people as a recipe against Dunning-Kruger. As an alternative to the usual maturity model I once created a “Civilisation”-inspired technology tree for Continuous Delivery with one of my clients, you can read more about this here.

Rather than looking for maturity models, perhaps it is better to look at outcomes and use maturity models as a conversation starter with a coach to help us identify which steps to take next rather than an evaluation of our teams. It’s only one of the tools in our toolbag and we should not put too much emphasis on it.

I will leave you with this final picture that I came across, unfortunately I don’t know the origin (Update: Martin in the comments told me it’s from Simon Wardley), but it does describe the danger zone very nicely in which many people new to Agile and DevOps find themselves:

In November I had one of the most stressful, most nerve-wracking and most amazing days of my life. Of course it won’t compete with the birth of my son or my wedding day, but it was still very special. I was at the DevOps Enterprise Summit to speak and to hand out preview copies (called galleys) of my upcoming book “DevOps for the Modern Enterprise” (available in April 2018 from Amazon, Bookdepository and many other sources).

When I saw a physical copy of the book for the first time in the morning, it was an unreal feeling. There was certainly pride but also a level of disbelief. But damn does it look good 😉 I went on to an interview with Alan Shimmel (which you can find here) to discuss the book. I can’t wait to hear from Alan what he thinks about the book. Before this day only a small number of people had read the book and they are people I consider friends, hence they would have been kind to me in any case, now the book is out in the wild and I am nervously awaiting the feedback from the community, a community that is passionate, opinionated and full of experts in their own rights. Let’s see.

Later in the afternoon I presented my talk “What got you here, wont get you there – a story of transformations”. You can see the video of the presentation here. I think the talk went reasonably well as several people over the next couple of days commented on it with positive feedback. The talk gives a nice glimpses into the content of the book and is a good teaser. If you like the talk, I think you will like the book as it is very similar in tone and content.

And then came the big moment, a 2 hour book signing event in the evening. I was more nervous about this than anything else to be honest. Would anyone even come by my desk when the alternatives are great authors like Mark Schwartz, Gene Kim, Dominica deGrandis and Nicole Forsgren? I didn’t have to worry, many people stopped by to get a signed copy of the book. Lots of people who had been at my talk and wanted to read more. I think I got the hang of the book signing process after a while and apologise to some of the early people in the queue when I was clearly still a bit overwhelmed by the experience. I enjoyed talking to so many people about the conference, the book and my talk.

After the book signing I thought I might go for a drink, but had to admit that my social batteries were depleted after the experience. I am humbled by all the positive comments from people and by so many people from the DevOps community turning up to grab a copy of my book. You made this a really special experience and I hope you enjoy the book. Do let me know what you think.

Happy New year to you all. I am now half-way through my paternity leave and I will have to admit that the “Father of the year” award might be slightly out of reach for me this year. I will get to that, but first let me tell you that I will write to the government and make it clear that this should not be called paternity leave. It is paternity work. I have so much respect for all those full-time parents, your day is fully organised by eating, putting little one to bed, cleaning up, making food, changing baby,… I am keeping up with the schedule but will admit that my wife sent me some handy SMS reminders during the day in the beginning. And then you need to use the few free moments to get the basics done: Shower, eat, check on the Ashes and soccer results,…

You wonder why I don’t think I will win the Father of the year award…well…okay here some not so pretty truth:

• I always thought I would not buy many toys for my kids…but man are they fun. I am buying those toys because its fun to teach him something new with building blocks, and shape sorters, and Duplo blocks, and have you seen the Ballapalooza…you get the gist. By now we are drowning in toys that Papa thought might be nice to play with. And really for an adult who keeps buying lego sets for himself (have you seen the recent millennium falcon set?), should I have really expected anything else from myself?
• Somewhat related is the “we will keep it tidy” idea that I had. And then there is the realisation that your trip from the bedroom to the bathroom at night is now a dangerous endeavour through a minefield of sharp or loud objects…I wonder is it worse to have a sharp pain under your foot or to trigger a loud noise that makes the little one wake up…I let you judge that…I swear the enemy (see picture below) is getting cleverer every day
  
• And of course there is the “we will not let him watch TV until he is old enough”…”Are you thinking what I am thinking B1?”, “I think I am B2”, “It’s papa let’s me watch TV time” Papa gave up and uses a bit of TV once in a while to keep him under control when feeding. And I let you guess which one is his favourite TV show? Oh well I have fond memories of the shows I watched as a kid.

But apart from that the two of us get along well and time flies. We go to weekly swimming lessons together which is fun. We explore the local parks. And since he started walking around Christmas, I am chasing around the house to protect him from the furniture (or is it the other way around?!?).

And then there are those moments when you wonder what is going through his head. I bought him a set of the construction toys, one of them a CAT excavator. And then one of the real ones turns up next door to demolish the house next door. What goes through the head of the little man when his toy all of sudden has a gigantic brother? Is that scary or incredibly cool? I wish he could tell me.

My leave was sweetened by getting promoted to Managing Director in December which gave me reason to celebrate with family – and again just like later at Christmas, I wonder what the little one is thinking? Perhaps: Why is everyone so happy today and why am I not the centre of attention as usual? Our joint birthday is coming up and will mark the end of my paternity leave and for that celebration the little man will be the centre of all attention. The other birthday boy will gladly stand aside and enjoy the loudest birthday party he has had in a long time with the coolest birthday cake (yes I know it’s for the little one, but still 😉) I will post one more time to reflect back before heading back into the office in February.

After writing about behaviours that help new joiners to be successful I finally got around to write about managers. What does a successful manager look like. And here I don’t mean specific practices like the ones you can learn from www.managertools.com but rather the more high level behaviours. This is what I expect from a manager who works for me. It surprises me that so little guidance is provided for new managers (or at least I didn’t know where to look when I first got promoted). The below is what is in my head when I think of successful managers, with many years of managing managers it was well overdue to move this from the frequent coaching discussions into a blog post.

• Servant leadership to set a positive example
  Servant leader sounds good, but what does it mean? A servant leader puts the team before his own interest. There are a number of behaviours that servant leaders show:
  – External blame stops with him – if there is a problem with something that the team has done, a good manager never blames an individual on the team. To the outside world he absorbes the critique and takes it on himself
  – Praise gets spread liberally – The opposite is true for praise. When the team has achieved success, a good manager goes to extra length to make sure everyone in the team gets credit. A servant leader knows that credit multiplies and by giving credit to team members, he will also be getting credit for managing that team
  – Sets good examples by doing what he expects from the rest of team himself. For example he also comes in when he asks the team to work weekends and makes sure the team has what it needs to be successful – and yes that might mean as a manger to get the coffees or pizzas if the rest of the team is busy and the manager can afford the time.
• Ownership of the outcome
  As a manager you own the outcome of your team and yes that includes any dependencies that the team has to manage. As a manager it is your job to get things done with the help of your team – the times of “the dog ate my homework” are now truly over…sadly 😉 You should consider any deliverable that your team produces as your own deliverable, so make sure you are comfortable with it. Trust the team to get it right, but check on it as you need to. You cannot claim ignorance if someone in your team produces sub-par deliverables.
• Has financial and schedule control for the project
  No matter whether it is explicitly stated, the expectation is that a manager knows what budget and schedule he has to work with and that at any point he knows how much of it has been consumed. If it has not been explicitly mentioned, you should get this information and understand how you can track it. If no support is provided create your own tracking mechanism. Communicate the status of budget on a regular basis to make sure that people are aware when things change and can let you know when external factors change (e.g. budget reductions). It doesn’t matter whether you run a small team or a large project this basic is important and the earlier you learn it the better. You might never get asked for it (good news) but when you are (mostly when things start to turn bad) you are ready and have all the answers
• Communicates double as much as he think he should
  As a manager you are likely not working on your own tasks anymore (if you do, lucky you!) so a lot of the work is in stakeholder management. It does not matter what you think, it is likely that you don’t communicate enough. There is so much noise out there in regards to emails, collaboration tools, text messages, etc. that your job is to communicate with the target of being understood. You will have to tailor your approach to each individual. If you have to refer back to a memo from a while back when being questioned about a decision is bad news for you. You want to make sure your stakeholders are well informed and never surprised. That means over-communicate at all times and check whether you have been understood. Sometimes you will have to try several times before you are understood. If anything changes on the project, communicate this and the reason for it as early as possible even when its bad news. Bad news only get worse as time goes by.

• Respects the basics and makes sure his team adheres to them
  The manager feels responsible that his whole team adheres to the basics of professional behaviour and provides feedback when this is not the case:
  – Meetings run efficiently, have agendas and notes are sent around in timely fashion
  – Actions are defined by “Who does What by When”
  – All major work products are documented at sufficient level
  – Commitments are being upheld and when things change it gets communicated early
  – Everyone in the team treats other people with respect and behaves professionally
  – Team members adhere to corporate policies
• Knows his team and himself
  Besides delivery of the business outcome, a manager helps everyone to improve. This means he understands the ambitions of each person in the team and coaches them to achieve these ambitions. He has a plan for everyone in the team and a succession plan for when its time for someone to leave the team for bigger and better things. A good manager plays to the strength of the team members and does not focus on the weaknesses. He staffs his project by getting a diverse group of people with different mindset and working preference to avoid group think. He encourages productive arguments and discussions and steps in when they become unproductive. He also knows his own limitations and is able to ask for help from his team. Last but not least he always knows who is doing what and is aware of the status in case someone asks.
• Escalates early and transparently
  A manager is careful when reporting status and risks, which means if in doubt she adds a risk or changes the status to something different than green. While many supervisors don’t like additional risks or non-green status, the ethical correct and best thing to do for the company is to provide the status to the best of the knowledge of the manager. Any doubts or concerns should not just be in the managers head (she might win the lottery tomorrow and disappear to the Bahamas) but rather be documented and represented in the status report. Someone once told me that a project status is amber until you have the first working version, as any green before that is absent of any objective validation. I like that, but perhaps wouldn’t go as far with the reporting. A good manager also never hides anything. While it is not strictly lying, omitting something relevant is pretty close to it, so similar to the earlier point you should rather overcommunicate so that there a no surprises, ever!

I know that there is probably a lot more to be said, but the above are for me critical things I want to see from a manager above and beyond and project or technology specific things. After all each project is different but the above characteristics create successful teams and winning cultures. Something we all aim for.

The good news first: our little one is still alive 😉  All the abstract talk about being a full-time dad has now become my reality. We do not have family around us here as our families live in Sri Lanka and Germany respectively, so there is not a lot of help available. I have been pretty hands-on before as well, getting the little man to sleep, making him food and feeding him and of course changing lots of, let’s say smelly, diapers. But that has been nothing in comparison with being a full time dad.

One could say I swapped getting screamed at for not fulfilling unintelligible demands by my supervisors with having the same experience just from a much younger little man. I have to say this little man is much cuter than the alternative. I do think the little one is missing his mama during the day and first had to get used to me being around all the time, but by now we are making an okay team.

We are definitely rocking the ball slides and the construction toys – two men hard at work. And yes i have already bought two new toys for us to explore this week. Might be an expensive few months if I keep finding cool new things to explore with him. This week still held a few work related activities that I had to finish off for me – after all it’s salary update time and the people who report to me deserve to hear the news from myself. Although in this case with a very excited little man making himself heard in the background. From next week, I hope to go cold turkey on work emails and we will see how it goes…perhaps just the odd glimpse when the little one is sleeping…we will see how strong the addiction to email is.

This week I also had the chance to show the little one our new office and he had a great time meeting many of my colleagues and looking around the modern office. Of course his daily routine got disrupted by our little adventure. So far I am enjoying the experience and the world at work is not coming to an end. It was a slow letting go of work as I spoke to someone at work every day. Next week I will hear less from work I think. Let’s see. The weather is certainly incentivising me to spend more time outside in parks and cafes, that’s a performance objective I am happy to work on…

It’s over again. The annual DevOps family reunion in San Francisco. This year was extra special because I feel I learnt even more than in previous years and because I was able to hand out preview copies of my upcoming book. I am really looking forward to hear from those of you who got a copy what you think. Reach out and let me know. I will talk a little bit about the book in an upcoming blog post.

The summit had a lot more variety of topics than previous years (at least the talks I attended), which I found very refreshing. Technology talks, culture talks, security, case studies, agile – so many different perspectives on IT transformation. Congratulations to the program committee for getting such a good mix and balance.

So let’s recap what I have learned from the summit.

Keybank presentation

This was clearly one of my highlights. Hearing from a bank which is using microservices and the Netflix OSS to stabilise environments was great. To then hear they were able to outperform expectations and delivery faster and with increased scope, just shows what is possible when you get this right. Well done team and looking forward to hear more in the upcoming years. They mentioned one really interesting ideas that I will take away from the conference: “It is not a reason not to automate something if you don’t do it frequently. In fact you should automate in those cases as you don’t get a lot of practice at it.” I will remember this.

John Allspaw

Of course the expectations on my side were high when I saw John will speak. And boy did he deliver. A fascinating talk about how we cannot see what we do directly but rather work with models in our head and manipulate it via a keyhole – the screen – to interact with the invisible system. This makes you think. It requires us to move from incidents as motivators for policy – towards incidents as messages of the invisible system to us that we should use to update the mental model. Incidents show where the models are misaligned. This is tricky to operationalise and speaks to us as individuals.

We should then look at incidents as unplanned investments where the cost is already fixed for us – so how do we maximise the ROI on it? Commonly Post Mortems value the actions items at the end, but more important is the updated mental model we should have at the end. Questions to ask beyond “What went wrong?” and “How did it break?” We should talk about what made it not nearly as bad is could have been. And how can we continue to learn about the invisible systems. This talk created a lot of conversation over drinks. Mind Blown!

Scott Prugh and Erica Morrison from CSG

The continuation of the CSG story which is familiar to many of us. Good to hear they continue to challenge the status quo and push forward. The metric of the conference was “how much sleep do I get when we make changes” which moved from very little to a lot. It also showed the need to shift from a more dev focused devops to a more ops focused or balanced view and what it does to the incidents in production. And of course we might never forget Scott with a sledgehammer destroying mode 1 once and for all…

Columbia’s Scott Nasello

A story of just getting on with it and doing the right thing to improve the situation. Not a transformation with funding etc. A good reminder of what can be done if you really want to do something. The stories around Configuration management as foundation to everything – from emailing scripts to proper SCM – sound very close to some of the things I have experienced. And then the innovative approach of swapping people out regularly to create that constant beginner mindset that allows you to question things and to learn new things. Really interesting approach.

And of course the coolest random fact: It takes only 29 dominoes to take down the empire state building. Yes really!

Damon Edwards

I liked this presentation just like all the other ones that Damon has done. A good reminder that Ops is more than deployments and pipelines. I liked the insight that ticket driven queues are a sign for silos in your org. And that tickets should be for exceptions not for actual work. He went on to define Ops as a service – definition of automated procedure, execution of it and governance – which is a framework I will surely use in the future. Thanks Damon.

Amazon

This was a good industry story of the need for immediacy and how this will continue to increase. They had to learn how to integrate across multiple teams and how you need have teams that look after the end to end business service. I also learned a new word: “hyperconvenience”

Disney – Jason Cox

Phew – Finally I could attend a full talk from Jason – yay! Of course the videos were awesome and a Star Wars trailer always gets my full attention. I could very much relate to the analogy of “corporate” services perceived as the empire. Even though all you want to do is help the team and how they had to overcome this perception. I also really liked the technology rotation program for managers to continually challenge the status quo and build empathy across the business. And of course I wish I could call my training program “Jedi Engineering Training Academy” – best name ever for a training program 😉

Pivotal – Cornelia Davis

Really good semi technical talk about cloud native applications – I will definitely will buy her book. She spoke about:

• Dynamic load balancing
• Statelessness in the architecture
• Application lifecycle – events have rippling effects – you need to ping and deal with it automatically
• Versioned services and deployment in parallel not replacement of services
• Dynamically updated router for service discovery or a dedicated server to manage it
• Data APIs and caching is important to decouple from database
• Or a database per microservice and event driven data propagation, commonly using kafka as unified log and universal source of truth

Nicole Forsgren

The queen of DevOps data did not disappoint. Nicole went through 4 years of learnings. Most importantly how throughput and stability move together and are not a tradeoff.

That you should use MTTR, lead time to change and deployment frequency as good measure to understand improvements. And that when you improve DevOps performances it is likely to improve organisational performance. Nicole also shares my scepticism about maturity models which are aging too fast due to changes in capabilities. I think they can still be useful in the right hands, but one has to be careful. In a room full of techies she challenged us with “Tech plus”: It takes  IT combined with other things to make companies successful.

Her litmus test for DevOps success: “can you deploy on demand, independently and during business hours?” And if you don’t know where to start, take her advice and look at  Architecture, Continuous Integration and a lightweight change approval process as good starting points

Unfortunately I could not attend the third day of the conference, but I will surely catchup on the videos later. I will certainly be back next year and look forward to hear what everyone else learned this year.

Thanks Gene, Thanks organising team, thanks DevOps family – looking forward to see all you brothers, sisters and cousins at the next family gathering with Papa Gene 😉

You might have wondered why there have been so few updates recently on my blog. The answer is twofold a) my creative juices have gone into finishing off my book (DevOps for the Modern Enterprise) and b) earlier this year my son was born, which is the best possible reason to spend less time in front of the PC. As things are settling down I will start to write more frequently again, which brings me to today’s post: I have decided to write some blogs about my upcoming paternity leave, so you will see some less technical posts in between technical posts.

The reason I decided to do this is to encourage more fathers to take paternity leave and get them some honest first hand descriptions on how it plays out. I have learned over the last few months that many fathers have not taken paternity leave for many different reasons: career, company policies, being unsure what to expect as full time dad. So I decided to write about my experience.

I am about two weeks away from taking around three months off as full time dad. My son is 9 months old and he is a handful. I have heaps of respect for the work that my wife is doing to keep him on a schedule and look after him. The long nights of getting him back to sleep and looking after him during the days sound tough…soon I will know first hand how it is, something that so far has been limited to weekends when I don’t have to work.

You might wonder whether I am worried about what will happen at work when I am gone for such a long time and the truthful answer is “a little”. Of course there is that little voice in my head telling me that lots of important things will happen while I am gone and that I should really be there for it. But then I think of my incredible team that will cover for me when I am off and I know they will do a fantastic job. Over the last few months they already picked up a lot of the scope of work that I would usually deal with and have done great. My bosses are extremely supportive of me taking my paternity time and understand that things will be a little bit different for a period of time.

Until recently I thought that work (and potential career impacts) are the only things to worry about, but then I started to hear from mums and dads on how tough it can be to be the full time parent, the days where you only speak “baby language” and miss a meaningful conversation. The days when all hell breaks loose and you struggle to keep the baby fed and clean. The days when finding the time for a shower and a warm meal is a real challenge. The days when you had hardly any sleep and the little one wants his normal routine while you can hardly keep your eyes open. Phew…

Perhaps this gig is going to be tougher than I thought…but then I look at the little man and see him smile and I know I will enjoy the time no matter how hard it might be. Work will go on and I am sure I will catch-up when I am back. My team will do great and I will see my little man grow over the last 3 months of his first year in this world. I will keep you all posted how my paternity leave is playing out with regular blog posts.

There it was again the annual gathering of Agilists in Australia – This year in Sydney. The Accenture team turned up in force and we put together a nice little booth as well. Our Planning poker cards were popular with the audience (and of course with our own team too). The booth and the client drinks on the first day gave us the opportunity to talk to people who are adopting Agile in their organisations, many new faces but also many familiar faces who we have been working with for many years. It’s always good to catch-up and get the latest update on someone’s Agile journey. A lot of work goes into organising a conference. A thank you to SlatteryIT for getting a great conference produced each year. Our team put a lot of effort into our part of it, the booth, the presentation and manning the booth. Thanks team!

Of course there were many interesting sessions and choosing the best ones for each time slot proved as difficult as ever – and truth be told, not every session was a winner for me. I will focus on a few really good takeaways from the conference sessions. Of course the most ‘juicy’ information is always exchanged on the ‘hallway track’ – if you go make sure you spend time outside the session rooms talking to people.

Dom Price from Atlassian – He spoke about creating an organisation that enables knowledge workers to do their best work. It was great hearing from someone else about the problem with using factory or manufacturing principles in IT work. During the session I was waiting for the chance to take a photo of his team health framework and then he dropped that it is all freely available online under https://www.atlassian.com/team-playbook Go check it out!

Joshua Arnold on Cost of delay – in the deep dive we talked about uncertainty profiles and what it does for the cost of delay calculation. I found that a very interesting concept and jumped online to learn more. This blog post stood out for me if you want to learn more:  http://xprocess.blogspot.hk/2016/04/cost-of-delay-profiles.html

Barry O’Reilly spoke about the Lean Enterprise – Overall a great and entertaining talk. The one thing that stood out to me was the “delivery gap” which just shows how bad companies are in evaluating themselves – and for that matter how bad people are evaluating themselves ( remember Dunning-Kruger effect).

Sami Honkonen on Responsive Organisations – He had some great examples from his podcast in the talk on why incentives don’t work (the make you focus on the incentive not the work at hand) and why the military is not command and control anymore (something I wrote about here).

Jez Humble – I spoke to Jez after the Deep Dive because he mentioned something I absolutely agree with. Universities are teaching outdated management models. I am very passionate about using the wrong, legacy mental models, something I am speaking about at LAST conference in June 2017 and am writing a book about.

Near religious wars have been fought over which IT product to choose for a project or business function. Should you use SalesForce, SAP or IBM? I am not a product person, but I have learned over time that just looking at the functionality is not sufficient anymore. It is very unlikely that an organisation will use the product As-Is and the application architecture the product is part of will continue to evolve. The concept of an end-state-architecture is just not valid anymore. Each component needs to be evaluated on the basis of how easy it is to evolve and replace. Which is why architecture and engineering play a much larger role than in the past. This puts a very different view on product choice. Of course the choice is always contextual and for each company and each area of business the decision might be different. What I can do though is to provide a Technology Decision Framework that helps you to think more broadly about technology choices. I wrote about DevOps tooling a while ago and you will see similar thinking in this post.

My TDF (Technology Decision framework) is based on three dimensions for you to evaluate:

1. Functionality
2. Architecture Maturity
3. Engineering Capabilities

Functionality
As I mentioned in the introduction, very often the functionality provided by the software package has been the key decision factor. The closer the functionality aligns with the process that you want to support, the better a choice it would be. For you to determine whether a software package is suitable or whether you should rather build a custom system (which hopefully leverages open source libraries and modules to not start from scratch) requires you to take a good hard look at your organization. Two factors will be important in this decision: your flexibility in the process you are trying to support and your engineering capabilities. If you are not very flexible with the process you are trying to support and you have a bespoke process, then leveraging a software product will likely require a lot of customisations which are very expensive. If you don’t have a strong engineering capability either in-house or through one of your strategic partners, then perhaps leveraging a software package is the better choice. You need to understand where you stand on the continuum from flexible process, low engineering capability (= package) to a bespoke process and high engineering capability (= custom solution).

If you land on the side of a software package then create an inventory of the required functionality either as requirements or user stories and evaluate the candidate packages. Ideally you want real business users to be involved in this. The idea is that a package is giving you a lot right out of the box and it shouldn’t be too much hassle to get a demo installed in your environment for this purpose. If it is a hassle than that’s a warning sign for you.

Architecture maturity
Architecture maturity is important to support your application. The better your IT capability is the more you can build these capabilities yourself and hence you can rely on custom applications, otherwise the product needs to provide it out of the box. Four aspects that you can start the assessment with are:

• Autoscaling
  When your application becomes successful and is being used more then you need to scale the functions that are under stress. The architecture should support the flexible scaling of different parts of the applications. It should do this intelligently (e.g. not just scale the whole application, but rather the functions that require additional scale)
• Self-Healing
  When something goes wrong, the application should be able to identify this and run countermeasures. This might mean the traditional restarting of servers/applications or spinning up a new version of the application/server.
• Monitoring
  You want to understand what is going on with your application. Which elements are being used, which parts are creating value for your business? To do this the application should allow you to monitor as many aspects as possible and make that data available externally for your monitoring solution.
• Capability for change
  You want to understand what it takes to make customisations. How modular is the architecture for you to make changes to. If there are a lot of common components this will hinder you from making independent changes and will likely increase your batch size due to dependencies on those common modules.

Engineering Capabilities
Engineering Capabilities increase in importance the more you believe that the application will have to evolve in the future, which in turn is often driven by the strategic importance of the application for your customer interactions. Good Engineering capabilities allow you to quickly change things and to scale up delivery to support increasing volumes of change. The better skilled your IT department is the more it will be able to leverage these capabilities – if you don’t have strong IT capabilities then you will focus more on the in-build architecture features. Here are a few things to look out for

• All Code and configuration should be extractable
  • You want to be able to use enterprise wide configuration management to manage dependencies between systems, to do that the exact configuration of an application must be extractable and quickly be able to be restored. Inbuilt or proprietary solutions don’t usually allow you to integrate with other applications, hence breaking the ability to have a defined state across your enterprise systems
  • You should be able to recreate the application in its exact state from the external source control system in case it is required, this means no configuration should be exclusive to the COTS product
  • The ease with which the extract and import can be done will give you an indication of how well this can be integrated into your delivery lifecycle.
  • The extracts should be text based so that SCM systems can compare different versions, analyse differences and support merge activities as required
• The application is built with automation in mind and provides hooks (e.g. APIs) to fully automate the lifecycle.
  • This includes code quality checks, unit testing, compilation and packaging. None of these activities should have to rely on using a graphical user interface.
  • The same is true for the deployment and configuration of the application in the target environments; there should be no need for a person to log into the environment for deployment and configuration purposes.
  • Build and Deployment times are short (e.g. definitely less than hours, ideally less than minutes)
• The application is modular
  • This reduces the build and deployment times
  • It also allows for smaller scale production deployments and overall smaller batch sizes by reducing the transaction cost of changes
  • It minimises the chance of concurrent development and developers having to work on the same code
  • This in turn reduces the risk of complicated merge activities
• The application is cloud ready
  • First of all its not monolithic so that required components can be scaled up and down as required, not the whole application
  • Licensing is flexible and supports cloud use cases
  • Mechanisms are built into the system so that application monitoring is possible at a granular level

I hope this differentiated look at things will help you make the right choice, a choice that you won’t regret down the line. Next time you have to make a choice use this framework to help you decide.

If you work in the DevOps space long enough you would have been offered many “Silver Bullets” over the years. Everything from a specific tool, over just putting the Dev and Ops team together to specific practices like Continuous Delivery. Of course the truth is, there is no “Silver Bullet”. Last year at the DevOps Enterprise Summit I sat together with some of the brightest in this space. We spoke about what the potential “Silver Bullet” could be. It was surprising how quickly we all agreed what the one thing is that predicts success for DevOps in an organisation. So let me reveal the “Silver Bullet” we casted in that room and it is unfortunately a “Silver Bullet” that requires a lot of discipline and work: Continuous Improvement.

When we surveyed the room to see what the one characteristic is of companies that end up being successful with their DevOps transformation, we all agree that the ability to drive a mature continuous improvement program is the best indicator of success.

But what does this mean?

For starters it means that these companies know what they are optimizing for. There is plenty anecdotal evidence that optimizing for speed will improve quality and reduce cost in the long run (as both of those influence speed negatively otherwise). On the flipside trying to optimize for cost does not improve quality or speed. Neither does a focus on quality, which often introduces additional expensive and time consuming steps. If you want to be fast you cannot afford rework and will remove any unnecessary steps and automate where possible. Hence speed is the prime directive for your continuous improvement program.

In my view it’s all about visibility and transparency. To improve we need to know where we should start improving. After all systems thinking and theory of constraints have taught us that we only improve when we improve the bottleneck and nowhere else. Hence the first activity should be a value stream mapping exercise where representation from across the IT and business come together to visualize the overall process of IT delivery (and run).

I like to use the 7 wastes of software engineering (https://www.scrumalliance.org/community/articles/2013/september/how-to-manage-the-7-wastes%E2%80%9D-of-agile-software-deve) when doing value stream mapping to highlight areas of interest. And this map with the “hotspots” creates a target rich environment of bottlenecks you can improve.

The key for improvement is to use what basically comes down to the scientific method: Make a prediction of what will improve when you make a specific change, make the change, measure the change to see whether you have in fact improved it. Too often people don’t have the right rigor in the continuous improvement program. As a result, changes wont get implemented or if they get implemented no one can say whether it was successful or not. Avoid this by having the right rigor up front.

The other challenge with continuous improvement is that it unfortunately is not linear. It follows a slightly adjusted J-Curve. Initially you will find easy wins and things will look splendidly. But then things get harder and often regress a litte. You could jump ship here, but that would be a bad choice. If you stick to it you will see real improvements above and beyond easy wins later.

As a result the goal if continuous improvement needs to be to find many smaller J-Curves rather than working on one huge Transformation-type J-Curve. When I saw this concept explained at DevOps Enterprise Summit by Damon Edwards, it was quite a revelation. Such a simple graphical representation that explains so well why many DevOps adoptions struggle. It is too easy to assume a straight line and then get distracted when the initial downturn occurs after the easy wins.

I can tell you from experience that the curve is real and that no matter how much experience I gather, there are always context specific challenges that mean we have that dip that requires us to stay strong and stick with it. You need grit to be successful in the DevOps world.

So the Silver Bullet I promised you does exist, but unfortunately it requires discipline and hard work. I was encouraged to see some of the most experienced enterprise coaches and change agents agree on the need for a rigorous continuous improvement culture.

Look around and see whether you are in a place with such a culture and if not, what can you do tomorrow to start changing the culture for the better?