This week you will see more content than the usual once per week posting. I am training technology architecture and as such I will include a daily post as basis for discussions in the course. Post the discussion in class I will update the post with a summary of the discussion. I hope you find this an interesting concept as I am experimenting with the blog medium. On to the first topic of four: Technical Debt.

Much has been said about technical debt and how hard it is to pay it down. In the last week I had a few discussions about this and I thought I put my thoughts down on paper (or really the keyboard).

What is technical debt?
To get everyone on the same page let’s define what technical debt actually means. Technical debt is best explained by what it causes. Similar to debt, it causes interest over time. And as you all know interest is basically paying money for which you don’t really get anything in return. You are paying for an earlier decision (e.g. a purchase you made when you could not yet afford it) and if you don’t start paying down the debt then you will be able to afford less and less with the same amount of money as the interest takes over.
In IT what happens is that you set out to implement a new solution and of course you try to deliver the best solution possible. Over time decision points come up where you could implement something that costs a bit more but would provide better maintainability later on, like automated unit testing or separating out a function that you might reuse later rather than keeping it within a different function. You now need to decide whether to invest the extra time/money for this non functional aspect or to focus purely on the functionality that your business stakeholder requires. Every time you choose the short-term solution you increase your technical debt as next time you want to change something or use the functionality that you could have split out, you now require an additional effort. Of course there are many more ways to incur technical debt than just the lack of automation or modularisation, but these serve as examples.

Why is it so hard to avoid technical debt?
The crux of the matter is that by making all the right decisions (according to some criteria), you can still incur an increasing amount of technical debt. Imagine you are working on an application and you have exactly one project and you don’t know whether there will be any other projects after you. Should you still make the investment in good automation and modularisation practices? What if you know there are other projects, but you don’t know whether it will impact the same areas of the code or would use the same automation? …
You can see its a slippery slope.
Look at the graph on the left. It shows the total cost of change over time, initially it is cheaper to just implement the functionality without investing in good practices, but then over time the cost of changes increases as the technical debt makes it more costly to make changes. At some stage the total cost of change means each change is now more expensive than if you had implemented all the good practices from the beginning, but now you have to pay down all that debt and it is costly to jump back to the other cost line. You also see that even with great practices the cost of change generally increases a bit over time, although there are people arguing that great modularisation and reuse can actually reduce the total cost of change over time as you recombine existing services to create new ones, but that is for another post.

What does it take to pay it down?
The challenge with paying down technical debt is that it usually takes a lot of time and while you can accelerate it through a dedicated program, the only long-term solution is to leave the software in a better state every time you make a change. Otherwise you run a “paydown” project to reduce technical debt but then increase it with each subsequent functional project until you do the next “paydown” project. If you do it little by little you will have a much more sustainable model and the cultural shift that is required to do this will be beneficial for your organisation in any case. If your Agile implementation can help by making the technical debt more visible and by visually tracking each time you pay a bit of debt down, then you are onto a model that get you to the total cost of change curve that you aspire to. And my personal view is that you need to make sure that the PMO organisation and the project managers are clear about their responsibility in all this.  They should be evaluated not only by the on-time and on-budget delivery of functionality but also by how much they have done with their projects to pay down technical debt, otherwise PMs are too often likely to choose the short-term answer to achieve on-time, on-budget delivery at the cost of technical debt for the next project or in not so kind terms by “kicking the can a bit down the road”.

How can you measure it?
Here I am still without a really good answer…theoretically I think it is the sum of the additional cost that each change costs at the moment minus what it would have costed if you had all the good practices in place. But that is really hard to calculate. Alternatives that i have seen are to create a backlog of all the things that you should have done and to add to it every time you make a short-term decision. The size of this backlog is you technical debt in this case. Not yet great answers, but i keep looking. Please reach out if you know of a better way.

Picture: 3D Shackled Debt by Chris Potter from http://www.stockmonkeys.com
License: Creative Commons