You know what is my favorite joke about consultants?

A man walks into a pet shop and sees a monkey in a cage with a label ‘C Monkey – $2,000’. The store owner comes and the customer tells him « This monkey is expensive. He must be very special ». And the owner explains « It’s a monkey that knows how to program in C. Very good programmer, fast, he produces good quality code and bug-free. At that price, it’s a good deal ».

The customer looks at another cage with a sign ‘C++ Monkey – $3,000’ and says « This one is even more expensive. What does he know? ». « Same as the first one, but with C++, an object oriented language, more complex. He too is a very good programmer. And he also knows a bit of Java. ».

The customer then sees a third cage with a panel ‘Monkey – $5,000’. « Oh, this one is as expensive as the other two combined. He must be really good. What he can do? ».

« Well, I don’t really know » replies the owner. « But he says he is a consultant ». (1)

Why am I telling you this joke? Because I see more and more consultants who take the concept of technical debt with the intention, worthy of prize, to update or improve it but eventually turn it into a general theory, not practical and uncertain, not to say impossible to use.

I already did a post some months ago ‘Tecnical debt vs. Marketing guys’ in which I spoke about the ‘marketing guys’ who, with great inventiveness, extend the concept of technical debt to all sorts of very scary financial variants. For example, the analogy with toxic financial assets: some applications are toxic and wait the worst possible time to explode all by themselves with the intent to cause as most damage as possible.

And I see more and more Quality consultants do the same, as in this video ‘Technical Debt A Finer Edge‘ whose main points I will summarize:

• The definition of technical debt introduced by Ward Cunningham in 1992 is archaic because too focused on the code and the programmers.
• It should be updated to take into account all defects, deficiencies, errors, negligences, malpractices that occur in the production process and software project lifecycle, in all its phases (design, testing, etc.).
• It must also take into account the management errors and incorrect or insufficient process: workload estimation, planning, risk management, etc.).

This video develops the same idea that this article by Capers Jones ‘The Errors and Hazards of Technical Debt’ in which he defines technical debt as the cost of correcting errors made during the development of an application.

« The essential idea of technical debt is that mistakes and errors made during development that escape into the real world when the software is released will accumulate downstream costs to rectify. »

The disadvantages are the lack of consideration of quality defects in this definition of technical debt : « In order to evolve from a novelty into an effective metric, technical debt needs to encompass all quality costs and not just post-release code changes ».

This article concludes that technical debt is only part of the cost of quality and that this last measure (Cost Of Quality or COQ) is more appropriate when one wants to study the economic dimension of quality.

Intentional and unintentional debt

I totally agree with Capers that technical debt is only part of the cost of non-quality, but I do not agree that we should extend the concept of technical debt to include all these costs, as said in the previous video.

The concept of technical debt is understood and interpreted in very different ways and Ward himself explained in this article ‘Ward Explains Debt Metaphor’ that there has been a lot of confusion and that he never said that technical debt is caused by poor code quality.

The analogy used by Ward is that by getting a loan, you can make your project more quickly, but at some point, you will have to repay your debt. To make a first prototype is the equivalent of an investment that allow to deliver something quickly to users and therefore go faster than if you wait to have complete functional specifications and the development phase completed.

But refactoring will be required to go from prototype to final version, and thus repay your investment.

The original concept aims to promote Extreme Programming, a development methodology based on iterative cycles to complete our knowledge of functional specifications, adapt the design of the application and gain experience (Ward’s team decided to program in Smalltalk, an object language new to them).

As explained in ths article on Uncle Bob’s blog, ‘A mess is not a technical debt’, technical debt is a voluntary decision – deliver quickly a first incomplete version – to cope with project constraints. A code of poor quality is not a rational decision, and therefore is not part of the technical debt, but a loss.

I would say that in both cases, it implies a refactoring done by developers. And therefore how to identify changes made to improve the architecture of the code and those to correct a defect? A programmer decides to split a class into two classes because additional functional requirements impose to specialize them or maybe he decides to do that to reduce the class complexity and improve code readability. Technical debt or not?

Martin Fowler explains in this article ‘TechnicalDebtQuadrant‘ that the question is irrelevant. The value of the concept is that the metaphor is a powerful communication tool that can justify to stakeholders and management that you should do some refactoring, whatever the nature of the technical debt, intentional or unintentional. A good developer knows best programming practices but may omit some, deliberately waiting for the next version or just by lack of attention. Perfection is not of this world.

Different kinds of debt

But in all cases, whatever the nature or origin of the technical debt, it is clear that the programmers are the ones who manage it, in Ward’s metaphor. And this is one of the reasons I believe that this notion should not be extended to other non-quality costs.

An error or an omission in functional requirements or during the design phase will certainly lead to modification in the code, but the origin of this change is not in programming choices or failures of developers.

An incomplete or inadequate test plan can result from the management’s will to deliver as soon as possible a prototype that enables users to specify their needs, without worrying about bugs, or may be caused by lack of time, poor knowledge or experience of the QA team, incorrect documentation or inadequate equipment.

The metaphor of the debt can apply to all these examples, but if so, let’s just talk about functional debt or QA debt, rather than trying to integrate into technical debt all costs of non-quality.

Another reason why I think technical debt should apply only to programmers and code:

• Bugs that find their origin in the code are the most numerous: about one in three.
• Code defects are easier to detect and correct: about 85% and this figure can rise to 95% by using a code analysis tool.

Bugs caused by defects in requirements and design are fewer but more difficult to identify and remove.

Non-quality has its origin in very different kinds of defect with very different risks and impacts for the users. To put them all in the same basket creates confusion and makes impractical the notion of technical debt.

Technical debt and programmers

Code defects are the only ones that can be identified automatically, using a code analysis tool. You can never automate the detection of an absent or incomplete requirement, or lack of testing. These are necessarily manual tasks.

You can analyze 5 000 Java classes and 100 000 lines of code in a few minutes and quickly and easily identify all violations to best practices. You can even automatically build remediation plans that will update the developer’s list of tasks.

If we want to integrate other defects into technical debt, then we will need to perform manual risk assessment, estimate requirement completeness or measure the level of test coverture for each of these 5 000 classes. Impossible.

Programmers already understood how to use the concept of technical debt at an operational level, as shown in this post ‘Effective Code Review with Sonar’ on Sonar’s blog. A Continuous Inspection process measures the level of technical debt and ensures that it does not grow.

I therefore agree with Capers that technical debt should not be confused with the cost of non-quality (COQ) or the concept becomes impractical.

And while us consultants are discussing the exact definition of the concept, developers use technical debt as a daily operational metric that monitors the level of code quality, regardless of an exact definition.

To update or extend the concept of technical debt can be intellectually interesting but is useless without a practical and operational application or otherwise, as in the joke about consultants, we do not really know what it is, what it can do, what it is worth.

 

(1) Variant : « Well, I don’t really know » replies the owner. « But the other two call him ‘Boss’». Don’t try this one with your boss or he will believes that you are saying that nobody knows what he does.