Technical Debt versus Cruft

Am I Tilting at Windmills?

I repeatedly find myself in discussions with really smart and experienced people regarding Technical Debt. And I consistently find myself ranting about how cruft and technical debt are not the same thing. I am beginning to feel I am merely tilting at windmills.

I don't know if my prior rants on the topic of Technical Debt have provided sufficient clarity. In them, I try to better define Technical Debt and point out the differences between it and cruft. But I do not provide any tangible examples. Herein, I attempt to remedy that situation. Today's installment is in narrative form. I intend to provide a later post with code examples.

Technical Debt - A brief reprise

Ward Cunningham coined the phrase Technical Debt
"Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise."
As he clarifies later in a You-Tube video, Ward was specifically talking about design decisions made in the course of developing software that allowed for more rapid delivery. Rapid delivery not for the sake of meeting a deadline. Rapid delivery to illicit quick feedback, thereby providing the data necessary to adjust the design to be more congruent with actual needs. The object design should change to reflect your current understanding of the domain, even if that understanding is partial. Technical debt is payed back through code refactoring as our understanding of the domain matures.

Ward clearly states, "The ability to pay back debt [...] depends upon you writing code that is clean enough to be able to refactor as you come to understand your problem."

Clean Code is actually a prerequisite for Technical Debt. If you don't have Clean Code, you cannot expect to pay the debt back. If you have cruft, you cannot reasonably incur Technical Debt. In fact, you are not incurring debt, you are merely adding to the existing mess.

Examples (in Narrative Form)

Technical Debt

Example One
We are building a new application to allow our customer to sell widgets and parts to distributors.
  • Widgets represent 75% of the sales.
  • 82% of the customers order on terms.
  • The cross-section of these two groups represents 69% of all sales.
In order to get the application running and generating revenue, we decide to launch with only widgets and terms. We will sell parts later. We will also add other forms of payment later.

This is a strategic decision. We are creating a design that we know will have to change. We know we will re-visit this code and perhaps even throw away some (or much) of it.

But we write it well, we test drive it, and we do not compromise quality. We will defer to the last responsible moment and make the design adjustments based on what the business determines is the next highest value activity when it is decided.
Example Two
We are working with a third party API for determination of flood plane coverage for a given property. We want to know this so we can decide if we will or will not provide an applicant a loan. We don't give loans on properties in flood planes.

We can make numerous calls to the service. One, which is simple, will tell us if the property is in a flood plane or not. This is all we need to know for an accept or reject call.

We decide to go ahead with this basic functionality and make sure the service is consistently reliable in terms of uptime, speed, and accuracy. We are going to run one server against the new service and leave the others running against the old (slow and more expensive) service.

Once we've determined the service provider is reliable, we will also code the pieces for garnering more specific data. It would be good to have this on the loan application record if there are any questions in the future, but it is not mandatory.

This is a strategic decision. We are creating a design that we know will be either eliminated or expanded.

We write all of the code well. We test drive it. After a period of learning, we will either toss the code or we will add new features.

Cruft

We are working on a system to schedule production of widgets on the factory floor.

We have a bonus initiative to get the application done by the end of the year. It is September and we're just getting started due to delays in the delivery of our prior project. According to the schedule, we should have started in May. If we miss the deadline, there will be much gnashing of teeth. If we miss the deadline, we will not get our bonus. We ask our boss if we can deliver only part of the application, but her bonus is much more significant and requires complete delivery. She denies the request, admonishes us for putting her in this situation, and reiterates the priority of this project for "the business". She let's us know that the CEO is watching this one. The CEO was watching the last project too. That's how important IT is to the corporate strategy.

So we work like crazy. We skip unit testing. We work in silos and make decisions on the fly about ambiguous requirements. We notice the FillQueue class is getting quite large and everything is dependent on it. But we don't have time to change it. It works and we are under pressure to deliver. Besides, we could no longer test these classes in isolation if we wanted to. We'd have to tease them apart at the seams and that could take weeks. It is getting more and more difficult to write the code. It is like a bowl of gelatin; you touch it down here and it wiggles up there? Why does it do that? Whatever. We need to make the deadline.

This is the wrong decision for the business. This is not a strategic initiative. This is not professional or even ethical behavior. This is a failure.

But bonuses got paid.

What about you?

I've been told delivery teams don't take on Technical Debt as a strategic initiative, rather most produce cruft and then call it Technical Debt. Help me prove otherwise. Please post a comment with an example of Technical Debt (not Cruft) taken on by a team you worked on.

4 comments:

  1. I'm not sure if I'm clear on how you're defining technical debt. From the examples above, it sounds like you're saying technical debt refers to the domain model being incompletely implemented (example 1), or when it's still being iterated (example 2). However, I think you're defining cruft as implementation decisions (as opposed to domain design decisions) that were made for expediency, with the idea that it needs to be fixed up later on.

    If that is what you're saying, then I'd disagree. For me technical debt is a very useful metaphor for discussing the cruft with the whole team, including management, so they can get their head around the importance of spending time "paying down debt".

    Although I could see calling incomplete domain design a type of technical debt, I don't think it needs that metaphor; it's pretty easy to describe to non-technical types what it means, and for them to see the value of spending time on it.

    ReplyDelete
  2. Have you seen Martin Fowler's technical debt quadrant? http://www.martinfowler.com/bliki/TechnicalDebtQuadrant.html

    You're constraining Technical Debt to the Prudent-Deliberate quarter. That's valuable to talk about, but just as there are many types of debt in the real world, there are many types of technical debt. Prudent and deliberate debt is like a home mortgage with a 20% down payment. Reckless debt is like running up a credit card. I've found a lot of value in the metaphor comparing paying off credit card debt with getting rid of cruft as you call it.

    ReplyDelete
  3. Dennis - I have seen Fowler's technical debt quadrant. I have a blog entry on this as well.

    http://www.docondev.com/2009/10/martin-fowler-on-technical-debt.html

    Cruft, as I call it, is cruft as defined:

    http://searchsoftwarequality.techtarget.com/sDefinition/0,,sid92_gci1274102,00.html

    http://www.google.com/search?hl=en&defl=en&q=define:Cruft&sa=X&ei=mdytTKKVL4OClAeZkIH6AQ&ved=0CBIQkAE



    While it is certainly comfortable to talk about having written cruft in terms of debt, the metaphor falls apart, hides a serious issue, avoids accountability, and does us a significant disservice. If I believed there was a significant amount of healthy conversation happening around the metaphor as commonly expanded, I would not be so adamant about trying to educate people on the source and intent of the metaphor.

    I know of no other profession where such a phrase is acceptable.

    "Yes sir, your walls are crooked, but we incurred structural debt in order to meet your deadline. We should discuss that debt and set a plan for paying it back later."

    "I am sorry you lost the limb, but we incurred health debt due to our inexperience in that form of surgery. You see, it is like we paid for the surgery with a credit card instead of a home equity loan..."

    "Your car does run poorly, but we incurred mechanical debt because we used less expensive oil to stay in budget. We should get metrics around that and make sure we pay the debt down in the future."

    If, prior to coding, the customer agrees it is best that you write bad code that will not have tests and will be very difficult and costly to maintain, then I agree under that specific circumstance, cruft is technical debt. Of course, I further argue that the only way to go fast is to go well, so the reason for writing cruft is invalid. But that is another blog post.

    http://www.artima.com/weblogs/viewpost.jsp?thread=51769

    ReplyDelete
  4. I concur with your point that there is no reasonable justification for deliberately being reckless when writing code. However it happens and I'm thinking from the perspective of how to get rid of it once it's there.

    So called developers who have deliberately or ignorantly written garbage should be held accountable.

    I guess the strength of the metaphor of debt for cruft might depend on your view of credit card debt. If you view credit cards as an evil device that sucks people into debt without them really considering the ramifications, then I believe that applies to cruft. When presented with a code base that's full of cruft, you need to either rewrite it (ie. get it paid off) or throw it away, which would be the equivalent of declaring bankruptcy and starting over again.

    I don't think anyone goes into major credit card debt with a deliberate well thought out plan! It happens a little bit at a time and when they finally realize they're in trouble, it's often too late. The same is true with many developers I've met. They're hacking away and getting short term results, but the code they've written is impossible to maintain.

    ReplyDelete