A Debugging Guide

"Debugging is twice as hard as writing a program in the first place."

- Brian Kernighan, The Elements of Programming Style

Introduction

More than programming, debugging requires the mindset of a problem solver. Code can be written sloppily and in an ad-hoc way; debugging must be done methodically and with attention to detail. This guide is about the concrete steps of debugging, but I highly recommend first reading Ryan Chadwick's problem solving tutorial for an introduction to the appropriate attitude and mindset. This guide is heavily inspired by a similar guide by John Regehr.

Terminology

To make things clearer, I will use the following terms for the rest of this guide:

It is important to remember here symptoms are not the same as bugs. Your program may be buggy, but could run fine without symptoms on the input you are using. Sometimes, different symptoms under different inputs may ultimately be caused by the same bug. Other times, a symptom may be due to multiple bugs in your code, all of which must be fixed before the symptom goes away. On occasion, the symptom may lead to the realization that your current approach is completely infeasible, forcing you to rewrite your code from scratch; in that case, there is no single "bug" that's causing your program to fail.

Debugging is process where, given one or more symptoms, you determine the bug(s) that led to it, and (ideally) removing those bugs from the code. In this sense, debugging is like being a detective trying to figure out what happened from evidence left behind, or like a scientist trying to understand some puzzling phenomenon. Lucky for us as programmers, we have the ability to replay what happened by running the program again, and to ask the computer to tell us more about what is happening by adding diagnostic code.

The Mental Game of Debugging

"Would you tell me, please, which way I ought to go from here?"

"That depends a good deal on where you want to get to," said the Cat.

"I don't much care where -" said Alice.

"Then it doesn't matter which way you go," said the Cat.

- Lewis Carroll, Alice in Wonderland

This guide is written for what's known as "print debugging" - that is, I'm assuming that the only thing you have is the source code and the ability to add print statements. Many languages and IDEs have debuggers, which will do the printing for you or offer other ways of understanding what your program is doing. These are highly useful, and I recommend learning to use them in your programming language/environment of choice. Debuggers may not always be available, however, and even with debuggers, the thinking process of debugging is the same, which is why this guide uses print debugging.

A lot of programmers, when they first start writing code, engage in what I call "trial-and-error debugging". That is, when they see that their program doesn't work, they randomly change their code, then run it to see if that fixed the problem. This habit is counter-productive, and the sooner you get rid of it the better. While this strategy might get you through your first (or even second) programming class, it will not work as your program gets more complicated. There are simply too many possible things to change, and without knowing what you're doing, proceeding with guess-and-check will never fix your program - or worse, introduce additional bugs.

Instead, debugging requires having a clear understanding of what your code should be doing and what your code is actually doing. After all, what is a bug if not a mismatch between expectation and reality? As per the Alice in Wonderland quotation, if you don't know what your code is supposed to do, then it doesn't matter what changes you make. The mental game of the debugging process starts here, and the rest of this guide is about the mechanics of figuring out where this mismatch occurred.

A Debugging Guide

Now that you've discovered a symptom of a bug in your program, what do you do next? Here's a brief guide:

  1. Verify that the symptom exists, and determine the correct behavior. As I said, that second part is probably the most important step of debugging: you can't debug your code if you don't know what your code is supposed to do. And I don't mean what your code is supposed to do overall; I mean what your code is supposed to do at every single line, why that line is necessary, and what value each variable is supposed to have at that point. If you can't answer that question, this is the time to pause and review the step-by-step outline for your code - you do have a step-by-step outline, right?

  2. (Optional) Create a minimal test case. When you first discover a bug, the inputs required to trigger the symptom are often complicated, which makes the debugging process itself complicated. The goal of this step is to see if you can simplify the input so it's easier to understand what is going on and, ideally, create the smallest possible input that would still trigger the bug. How to create a minimal test case is highly dependent on the program, and may only be possible after some of the later steps, which is why this step is optional.

  3. Add some diagnostic code immediately before the symptom. Since we are using print debugging, this means printing out relevant variables so that you can peek inside your program as it runs. You can think of this as trying to create a new "symptom" - you know what the diagnostic code should print, and you can now check that it is (or isn't) printing that. What variables to print depends on your program; you want to balance having useful information with not being overwhelmed with data. You can always add more diagnostics, however, so start with what you think might be wrong and go from there.

    Note that this step may involve more than just adding a single print() statement. If your code is dealing with nested lists and classes that contain each other, you might have to write loops or even recursive functions to print them all out. One piece of advice here is try to minimize how much of this support code is in your main program. Instead of writing your loops directly above where the symptoms show up, put that loop in a function, and call that function before your symptom. This helps keep your actual code clean, and will also let you reuse that diagnostic function in other places.

  4. Make sure your diagnostic code is correct and that its output is easy to understand. Never forget that your diagnostic code is still code, and it can have bugs of its own. You will never fix your bug(s) if your diagnostics is feeding you bad information. Similarly, if your program now crashes in your diagnostic code, you're not learning anything about where your code might be incorrect.

    Slightly related to making sure your diagnostic output is correct is making sure your diagnostic output is understandable. Don't just print out a variable; print out the variable and some text that tells you what variable it is and where it's printing from. There is nothing more draining than having to count the lines to figure out which value corresponds to which variable.

  5. Identify which variable(s) are incorrect, and what its correct value(s) should be. If you don't know which variables are incorrect, see the note in Step 1.

  6. Add the diagnostic code earlier in your program execution when you are sure your code is correct. Double-check that the diagnostic output is indeed what it should be. This step is not strictly necessary, but it's good practice regardless, for reasons you will see in the next step.

  7. Keep adding diagnostic code until you have identified where the bug is. If the diagnostic output is correct for some part of your program, add diagnostic code to a later line. If the diagnostic output is incorrect for some part of your program, add diagnostic code to an earlier line. The goal here is to isolate a single line of code where your bug might be, before which your diagnostic code shows no symptoms, after which the diagnostic output is incorrect. If this line of code calls a function, you may have to continue debugging inside that function.

    To beat on a dead horse, doing this step effectively requires you to know exactly what your program should do. For example, let's say your diagnistic code showed that a variable is zero. Is it correctly zero, so you need to add a special case to handle that? Or is it incorrectly zero, so you need to track down where it's being calculated? Just knowing what value each variable has is insufficient; you must also be able to identify which of those values are correct and incorrect, in order to trace down the bug.

    It is important to remember here that you might have to chase through multiple layers of apparent symptoms and bugs. For example, the original symptom might be caused by a variable being zero, but that variable is being set to the return value of a (correct) function, so now you have to chase down where the arguments of that function is set. Also keep in mind that a symptom may be hiding multiple bugs, as might be the case if two of the arguments to that function are wrong. In that case, you will have to focus on one symptom/bug first before the other, but the process of debugging is the same.

    Assuming there is only one bug, there are two main ways this step can go wrong:

  8. Fix your code. This is the step that most people imagine when they think about debugging, but as you can see, this step is only possible after a lot of thought and work. If you got to this step, then you have identified the line where the bug lives. Even then, however, fixing the bug may not be trivial. In the best case scenario - which most bugs fall into - you made a typo or forgot to do some basic operation, in which case you can fix that and be on your way. In the worst case scenario, the bug has exposed a gaping hole in how your program works. Maybe there's an edge case you didn't think of; maybe the input that triggers the bug violates some fundamental assumption you had. It's entirely possible that your program is unfixable, and you will have to start from scratch without that assumption.

  9. (Optional) Create a test for this bug. In the ideal case, you already have a suite of tests to automatically check that your code is correct. Now that you've fixed the bug, you want to make sure that it never shows up again. If you created a minimal test case in Step 2, then add that test case into your test suite. If you don't have a minimal test case, you can create one now (much more easily, now you know what the bug is), then add that to your suite.

Conclusion

To summarize: debugging is about finding the mismatch between what your code is doing and what you expect your code to do. You can't debug without knowing both. Figuring out former is "easy": you add diagnostic code. It's not knowing the latter where most people get stuck with debugging.

Good luck, and happy bug hunting!

Other Resources