Hunting Bugs or: How I Learned to Stop Worrying and Love git bisect
Getting assignments to write new features on a web app is always way more exciting than looking at Pivotal Tracker and seeing a list of bugs to fix. But I have to admit that probably the most satisfying feeling as a programmer is the feeling you get when you’ve solved a complicated bug in a clean way. I’m not talking about bugs that are caused by syntax errors or accidentally using the wrong methods. I’m talking about the bugs that have seemingly innocuous symptoms, but end up taking you deep into the rabbit hole of your app.
The work here is the hunt — once you’ve actually tracked down what’s causing the problem, it’s generally very easy to fix the issue. When you’re working on a large codebase with multiple contributors for an asynchronous web 3.0 app, this can get really interesting (and frustrating). What do you do when the problem isn’t immediately apparent?
About 25% of the time, you get lucky and Google does all of the work for you — simply copy and paste the error message that the application is giving you or type in a short description of the issue and you’ll be surprised how many people have encountered the same problem and have solved it already. Stack Overflow is a goldmine of bug solutions, and so is the vast ecosystem of tech bloggers (like me) who give you step-by-step instructions on how to solve bugs.
70% of the time, it’s a bit more of a challenge. The bug is specific to the way your team has set up the application and no one has encountered a bug quite like the one you’re currently facing. Google doesn’t help at all. Here’s where bug-hunting becomes much more of an art than a science.
Typically the process is fairly quick if you’ve written the code that you’re debugging — you already know what assumptions you’ve made in developing the feature and what events are firing when, and so strange behavior can often be tracked down fairly quickly. Luckily, you’re using git (right?), so even if you didn’t write the code you’re working with, you can quickly track down who did. Simply run
git blame path/to/file in your console and git will output a line-by-line summary of who wrote each line of code.
If that doesn’t get you closer to a solution, the next step is to console.log (or its equivalent) everything. I’ve had the opportunity to see expert programmers hunt down bugs at Art.sy, and that’s exactly what they do. There’s generally no need to use fancy debugging software or set breakpoints or anything complicated like that — just output to the console when events are firing, when methods are being called, and what the value of variables are at specific points in time. Obviously, you need to make educated guesses as to what might be causing the problem, and then figure out whether you’re right by outputting values to the console.
What about the last 5%? It’s reserved for the worst kind of bug. The bug that is impossible to fix. You’ve searched all over the internet and no one seems to ever have had the same problem. The code you’ve written and poured over is perfect and definitely is not the cause of the problem. The project you’re working on has thousands of lines of asynchronous code — literally anything could be going wrong and you have no leads whatsoever. Don’t worry, there’s still hope. git bisect is still your friend.
git bisect works when you know that a feature worked at one point in your app’s history and no longer works in your most recent commit. Don’t know offhand when the feature actually worked? git blame can help here — find out when the feature that’s no longer working was merged into master, and you can hopefully safely assume that the feature was working at that point.
Once you’ve located a good point and a bad point in your code, run
git bisect start in your current branch. Then, since a git repository is basically a sorted list of previous states of your app’s codebase, git can perform a step-by-step binary search to find the first point in your app’s history where the feature stopped working. At each step, you run either
git bisect good or
git bisect bad in your console depending on whether the feature works or doesn’t work in the commit that git bisect picks. Eventually, the program will output the commit id of the commit that broke the feature. It’s an amazing feeling when you’ve been struggling with a bug for a day or two.
git bisect isn’t always that simple when you’re running it on an app that has a bunch of dependencies. Here’s how to get around any snags if you’re running git bisect on a rails app:
- Restart your rails server and run bundle install at each commit that git bisect chooses. You might also need to manually compile assets at each point if something isn’t doing that for you automatically.
- Make sure your “good” starting commit is the commit that merged the feature you’re debugging into master. Programmers don’t always commit 100% working code, but when it comes time to actually merge a pull request, you can be sure that the app ran smoothly and specs were passing.
- Sometimes the app won’t run on old commits because paths in old versions of your Gemfile aren’t valid anymore. For example, when I used git bisect on Art.sy’s codebase, a co-worker’s change to his Github username caused multiple bundle install failures. You’ll have to address these issues manually. If you can’t figure out why a gem isn’t installing, just replace the line with the corresponding line in the app’s most recent Gemfile.
Aside from those potential issues, git bisect feels like magic for those epic bugs that refuse to be solved. Once you’ve found the commit that broke the feature, you’re almost certainly 99% of the way to solving the problem.