Have you ever found a bug in your application, but been unable to find exactly *where* that bug was in your code? Without this important bit of knowledge, it’s difficult to properly shame the developer that introduced the bug fix the bug. It can be a bit like finding a needle in a haystack. In this post, I’ll show you how to significantly shrink the size of the haystack and find bugs more quickly by using ‘git bisect.’
The Woes of Late Testing
Finding out that your application has a bug is never a good feeling. It’s an even worse feeling on the eve of a release. I remember one particularly annoying JavaScript bug I found many months ago in just such a situation. It prevented *any* script in our app from executing in Internet Explorer 8. Just BAM, JavaScript error.
Not only was this an IE-only bug, but the problem was magnified by the fact that it only occurred when our scripts were compressed and minified, which made finding the exact line that was to blame very painful. Painful as in pretty much impossible. The statement it claimed to be blowing up on seemed pretty innocuous: a seemingly-simple assignment statement. Why, oh why, was this line now causing a problem?
To make matters worse, this wasn’t something that was recently introduced. My team and I were at the end of our three week sprint, on the eve of pushing the new version out the door. While we were practicing Continuous Integration (complete with a library of about 3000 unit and integration tests), we did *not* practice Continuous Deployment. We tested things locally, sure, and everything was peer reviewed, but we didn’t actually deploy the app in release mode, with our scripts minified, until the end of the sprint. This is all kinds of bad in hindsight, as this example illustrated, but I digress…
I dug through the code. I looked through every piece of JavaScript (and there was a LOT; picture metric-tons of JavaScript, and you’re still on the low-side). I combed through commits. I found nothing. After a couple of hours of pulling my hair out (no small feat for me), I gave up for the night and left with the app in a completely non-functioning state. That was a horrible feeling. If you’ve ever been in a similar situation, I’m sure you can sympathize.
The next morning, I knew what to do: I had to find the commit that introduced this problem. The app worked fine at the end of our previous release. It did not work now. I had a working version, and a broken version. This is all I needed to start my search with ‘git bisect.’
What is ‘git bisect’?
Great question! ‘git bisect’ is a powerful command that does a binary search through your revision history, zeroing in on the revision that introduced a bug. Basically, it helps you find the needle in this haystack:
(Note: illustration images are using SpecsFor’s history, not the project this story is about; that project is proprietary and all.)
Back to our story…
So as I said, I knew I had to find the commit, and I knew ‘git bisect’ could help me. Here’s what I did. First, I gave it two commits: the “good” commit, our working version from the end of our previous sprint; and the head of our release branch, the one we were trying to ship:
‘git bisect’ then began its binary search, checking out the middle revision:
After it checked the revision out, I opened up Visual Studio, built (in release mode), and tested for the bug. It was still there, *but* I had eliminated half of the commits from the search space! Not bad! Since the bug was still there, I ran ‘git bisect bad,’ and git checked out another commit for me to test:
I repeated the process again: I built the app and tested for the bug. This time, the app worked! I was getting closer. Now I ran ‘git bisect good,’ and git grabbed another commit for me:
I continued in this fashion, running ‘git bisect bad’ or ‘git bisect good’ depending on whether or not the version git checked out contained the bug or not.
Finally, after only a few minutes, I received this happy message from git:
I found the commit that introduced the bug!! Once I knew exactly where to look, spotting the bug was easy, as was fixing it.
The Moral of the Story….
First, TEST YOUR WEB APPS WITH SCRIPTS AND CSS MINIMIZED OFTEN! Don’t save that sort of testing for the end. That’s just a recipe for pain.
Second, ‘git bisect’ is awesome. If you do make the mistake of letting a bug creep in unnoticed for many weeks, and if you can’t easily find it, you can use ‘git bisect’ to find when it was introduced. That capability can be a real life (and hair) saver, plus it gives you the opportunity to assign demerits to the person that introduced the defect. 