Chrome Bisect

Chrome Bisect is a packaging of the bisect-builds.py tool. It's goal is to make it easy for IT admins to determine when Chrome browser began behaving differently and why it did so (deprecation, regression bug, etc).

Quick Start

Download the latest release for your OS, extract it and run chrome_bisect.

Running Chrome Bisect

You need to know a few things before you run Chrome Bisect effectively:

• Exact, minimal steps to reproduce (recreate) the issue you are having with Chrome. This would look something like:
  • Visit http://www.example.com
  • Log in.
  • Browse to Products > Widgets page
  • Attempt to view a widget
  • Issue occurs when widget does not show.
• The last version of Chrome browser where you DID NOT SEE this issue. This can be a major Chrome release number like M65.
• The first version of Chrome browser where you DID SEE the issue. This can be a major Chrome release number like M66.

Once you know these details, go ahead and run Chrome Bisect. Use chrome-bisect --help to understand all the proper arguments. Here's a sample command:

chrome_bisect --good M85 --bad M90 -- https://www.google.com/

• the following defaults will be used if not manually specified:

    --use-local-cache --verify-range --good <latest stable version minus 6> --bad <latest canary version>

• The tool will download numerous Chromium browser builds for you to test with. Each version was built chronologically sometime between the two milestone versions you specified for good and bad. This allows the tool to narrow down when the change that broke you occurred.
• Each time Chromium browser runs on your machine, try to reproduce the issue. If you cannot reproduce the issue, this version of Chromium is considered "good". If you can reproduce the issue, this version of Chromium is considered "bad".
• Once you exit a Chromium browser, the tool will prompt you to mark the build as good or bad. Be sure to enter accurate information or the results will be wrong.
• You can also retry a given Chromium build or quit completely if you think there are other issues.
• Depending on the range of Chromium builds you specified as known good and known bad, you'll need to keep on testing additional builds until the range is narrowed down as tightly as possible. This will mean running a dozen or more builds of Chromium which is why you want to get the reproduction steps as short as possible.
• Once the tool has a precise good vs bad build range, it will let you know and provide you with a URL that shows you all of the changes to Chromium browser between the two builds. With this information, you may be able to narrow down why the change was made that caused your issue.
  • Please be aware that not every issue you face is a regression bug in Chrome. For security and stability reasons, the browser regularly removes deprecated features / functionality. It may be on you to upgrade your site or environment to meet the new browser requirements.
  • If you still believe the issue is a regression bug in Chrome browser, please file an issue at crbug.com and/or open a Enterprise support case with full details including the log generated by Chrome Bisect.

An example Chrome Bisect

You own the site batstat.appspot.com, a website that shows users their current device battery status. After Chrome browser version 103 was released some users started reporting that battery status failed with a Chrome API error and you're not sure why it's failing. Let's run a bisect to understand exactly when between Chrome 102 and 103 it started failing and what the reason might be:

1. Run Chrome Bisect:

chrome_bisect --good M102 --bad M103 -- http://batstat.appspot.com/

2. The tool will start downloading Chromium builds. If the Chromium build launches and a green "GOOD" page loads with battery information, exit Chromium and mark the version as "g" GOOD. If a red page loads with BAD and complains of an API error, exit Chromium and mark the version as "b" BAD.
3. The tool will need to download and run about a dozen builds to narrow the build range as tightly as possible. Keep marking builds as good or bad based on the page it shows until the tool finishes. Be careful here, marking one build wrong will cause the whole bisect to fail and show invalid results.
4. Once the tool is finished with builds, it will tell you the exact builds between good and bad behavior. It also gives you a link to the changes that were made to Chromium browser between the last good and first bad build.
5. Open the changelog page and look through the list. (hint: here's the changelog you should get.)
   • Can you tell which exact change in the list caused the site to stop loading? (hint, search the list of changes for "battery")
   • Why was that change made?
   • What might you need to do to fix your site? (hint, notice how HTTPS://batstat.appspot.com works in current Chrome versions but HTTP://batstat.appspot.com only works in Chrome 102 and earlier)
6. For reference, here's the full output of an example run:

Expand for full command output

$ ./chrome_bisect --good M102 --bad M103 -- http://batstat.appspot.com/ running bisect-builds.py with options: --good 992738 --bad M103 --verify-range --use-local-cache --archive linux64 -- http://batstat.appspot.com/ Scanning from 992738 to 1002911 (10173 revisions). Downloading list of known revisions... Loaded revisions 41523-1118151 from /home/jayhlee/cb/chrome_bisect/.bisect-builds-cache.json Downloading revision 992740... Received 154342263 of 154342263 bytes, 100.00% Trying revision 992740... Revision 992740 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 1002910... Trying revision 1002910... Revision 1002910 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b Downloading revision 998107... Bisecting range [992740 (good), 1002910 (bad)], roughly 11 steps left. Trying revision 998107... Revision 998107 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b Downloading revision 995067... Received 154537868 of 154537868 bytes, 100.00% Bisecting range [992740 (good), 998107 (bad)], roughly 10 steps left. Trying revision 995067... Revision 995067 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 996538... Received 154807698 of 154807698 bytes, 100.00% Bisecting range [995067 (good), 998107 (bad)], roughly 9 steps left. Trying revision 996538... Revision 996538 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997440... Received 154925953 of 154925953 bytes, 100.00% Bisecting range [996538 (good), 998107 (bad)], roughly 8 steps left. Trying revision 997440... Revision 997440 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997790... Received 155344479 of 155344479 bytes, 100.00% Bisecting range [997440 (good), 998107 (bad)], roughly 7 steps left. Trying revision 997790... Revision 997790 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b Downloading revision 997598... Received 154954643 of 154954643 bytes, 100.00% Bisecting range [997440 (good), 997790 (bad)], roughly 6 steps left. Trying revision 997598... Revision 997598 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997664... Received 154954126 of 154954126 bytes, 100.00% Bisecting range [997598 (good), 997790 (bad)], roughly 5 steps left. Trying revision 997664... Revision 997664 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997718... Received 154953295 of 154953295 bytes, 100.00% Bisecting range [997664 (good), 997790 (bad)], roughly 4 steps left. Trying revision 997718... Revision 997718 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997744... Received 154955563 of 154955563 bytes, 100.00% Bisecting range [997718 (good), 997790 (bad)], roughly 4 steps left. Trying revision 997744... Revision 997744 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g Downloading revision 997775... Received 155344747 of 155344747 bytes, 100.00% Bisecting range [997744 (good), 997790 (bad)], roughly 3 steps left. Trying revision 997775... Revision 997775 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b Downloading revision 997757... Received 155345317 of 155345317 bytes, 100.00% Bisecting range [997744 (good), 997775 (bad)], roughly 2 steps left. Trying revision 997757... Revision 997757 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g You are probably looking for a change made after 997757 (known good), but no later than 997775 (first known bad). CHANGELOG URL: https://chromium.googlesource.com/chromium/src/+log/03e9c2b23047e8bc7cdbbd5843b52112817ed1d6..53d540703b72faa87436f51279043c094939418f

Known Limitations

As mentioned above, Chrome Bisect has a few limitations that you should be aware of:

• Chrome Bisect downloads public Chromium browser binaries to test with, these open-source public builds do not contain everything the official Chrome browser contains and may not be good for testing issues related to:
  • The PDF Viewer
  • Adobe Flash Player
  • Widevine DRM
• If you have an issue on Chrome OS try reproducing it in Chrome browser on Windows, Mac or Linux and see if you can utilize the tool there. The tool does support --archive chromeos on Linux which runs like Chrome OS but is not really full Chrome OS, your mileage may vary.