Versions/Builds

Summary

The webapp maintains version/build information for Firefox.

digraph G { rankdir=LR; splines=lines; archivescraper [shape=rect, label="archivescraper"]; model [shape=box3d, label="crashstats_productversion"]; betaversionrule [shape=rect, label="BetaVersionRule"]; archivescraper -> model [label="produces"]; model -> betaversionrule [label="uses"]; }

Tables

The data is stored in the crashstats.ProductVersion Django model. This is the crashstats_productversion table in PostgreSQL.

Where the data comes from

Data can get in the table in two ways:

  1. the archivescraper Django command scrapes archive.mozilla.org data every hour and adds new things to the table

  2. the Django admin page lets administrators add new data and edit existing data when there are problems

What uses this data

Incoming crash reports for Firefox have a Version field. There are some cases where the value is “a lie”. In these cases, the processor’s BetaVersionRule will look up the (product, channel, build id) in the crashstats_productversion table to find the actual version.

About archive.mozilla.org

Every time a build is created, the binaries and metadata files are published at:

https://archive.mozilla.org/pub/

Rough directory structure:

pub/
  firefox/         Firefox builds
    candidates/    beta, rc, release, and esr builds
    nightly/       nightly builds

  devedition/      DevEdition (aka Firefox aurora)
    candidates/    beta builds for Firefox b1 and b2

In the candidates/ subdirectories are build directories like build1/. Each of those is a release candidate for that version and the last one is the final.

For example, here’s the direcotry for Firefox 64.0b4:

https://archive.mozilla.org/pub/firefox/candidates/64.0b4-candidates/

In it are two build directories: build1/ and build2/. The first is 64.0b4rc1 and was not released to anyone. The second is 64.0b4rc2, but since it’s the last build in that series, it was released as 64.0b4.

Here’s the build file for that for linux-i686 in en-US:

https://archive.mozilla.org/pub/firefox/candidates/64.0b4-candidates/build2/linux-i686/en-US/firefox-64.0b4.json

Note how the directory name has “64.0b4”, but the moz_app_version is set to “64.0”.

With recent builds, there’s an additional buildhub.json file:

https://archive.mozilla.org/pub/firefox/candidates/64.0b4-candidates/build1/linux-i686/en-US/buildhub.json

That includes similar information, but is built to be ingested in Buildhub.

Things to know:

  1. In a given product, different platforms can have different build ids for a version. For example, 54.0b5 has build id 20170504103226 for Windows and Mac builds and 20170504173217 for Linux builds.

  2. Firefox beta 1 and beta 2 are released in the DevEdition product in the aurora channel. That’s been happening since Firefox 55.

  3. https://archive.mozilla.org/pub/*/candidates/ is periodically purged of old data. For example, at the time of this writing, there’s a ton of stuff that’s missing between Firefox 40 and 49. The builds are still in /releases/, but that doesn’t include the JSON files with build information.

    Socorro has dumps of the old product_versions table with the older product/build information in Google Drive.