Socorro JSON Schemas

This folder (socorros/schemas/) contains JSON Schema files describing the documents that Socorro generates. At the moment only the Processed Crash (the one generated by Socorro’s processors) is described.

These files will be used as a “contract” between Socorro and other systems to which we might send our data.

java_exception

This schema covers data in the JavaException crash annotation in incoming crash reports.

telemetry_socorro_crash.json schema

This schema covers documents being sent to Telemetry ingestion.

Modifying

The JSON Schema should contain a key called $target_version.

• This is a monotonically increasing integer

• Don’t increment the version if you’re…

  • Adding more keys at the root level.

  • Editing comments (content of description values).

• Do increment the version if you’re…

  • Adding more keys inside a nested object.

  • Changing the type definition of an existing key in any way.

  • Add or remove keys from a required sub-key. For example, if a key was required but you’ve now removed it. This is applicable at any nested level.

For example, if you want to add a new field to the root like this:

+ "addons_checksum": {
+     "type": ["string", "null"],
+     "description": "Sample specimen"
+ }

then don’t change the version.

However, if you add a key inside a nested structure, you have to bump the $target_version number by 1. For example:

@@ -286,8 +286,12 @@
     "json_dump": {
         "type": "object",
         "description": "The dump as a JSON object.",
         "properties": {
+            "for_example": {
+                "type": ["string", "null"],
+                "description": "Brand spanking new field inside json_dump"
+            },
             "crash_info": {
                 "type": "object",
                 "properties": {
                     "address": {

Don’t change the type definition. That breaks existing data. You must create a new field and deprecate the old one.

Testing schema changes

After editing the telemetry_socorro_crash.json file verify it still validates.

After any change, you should test that at least 100 randomly picked crashes from prod. To do that, from a checkout of socorro run:

$ python socorro/schemas/validate_telemetry_socorro_crash.py

That will download 100 crashes, run the JSON Schema validator against those crashes with your local telemetry_socorro_crash.json file.

Note

The validate_telemetry_socorro_crash.py, by default, does a Super Search query for basically product=Firefox and takes the 100 most recent crash IDs. This might miss out on some more “rare” crashes whose additional values might better test your JSON Schema changes. To remedy that, go to Super Search in your browser, make a search that you know includes good crash IDs to test and paste that URL like this:

$ python socorro/schemas/validate_telemetry_socorro_crash.py \
      "https://crash-stats.mozilla.org/search/?dom_ipc_enabled=%21__null__&memory_images=%3E10&version=54.0a1" \
      "https://crash-stats.mozilla.org/api/SuperSearch/?memory_private=%3E100&product=Firefox&date=%3E%3D2017-02-24T16%3A14%3A00.000Z&date=%3C2017-03-03T16%3A14%3A00.000Z"