Crash pings¶
crash report vs. crash ping¶
We have two different methods of collecting crash data.
- crash report
The crash reporter assembles a crash report with crash annotations and minidumps and some other things. It prompts the user for permission to submit the crash report. If the user says “yes”, then the crash reporter submits it to the crash ingestion pipeline where it’s collected, processed, and available in Crash Stats.
Crash report data is off by default.
Crash report data contains protected data which includes sensitive data, PII, etc.
Not all crashes result in a crash report sent to the crash ingestion pipeline for various technical reasons.
Not all users say “yes” to submit the crash report.
The crash ingestion pipeline throttles crash reports. For example, it only accepts 10% of incoming Firefox desktop release channel crash reports and rejects the rest.
See also
- Collector throttle rules:
https://github.com/mozilla-services/antenna/blob/main/antenna/throttler.py
- crash ping
The crash reporter walks the stack and assembles a crash ping which a subset of crash annotation data. It sends this using Telemetry clients to the Telemetry data ingestion pipeline. It is available in the
telemetry.crash
BigQuery table.Crash ping data is on by default and shares opt-out mechanisms with Telemetry data submission.
Crash ping data does not contain protected data.
As of 2022-10-18, we only support crash ping data with Firefox.
See also
- Crash ping documentation:
https://docs.telemetry.mozilla.org/datasets/pings.html#crash-ping
See also
- Older blog post on crash reports vs. crash pings (2019):
https://bluesock.org/~willkg/blog/mozilla/crash_pings_crash_reports.html
Updating crash ping schema¶
After someone has added a new field to CrashAnnotations.yaml
which has ping: true
in it, we do the following:
From a mozilla-pipeline-schemas (https://github.com/mozilla-services/mozilla-pipeline-schemas/) checkout, run:
scripts/extract_crash_annotation_fields /path/to/mozilla-unified/toolkit/crashreporter/CrashAnnotations.yaml
If any exist, you will get a list of crash annotations that are contained in the ping but are not yet in the schema.
Add the annotation to
templates/telemetry/crash/crash.4.schema.json
under thepayload/metadata
section.Follow https://github.com/mozilla-services/mozilla-pipeline-schemas#cmake-build-instructions to build the schema files.
Create a pull request against the main branch in mozilla-pipeline-schemas referencing that bug.
Example: https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/711