Configuration¶

Processor configuration
Webapp / crontabber configuration

Processor configuration ¶

Honcho runs a processor process and a disk cache manager process. Both share the same configuration.

Configuration¶

Configuration summary:

Setting	Parser	Required?
`TOOL_ENV`	bool
`LOCAL_DEV_ENV`	bool
`HOST_ID`	str
`SENTRY_DSN`	str
`LOGGING_LEVEL`	str
`LOCAL_DEV_AWS_ENDPOINT_URL`	str
`CLOUD_PROVIDER`	cloud_provider_parser
`CACHE_MANAGER_LOGGING_LEVEL`	str
`SYMBOLS_CACHE_PATH`	str
`SYMBOLS_CACHE_MAX_SIZE`	or_none(parse_data_size)
`BETAVERSIONRULE_VERSION_STRING_API`	str
`PROCESSOR_TEMPORARY_PATH`	str
`STACKWALKER_COMMAND_PATH`	str
`STACKWALKER_KILL_TIMEOUT`	or_none(parse_time_period)
`STACKWALKER_SYMBOLS_URLS`	ListOf(str)
`SYMBOLS_TMP_PATH`	str
`SQS_STANDARD_QUEUE`	str
`SQS_PRIORITY_QUEUE`	str
`SQS_REPROCESSING_QUEUE`	str
`TELEMETRY_S3_ACCESS_KEY`	str
`TELEMETRY_S3_SECRET_ACCESS_KEY`	str
`TELEMETRY_S3_REGION`	str
`PUBSUB_PROJECT_ID`	str
`PUBSUB_STANDARD_TOPIC_NAME`	str
`PUBSUB_STANDARD_SUBSCRIPTION_NAME`	str
`PUBSUB_PRIORITY_TOPIC_NAME`	str
`PUBSUB_PRIORITY_SUBSCRIPTION_NAME`	str
`PUBSUB_REPROCESSING_TOPIC_NAME`	str
`PUBSUB_REPROCESSING_SUBSCRIPTION_NAME`	str
`TELEMETRY_S3_BUCKET`	str
`ELASTICSEARCH_INDEX`	str
`ELASTICSEARCH_INDEX_REGEX`	str
`ELASTICSEARCH_URL`	str	Yes
`STATSD_HOST`	str
`STATSD_PORT`	int
`PROCESSOR_NUMBER_OF_THREADS`	or_none(int)
`PROCESSOR_MAXIMUM_QUEUE_SIZE`	or_none(int)

Configuration options:

TOOL_ENV¶

Parser: bool
Default: “False”
Required: No

Whether or not this is running in a tool environment and should ignore required configuration.

LOCAL_DEV_ENV¶

Parser: bool
Default: “False”
Required: No

Whether or not this is a local development environment.

HOST_ID¶

Parser: str
Default: “socket.gethostname()”
Required: No

Name of the host this is running on.

SENTRY_DSN¶

Parser: str
Default: “”
Required: No

Sentry DSN or empty string to disable.

LOGGING_LEVEL¶

Parser: str
Default: “INFO”
Required: No

Default logging level. Should be one of INFO, DEBUG, WARNING, ERROR.

LOCAL_DEV_AWS_ENDPOINT_URL¶

Parser: str
Default: “”
Required: No

Endpoint url for AWS SQS/S3 in the local dev environment. Don’t set this in server environments.

CLOUD_PROVIDER¶

Parser: cloud_provider_parser
Default: “AWS”
Required: No

The cloud provider to use for queueing and blob storage. Must be AWS or GCP.

CACHE_MANAGER_LOGGING_LEVEL¶

Parser: str
Default: “INFO”
Required: No

Default logging level for the cache manager. Should be one of INFO, DEBUG, WARNING, ERROR.

SYMBOLS_CACHE_PATH¶

Parser: str
Default: “os.path.join(tempfile.gettempdir(), “symbols”, “cache”)”
Required: No

Directory to use for the on-disk LRU-cache for symbols files.

SYMBOLS_CACHE_MAX_SIZE¶

Parser: or_none(parse_data_size)
Default: “40gb”
Required: No

Max size (bytes) of symbols cache. You can use _ to group digits for legibility. You can use units like kb, mb, gb, tb, etc.

BETAVERSIONRULE_VERSION_STRING_API¶

Parser: str
Default: “https://crash-stats.mozilla.org/api/VersionString”
Required: No

URL for the version string API endpoint in the Crash Stats webapp.

PROCESSOR_TEMPORARY_PATH¶

Parser: str
Default: “tempfile.gettempdir()”
Required: No

Directory to use as a workspace for crash report processing.

STACKWALKER_COMMAND_PATH¶

Parser: str
Default: “/stackwalk-rust/minidump-stackwalk”
Required: No

Aboslute path to the stackwalker binary.

STACKWALKER_KILL_TIMEOUT¶

Parser: or_none(parse_time_period)
Default: “2m”
Required: No

Timeout in seconds before the stackwalker is killed.

STACKWALKER_SYMBOLS_URLS¶

Parser: ListOf(str)
Default: “”
Required: No

Comma-separated list of urls for symbols suppliers.

SYMBOLS_TMP_PATH¶

Parser: str
Default: “os.path.join(tempfile.gettempdir(), “symbols”, “tmp”)”
Required: No

Directory to use for temporary storage of files being downloaded.

SQS_STANDARD_QUEUE¶

Parser: str
Default: “standard-queue”
Required: No

Name for the standard processing queue.

SQS_PRIORITY_QUEUE¶

Parser: str
Default: “priority-queue”
Required: No

Name for the priority processing queue.

SQS_REPROCESSING_QUEUE¶

Parser: str
Default: “reprocessing-queue”
Required: No

Name for the reprocessing queue.

TELEMETRY_S3_ACCESS_KEY¶

Parser: str
Default: “”
Required: No

S3 access key for telemetry bucket.

TELEMETRY_S3_SECRET_ACCESS_KEY¶

Parser: str
Default: “”
Required: No

S3 secret access key for telemetry bucket.

TELEMETRY_S3_REGION¶

Parser: str
Default: “”
Required: No

S3 region.

PUBSUB_PROJECT_ID¶

Parser: str
Default: “test”
Required: No

Google Compute Platform project_id.

PUBSUB_STANDARD_TOPIC_NAME¶

Parser: str
Default: “standard-queue”
Required: No

Topic name for the standard processing queue.

PUBSUB_STANDARD_SUBSCRIPTION_NAME¶

Parser: str
Default: “standard-queue”
Required: No

Subscription name for the standard processing queue.

PUBSUB_PRIORITY_TOPIC_NAME¶

Parser: str
Default: “priority-queue”
Required: No

Topic name for the priority processing queue.

PUBSUB_PRIORITY_SUBSCRIPTION_NAME¶

Parser: str
Default: “priority-queue”
Required: No

Subscription name for the priority processing queue.

PUBSUB_REPROCESSING_TOPIC_NAME¶

Parser: str
Default: “reprocessing-queue”
Required: No

Topic name for the reprocessing queue.

PUBSUB_REPROCESSING_SUBSCRIPTION_NAME¶

Parser: str
Default: “reprocessing-queue”
Required: No

Subscription name for the reprocessing queue.

TELEMETRY_S3_BUCKET¶

Parser: str
Default: “”
Required: No

S3 bucket name for telemetry data export.

ELASTICSEARCH_INDEX¶

Parser: str
Default: “socorro%Y%W”
Required: No

Template for Elasticsearch index names.

ELASTICSEARCH_INDEX_REGEX¶

Parser: str
Default: “^socorro[0-9]{6}$”
Required: No

Regex for matching Elasticsearch index names.

ELASTICSEARCH_URL¶

Parser: str
Required: Yes

Elasticsearch url.

STATSD_HOST¶

Parser: str
Default: “localhost”
Required: No

statsd host.

STATSD_PORT¶

Parser: int
Default: “8125”
Required: No

statsd port.

PROCESSOR_NUMBER_OF_THREADS¶

Parser: or_none(int)
Default: “4”
Required: No

Number of worker threads for the processor.

PROCESSOR_MAXIMUM_QUEUE_SIZE¶

Parser: or_none(int)
Default: “8”
Required: No

Number of items to queue up from the processing queues.

Webapp / crontabber configuration ¶

Gunicorn configuration:

GUNICORN_TIMEOUT¶

Parser: str
Default: “300”
Required: No

Specifies the timeout value in seconds.

https://docs.gunicorn.org/en/stable/settings.html#timeout

Used in bin/run_webapp.sh.

GUNICORN_WORKERS¶

Parser: str
Default: “1”
Required: No

Specifies the number of gunicorn workers.

You should set it to (2 x $num_cores) + 1.

https://docs.gunicorn.org/en/stable/settings.html#workers

http://docs.gunicorn.org/en/stable/design.html#how-many-workers

Used in bin/run_webapp.sh.

GUNICORN_WORKER_CLASS¶

Parser: str
Default: “sync”
Required: No

Specifies the gunicorn worker type.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

GUNICORN_MAX_REQUESTS¶

Parser: str
Default: “10000”
Required: No

The number of requests before recycling the gunicorn worker.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

GUNICORN_MAX_REQUESTS_JITTER¶

Parser: str
Default: “1000”
Required: No

The range to generate a random amount to add to max requests so that everything isn’t restarting at the same time.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

Webapp configuration:

Configuration

Configuration summary:

Setting	Parser	Required?
`TOOL_ENV`	parse_bool
`LOCAL_DEV_ENV`	parse_bool
`DEBUG`	parse_bool
`STATIC_ROOT`	str
`ALLOWED_HOSTS`	ListOf(str)
`LOGGING_LEVEL`	str
`DJANGO_LOGGING_LEVEL`	str
`HOST_ID`	str
`BZAPI_BASE_URL`	str
`BZAPI_TOKEN`	str
`CACHE_IMPLEMENTATION_FETCHES`	parse_bool
`STATSD_HOST`	str
`STATSD_PORT`	int
`TIME_ZONE`	str
`DATABASE_URL`	str
`NPM_ROOT_PATH`	str
`SECRET_KEY`	str	Yes
`SESSION_COOKIE_SECURE`	parse_bool
`SESSION_COOKIE_HTTPONLY`	parse_bool
`X_FRAME_OPTIONS`	str
`OVERVIEW_VERSION_URLS`	str
`SENTRY_DSN`	str
`ANALYZE_MODEL_FETCHES`	parse_bool
`OIDC_RP_CLIENT_ID`	str
`OIDC_RP_CLIENT_SECRET`	str
`OIDC_OP_AUTHORIZATION_ENDPOINT`	str
`OIDC_OP_TOKEN_ENDPOINT`	str
`OIDC_OP_USER_ENDPOINT`	str
`LAST_LOGIN_MAX`	int
`NUMBER_OF_FEATURED_VERSIONS`	int
`VERSIONS_WINDOW_DAYS`	int
`VERSIONS_COUNT_THRESHOLD`	int
`LESS_BINARY`	str
`UGLIFYJS_BINARY`	str
`CSSMIN_BINARY`	str
`CACHE_LOCATION`	str
`CACHE_TIMEOUT`	int
`CACHE_KEY_PREFIX`	str

Configuration options:

TOOL_ENV

Parser: parse_bool
Default: “false”
Required: No

Whether or not we’re running in a tool environment where we want to ignore required configuration

LOCAL_DEV_ENV

Parser: parse_bool
Default: “false”
Required: No

Whether or not we’re running in the local development environment

DEBUG¶

Parser: parse_bool
Default: “false”
Required: No

Debugging displays nice error messages, but leaks memory. Set this to false on all server instances and true only for development.

STATIC_ROOT¶

Parser: str
Default: “path(“static”)”
Required: No

Absolute path to the directory static files should be collected to.

ALLOWED_HOSTS¶

Parser: ListOf(str)
Default: “”
Required: No

LOGGING_LEVEL

Parser: str
Default: “INFO”
Required: No

Logging level for Crash Stats code

DJANGO_LOGGING_LEVEL¶

Parser: str
Default: “INFO”
Required: No

Logging level for Django logging (requests, SQL, etc)

HOST_ID

Parser: str
Default: “socket.gethostname()”
Required: No

Name of the host this is running on. Used in logging.

BZAPI_BASE_URL¶

Parser: str
Default: “https://bugzilla.mozilla.org/rest”
Required: No

Base URL for Bugzilla API

BZAPI_TOKEN¶

Parser: str
Default: “”
Required: No

Bugzilla API token

CACHE_IMPLEMENTATION_FETCHES¶

Parser: parse_bool
Default: “true”
Required: No

STATSD_HOST

Parser: str
Default: “localhost”
Required: No

STATSD_PORT

Parser: int
Default: “8125”
Required: No

TIME_ZONE¶

Parser: str
Default: “UTC”
Required: No

DATABASE_URL¶

Parser: str
Default: “sqlite://sqlite.crashstats.db”
Required: No

NPM_ROOT_PATH¶

Parser: str
Default: “ROOT”
Required: No

SECRET_KEY¶

Parser: str
Required: Yes

Make this unique, and don’t share it with anybody. It cannot be blank.

SESSION_COOKIE_SECURE¶

Parser: parse_bool
Default: “true”
Required: No

If you intend to run WITHOUT HTTPS, such as local development, then set this to false

SESSION_COOKIE_HTTPONLY¶

Parser: parse_bool
Default: “true”
Required: No

By default, use HTTPONLY cookies

X_FRAME_OPTIONS¶

Parser: str
Default: “DENY”
Required: No

By default, we don’t want to be inside a frame. If you need to override this you can use the django.views.decorators.clickjacking.xframe_options_sameorigin decorator on specific views that can be in a frame.

OVERVIEW_VERSION_URLS¶

Parser: str
Default: “”
Required: No

Comma-separated list of urls that serve version information in JSON format

SENTRY_DSN

Parser: str
Default: “”
Required: No

Sentry aggregates reports of uncaught errors and other events

ANALYZE_MODEL_FETCHES¶

Parser: parse_bool
Default: “true”
Required: No

Set to true enable analysis of all model fetches

OIDC_RP_CLIENT_ID¶

Parser: str
Default: “”
Required: No

OIDC_RP_CLIENT_SECRET¶

Parser: str
Default: “”
Required: No

OIDC_OP_AUTHORIZATION_ENDPOINT¶

Parser: str
Default: “”
Required: No

OIDC_OP_TOKEN_ENDPOINT¶

Parser: str
Default: “”
Required: No

OIDC_OP_USER_ENDPOINT¶

Parser: str
Default: “”
Required: No

LAST_LOGIN_MAX¶

Parser: int
Default: “str(60 * 60 * 24)”
Required: No

Max number of seconds you are allowed to be logged in with OAuth2. When theuser has been logged in >= this number, the user is automatically logged out.

NUMBER_OF_FEATURED_VERSIONS¶

Parser: int
Default: “4”
Required: No

This is the number of versions to display if a particular product has no ‘featured versions’. Then we use the active versions, but capped up to this number.

VERSIONS_WINDOW_DAYS¶

Parser: int
Default: “60”
Required: No

Number of days to look at for versions in crash reports. This is set for two months. If we haven’t gotten a crash report for some version in two months, then seems like that version isn’t active.

VERSIONS_COUNT_THRESHOLD¶

Parser: int
Default: “50”
Required: No

Minimum number of crash reports in the VERSIONS_WINDOW_DAYS to be considered as a valid version.

LESS_BINARY¶

Parser: str
Default: “path(“node_modules/.bin/lessc”)”
Required: No

UGLIFYJS_BINARY¶

Parser: str
Default: “path(“node_modules/.bin/uglifyjs”)”
Required: No

CSSMIN_BINARY¶

Parser: str
Default: “path(“node_modules/.bin/cssmin”)”
Required: No

CACHE_LOCATION¶

Parser: str
Default: “127.0.0.1:11211”
Required: No

CACHE_TIMEOUT¶

Parser: int
Default: “500”
Required: No

CACHE_KEY_PREFIX¶

Parser: str
Default: “socorro”
Required: No

Configuration¶

Processor configuration¶

Webapp / crontabber configuration¶

Processor configuration ¶

Webapp / crontabber configuration ¶