Configuration

Processor configuration

Honcho runs a processor process and a disk cache manager process. Both share the same configuration.

Configuration

Configuration summary:

Setting

Parser

Required?

TOOL_ENV

bool

LOCAL_DEV_ENV

bool

HOSTNAME

str

SENTRY_DSN

str

LOGGING_LEVEL

str

STATSD_HOST

str

STATSD_PORT

int

ELASTICSEARCH_MODE

es_mode_parser

CACHE_MANAGER_LOGGING_LEVEL

str

SYMBOLS_CACHE_PATH

str

SYMBOLS_CACHE_MAX_SIZE

or_none(parse_data_size)

BETAVERSIONRULE_VERSION_STRING_API

str

STAGE_SUBMITTER_LOGGING_LEVEL

str

STAGE_SUBMITTER_DESTINATIONS

ListOf(str)

PROCESSOR_TEMPORARY_PATH

str

STACKWALKER_COMMAND_PATH

str

STACKWALKER_KILL_TIMEOUT

or_none(parse_time_period)

STACKWALKER_SYMBOLS_URLS

ListOf(str)

SYMBOLS_TMP_PATH

str

PUBSUB_PROJECT_ID

str

PUBSUB_STANDARD_TOPIC_NAME

str

PUBSUB_STANDARD_SUBSCRIPTION_NAME

str

PUBSUB_PRIORITY_TOPIC_NAME

str

PUBSUB_PRIORITY_SUBSCRIPTION_NAME

str

PUBSUB_REPROCESSING_TOPIC_NAME

str

PUBSUB_REPROCESSING_SUBSCRIPTION_NAME

str

TELEMETRY_GCS_BUCKET

str

ELASTICSEARCH_INDEX

str

ELASTICSEARCH_INDEX_REGEX

str

ELASTICSEARCH_URL

str

Yes

PROCESSOR_NUMBER_OF_THREADS

or_none(int)

PROCESSOR_MAXIMUM_QUEUE_SIZE

or_none(int)

Configuration options:

TOOL_ENV
Parser:

bool

Default:

“False”

Required:

No

Whether or not this is running in a tool environment and should ignore required configuration.

LOCAL_DEV_ENV
Parser:

bool

Default:

“False”

Required:

No

Whether or not this is a local development environment.

HOSTNAME
Parser:

str

Default:

“socket.gethostname()”

Required:

No

Name of the host this is running on.

SENTRY_DSN
Parser:

str

Default:

“”

Required:

No

Sentry DSN or empty string to disable.

LOGGING_LEVEL
Parser:

str

Default:

“INFO”

Required:

No

Default logging level. Should be one of DEBUG, INFO, WARNING, ERROR.

STATSD_HOST
Parser:

str

Default:

“localhost”

Required:

No

statsd host.

STATSD_PORT
Parser:

int

Default:

“8125”

Required:

No

statsd port.

ELASTICSEARCH_MODE
Parser:

es_mode_parser

Default:

“LEGACY_ONLY”

Required:

No

Elasticsearch transition mode. Should be one of LEGACY_ONLY or PREFER_NEW. When set to LEGACY_ONLY the processor and webapp will only use legacy es. When set to PREFER_NEW the processor will write to both and the webapp will only use new.

CACHE_MANAGER_LOGGING_LEVEL
Parser:

str

Default:

“INFO”

Required:

No

Default logging level for the cache manager. Should be one of DEBUG, INFO, WARNING, ERROR.

SYMBOLS_CACHE_PATH
Parser:

str

Default:

“os.path.join(tempfile.gettempdir(), “symbols”, “cache”)”

Required:

No

Directory to use for the on-disk LRU-cache for symbols files.

SYMBOLS_CACHE_MAX_SIZE
Parser:

or_none(parse_data_size)

Default:

“40gb”

Required:

No

Max size (bytes) of symbols cache. You can use _ to group digits for legibility. You can use units like kb, mb, gb, tb, etc.

BETAVERSIONRULE_VERSION_STRING_API
Parser:

str

Default:

https://crash-stats.mozilla.org/api/VersionString

Required:

No

URL for the version string API endpoint in the Crash Stats webapp.

STAGE_SUBMITTER_LOGGING_LEVEL
Parser:

str

Default:

“INFO”

Required:

No

Default logging level for the stage submitter. Should be one of DEBUG, INFO, WARNING, ERROR.

STAGE_SUBMITTER_DESTINATIONS
Parser:

ListOf(str)

Default:

“”

Required:

No

Comma-separated pairs of DESTINATION_URL|SAMPLE where the DESTINATION_URL is an https url to submit the crash report to and SAMPLE is a number between 0 and 100 representing the sample rate. For example:

  • https://example.com|20

  • https://example.com|30,https://example2.com|100

PROCESSOR_TEMPORARY_PATH
Parser:

str

Default:

“tempfile.gettempdir()”

Required:

No

Directory to use as a workspace for crash report processing.

STACKWALKER_COMMAND_PATH
Parser:

str

Default:

“/stackwalk-rust/minidump-stackwalk”

Required:

No

Aboslute path to the stackwalker binary.

STACKWALKER_KILL_TIMEOUT
Parser:

or_none(parse_time_period)

Default:

“2m”

Required:

No

Timeout in seconds before the stackwalker is killed.

STACKWALKER_SYMBOLS_URLS
Parser:

ListOf(str)

Default:

“”

Required:

No

Comma-separated list of urls for symbols suppliers.

SYMBOLS_TMP_PATH
Parser:

str

Default:

“os.path.join(tempfile.gettempdir(), “symbols”, “tmp”)”

Required:

No

Directory to use for temporary storage of files being downloaded.

PUBSUB_PROJECT_ID
Parser:

str

Default:

“test”

Required:

No

Google Compute Platform project_id.

PUBSUB_STANDARD_TOPIC_NAME
Parser:

str

Default:

“standard-queue”

Required:

No

Topic name for the standard processing queue.

PUBSUB_STANDARD_SUBSCRIPTION_NAME
Parser:

str

Default:

“standard-queue”

Required:

No

Subscription name for the standard processing queue.

PUBSUB_PRIORITY_TOPIC_NAME
Parser:

str

Default:

“priority-queue”

Required:

No

Topic name for the priority processing queue.

PUBSUB_PRIORITY_SUBSCRIPTION_NAME
Parser:

str

Default:

“priority-queue”

Required:

No

Subscription name for the priority processing queue.

PUBSUB_REPROCESSING_TOPIC_NAME
Parser:

str

Default:

“reprocessing-queue”

Required:

No

Topic name for the reprocessing queue.

PUBSUB_REPROCESSING_SUBSCRIPTION_NAME
Parser:

str

Default:

“reprocessing-queue”

Required:

No

Subscription name for the reprocessing queue.

TELEMETRY_GCS_BUCKET
Parser:

str

Default:

“”

Required:

No

GCS bucket name for telemetry data export.

ELASTICSEARCH_INDEX
Parser:

str

Default:

“socorro%Y%W”

Required:

No

Template for Elasticsearch index names.

ELASTICSEARCH_INDEX_REGEX
Parser:

str

Default:

“^socorro[0-9]{6}$”

Required:

No

Regex for matching Elasticsearch index names.

ELASTICSEARCH_URL
Parser:

str

Required:

Yes

Elasticsearch url.

PROCESSOR_NUMBER_OF_THREADS
Parser:

or_none(int)

Default:

“4”

Required:

No

Number of worker threads for the processor.

PROCESSOR_MAXIMUM_QUEUE_SIZE
Parser:

or_none(int)

Default:

“8”

Required:

No

Number of items to queue up from the processing queues.

Webapp / crontabber configuration

Gunicorn configuration:

GUNICORN_TIMEOUT
Parser:

str

Default:

“300”

Required:

No

Specifies the timeout value in seconds.

https://docs.gunicorn.org/en/stable/settings.html#timeout

Used in bin/run_service_webapp.sh.

GUNICORN_WORKERS
Parser:

str

Default:

“1”

Required:

No

Specifies the number of gunicorn workers.

You should set it to (2 x $num_cores) + 1.

https://docs.gunicorn.org/en/stable/settings.html#workers

http://docs.gunicorn.org/en/stable/design.html#how-many-workers

Used in bin/run_service_webapp.sh.

GUNICORN_WORKER_CLASS
Parser:

str

Default:

“sync”

Required:

No

Specifies the gunicorn worker type.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_service_webapp.sh.

GUNICORN_MAX_REQUESTS
Parser:

str

Default:

“10000”

Required:

No

The number of requests before recycling the gunicorn worker.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_service_webapp.sh.

GUNICORN_MAX_REQUESTS_JITTER
Parser:

str

Default:

“1000”

Required:

No

The range to generate a random amount to add to max requests so that everything isn’t restarting at the same time.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_service_webapp.sh.

Webapp configuration:

Configuration

Configuration summary:

Setting

Parser

Required?

TOOL_ENV

parse_bool

LOCAL_DEV_ENV

parse_bool

DEBUG

parse_bool

STATIC_ROOT

str

ALLOWED_HOSTS

ListOf(str)

LOGGING_LEVEL

str

DJANGO_LOGGING_LEVEL

str

HOSTNAME

str

BZAPI_BASE_URL

str

BZAPI_TOKEN

str

CACHE_IMPLEMENTATION_FETCHES

parse_bool

STATSD_HOST

str

STATSD_PORT

int

TIME_ZONE

str

DATABASE_URL

str

NPM_ROOT_PATH

str

SECRET_KEY

str

Yes

SESSION_COOKIE_SECURE

parse_bool

SESSION_COOKIE_HTTPONLY

parse_bool

X_FRAME_OPTIONS

str

OVERVIEW_VERSION_URLS

str

SENTRY_DSN

str

ANALYZE_MODEL_FETCHES

parse_bool

OIDC_RP_CLIENT_ID

str

OIDC_RP_CLIENT_SECRET

str

OIDC_OP_AUTHORIZATION_ENDPOINT

str

OIDC_OP_TOKEN_ENDPOINT

str

OIDC_OP_USER_ENDPOINT

str

LAST_LOGIN_MAX

int

NUMBER_OF_FEATURED_VERSIONS

int

VERSIONS_WINDOW_DAYS

int

VERSIONS_COUNT_THRESHOLD

int

LESS_BINARY

str

UGLIFYJS_BINARY

str

CSSMIN_BINARY

str

CACHE_LOCATION

str

CACHE_TIMEOUT

int

CACHE_KEY_PREFIX

str

Configuration options:

TOOL_ENV
Parser:

parse_bool

Default:

“false”

Required:

No

Whether or not we’re running in a tool environment where we want to ignore required configuration

LOCAL_DEV_ENV
Parser:

parse_bool

Default:

“false”

Required:

No

Whether or not we’re running in the local development environment

DEBUG
Parser:

parse_bool

Default:

“false”

Required:

No

Debugging displays nice error messages, but leaks memory. Set this to false on all server instances and true only for development.

STATIC_ROOT
Parser:

str

Default:

“path(“static”)”

Required:

No

Absolute path to the directory static files should be collected to.

ALLOWED_HOSTS
Parser:

ListOf(str)

Default:

“”

Required:

No

LOGGING_LEVEL
Parser:

str

Default:

“INFO”

Required:

No

Logging level for Crash Stats code

DJANGO_LOGGING_LEVEL
Parser:

str

Default:

“INFO”

Required:

No

Logging level for Django logging (requests, SQL, etc)

HOSTNAME
Parser:

str

Default:

“socket.gethostname()”

Required:

No

Name of the host this is running on.

BZAPI_BASE_URL
Parser:

str

Default:

https://bugzilla.mozilla.org/rest

Required:

No

Base URL for Bugzilla API

BZAPI_TOKEN
Parser:

str

Default:

“”

Required:

No

Bugzilla API token

CACHE_IMPLEMENTATION_FETCHES
Parser:

parse_bool

Default:

“true”

Required:

No

STATSD_HOST
Parser:

str

Default:

“localhost”

Required:

No

statsd host.

STATSD_PORT
Parser:

int

Default:

“8125”

Required:

No

statsd port.

TIME_ZONE
Parser:

str

Default:

“UTC”

Required:

No

DATABASE_URL
Parser:

str

Default:

“sqlite://sqlite.crashstats.db”

Required:

No

NPM_ROOT_PATH
Parser:

str

Default:

“ROOT”

Required:

No

SECRET_KEY
Parser:

str

Required:

Yes

Make this unique, and don’t share it with anybody. It cannot be blank.

Parser:

parse_bool

Default:

“true”

Required:

No

If you intend to run WITHOUT HTTPS, such as local development, then set this to false

Parser:

parse_bool

Default:

“true”

Required:

No

By default, use HTTPONLY cookies

X_FRAME_OPTIONS
Parser:

str

Default:

“DENY”

Required:

No

By default, we don’t want to be inside a frame. If you need to override this you can use the django.views.decorators.clickjacking.xframe_options_sameorigin decorator on specific views that can be in a frame.

OVERVIEW_VERSION_URLS
Parser:

str

Default:

“”

Required:

No

Comma-separated list of urls that serve version information in JSON format

SENTRY_DSN
Parser:

str

Default:

“”

Required:

No

Sentry aggregates reports of uncaught errors and other events

ANALYZE_MODEL_FETCHES
Parser:

parse_bool

Default:

“true”

Required:

No

Set to true enable analysis of all model fetches

OIDC_RP_CLIENT_ID
Parser:

str

Default:

“”

Required:

No

OIDC_RP_CLIENT_SECRET
Parser:

str

Default:

“”

Required:

No

OIDC_OP_AUTHORIZATION_ENDPOINT
Parser:

str

Default:

“”

Required:

No

OIDC_OP_TOKEN_ENDPOINT
Parser:

str

Default:

“”

Required:

No

OIDC_OP_USER_ENDPOINT
Parser:

str

Default:

“”

Required:

No

LAST_LOGIN_MAX
Parser:

int

Default:

“str(60 * 60 * 24)”

Required:

No

Max number of seconds you are allowed to be logged in with OAuth2. When theuser has been logged in >= this number, the user is automatically logged out.

Parser:

int

Default:

“4”

Required:

No

This is the number of versions to display if a particular product has no ‘featured versions’. Then we use the active versions, but capped up to this number.

VERSIONS_WINDOW_DAYS
Parser:

int

Default:

“60”

Required:

No

Number of days to look at for versions in crash reports. This is set for two months. If we haven’t gotten a crash report for some version in two months, then seems like that version isn’t active.

VERSIONS_COUNT_THRESHOLD
Parser:

int

Default:

“50”

Required:

No

Minimum number of crash reports in the VERSIONS_WINDOW_DAYS to be considered as a valid version.

LESS_BINARY
Parser:

str

Default:

“path(“node_modules/.bin/lessc”)”

Required:

No

UGLIFYJS_BINARY
Parser:

str

Default:

“path(“node_modules/.bin/uglifyjs”)”

Required:

No

CSSMIN_BINARY
Parser:

str

Default:

“path(“node_modules/.bin/cssmin”)”

Required:

No

CACHE_LOCATION
Parser:

str

Default:

“127.0.0.1:11211”

Required:

No

CACHE_TIMEOUT
Parser:

int

Default:

“500”

Required:

No

CACHE_KEY_PREFIX
Parser:

str

Default:

“socorro”

Required:

No