Configuration

Processor configuration

Honcho runs a processor process and a disk cache manager process. Both share the same configuration.

Configuration

Configuration summary:

Setting

Parser

Required?

TOOL_ENV

bool

LOCAL_DEV_ENV

bool

HOST_ID

str

SENTRY_DSN

str

LOGGING_LEVEL

str

LOCAL_DEV_AWS_ENDPOINT_URL

str

CACHE_MANAGER_LOGGING_LEVEL

str

SYMBOLS_CACHE_PATH

str

SYMBOLS_CACHE_MAX_SIZE

or_none(parse_data_size)

BETAVERSIONRULE_VERSION_STRING_API

str

PROCESSOR_TEMPORARY_PATH

str

STACKWALKER_COMMAND_PATH

str

STACKWALKER_KILL_TIMEOUT

or_none(parse_time_period)

STACKWALKER_SYMBOLS_URLS

ListOf(str)

SYMBOLS_TMP_PATH

str

SQS_STANDARD_QUEUE

str

SQS_PRIORITY_QUEUE

str

SQS_REPROCESSING_QUEUE

str

TELEMETRY_S3_ACCESS_KEY

str

TELEMETRY_S3_SECRET_ACCESS_KEY

str

TELEMETRY_S3_REGION

str

TELEMETRY_S3_BUCKET

str

ELASTICSEARCH_INDEX

str

ELASTICSEARCH_INDEX_REGEX

str

ELASTICSEARCH_URL

str

Yes

STATSD_HOST

str

STATSD_PORT

int

PROCESSOR_NUMBER_OF_THREADS

or_none(int)

PROCESSOR_MAXIMUM_QUEUE_SIZE

or_none(int)

Configuration options:

TOOL_ENV
Parser

bool

Default

“False”

Required

No

Whether or not this is running in a tool environment and should ignore required configuration.

LOCAL_DEV_ENV
Parser

bool

Default

“False”

Required

No

Whether or not this is a local development environment.

HOST_ID
Parser

str

Default

“socket.gethostname()”

Required

No

Name of the host this is running on.

SENTRY_DSN
Parser

str

Default

“”

Required

No

Sentry DSN or empty string to disable.

LOGGING_LEVEL
Parser

str

Default

“INFO”

Required

No

Default logging level. Should be one of INFO, DEBUG, WARNING, ERROR.

LOCAL_DEV_AWS_ENDPOINT_URL
Parser

str

Default

“”

Required

No

Endpoint url for AWS SQS/S3 in the local dev environment. Don’t set this in server environments.

CACHE_MANAGER_LOGGING_LEVEL
Parser

str

Default

“INFO”

Required

No

Default logging level for the cache manager. Should be one of INFO, DEBUG, WARNING, ERROR.

SYMBOLS_CACHE_PATH
Parser

str

Default

“os.path.join(tempfile.gettempdir(), “symbols”, “cache”)”

Required

No

Directory to use for the on-disk LRU-cache for symbols files.

SYMBOLS_CACHE_MAX_SIZE
Parser

or_none(parse_data_size)

Default

“40gb”

Required

No

Max size (bytes) of symbols cache. You can use _ to group digits for legibility. You can use units like kb, mb, gb, tb, etc.

BETAVERSIONRULE_VERSION_STRING_API
Parser

str

Default

https://crash-stats.mozilla.org/api/VersionString

Required

No

URL for the version string API endpoint in the Crash Stats webapp.

PROCESSOR_TEMPORARY_PATH
Parser

str

Default

“tempfile.gettempdir()”

Required

No

Directory to use as a workspace for crash report processing.

STACKWALKER_COMMAND_PATH
Parser

str

Default

“/stackwalk-rust/minidump-stackwalk”

Required

No

Aboslute path to the stackwalker binary.

STACKWALKER_KILL_TIMEOUT
Parser

or_none(parse_time_period)

Default

“2m”

Required

No

Timeout in seconds before the stackwalker is killed.

STACKWALKER_SYMBOLS_URLS
Parser

ListOf(str)

Default

“”

Required

No

Comma-separated list of urls for symbols suppliers.

SYMBOLS_TMP_PATH
Parser

str

Default

“os.path.join(tempfile.gettempdir(), “symbols”, “tmp”)”

Required

No

Directory to use for temporary storage of files being downloaded.

SQS_STANDARD_QUEUE
Parser

str

Default

“standard-queue”

Required

No

Name for the standard processing queue.

SQS_PRIORITY_QUEUE
Parser

str

Default

“priority-queue”

Required

No

Name for the priority processing queue.

SQS_REPROCESSING_QUEUE
Parser

str

Default

“reprocessing-queue”

Required

No

Name for the reprocessing queue.

TELEMETRY_S3_ACCESS_KEY
Parser

str

Default

“”

Required

No

S3 access key for telemetry bucket.

TELEMETRY_S3_SECRET_ACCESS_KEY
Parser

str

Default

“”

Required

No

S3 secret access key for telemetry bucket.

TELEMETRY_S3_REGION
Parser

str

Default

“”

Required

No

S3 region.

TELEMETRY_S3_BUCKET
Parser

str

Default

“”

Required

No

S3 bucket name for telemetry data export.

ELASTICSEARCH_INDEX
Parser

str

Default

“socorro%Y%W”

Required

No

Template for Elasticsearch index names.

ELASTICSEARCH_INDEX_REGEX
Parser

str

Default

“^socorro[0-9]{6}$”

Required

No

Regex for matching Elasticsearch index names.

ELASTICSEARCH_URL
Parser

str

Required

Yes

Elasticsearch url.

STATSD_HOST
Parser

str

Default

“localhost”

Required

No

statsd host.

STATSD_PORT
Parser

int

Default

“8125”

Required

No

statsd port.

PROCESSOR_NUMBER_OF_THREADS
Parser

or_none(int)

Default

“4”

Required

No

Number of worker threads for the processor.

PROCESSOR_MAXIMUM_QUEUE_SIZE
Parser

or_none(int)

Default

“8”

Required

No

Number of items to queue up from the processing queues.

Webapp / crontabber configuration

Gunicorn configuration:

GUNICORN_TIMEOUT
Parser

str

Default

“300”

Required

No

Specifies the timeout value in seconds.

https://docs.gunicorn.org/en/stable/settings.html#timeout

Used in bin/run_webapp.sh.

GUNICORN_WORKERS
Parser

str

Default

“1”

Required

No

Specifies the number of gunicorn workers.

You should set it to (2 x $num_cores) + 1.

https://docs.gunicorn.org/en/stable/settings.html#workers

http://docs.gunicorn.org/en/stable/design.html#how-many-workers

Used in bin/run_webapp.sh.

GUNICORN_WORKER_CLASS
Parser

str

Default

“sync”

Required

No

Specifies the gunicorn worker type.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

GUNICORN_MAX_REQUESTS
Parser

str

Default

“10000”

Required

No

The number of requests before recycling the gunicorn worker.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

GUNICORN_MAX_REQUESTS_JITTER
Parser

str

Default

“1000”

Required

No

The range to generate a random amount to add to max requests so that everything isn’t restarting at the same time.

https://docs.gunicorn.org/en/stable/settings.html#workers

Used in bin/run_webapp.sh.

Webapp configuration:

To be documented.