Configuration File Syntax

The rejected configuration uses YAML as the markup language. YAML’s format, like Python code is whitespace dependent for control structure in blocks. If you’re having problems with your rejected configuration, the first thing you should do is ensure that the YAML syntax is correct. yamllint.com is a good resource for validating that your configuration file can be parsed.

The configuration file is split into three main sections: Application, Daemon, and Logging.

The example configuration file provides a good starting point for creating your own configuration file.

Application

The application section of the configuration is broken down into multiple top-level options:

poll_interval How often rejected should poll consumer processes for status in seconds (int/float)
sentry_dsn If Sentry support is installed, optionally set a global DSN for all consumers (str)
stats Enable and configure statsd metric submission (obj)
Connections A subsection with RabbitMQ connection information for consumers (obj)
Consumers Where each consumer type is configured (obj)

stats

stats  
  log Toggle top-level logging of consumer process stats (bool)
  influxdb Configure the submission of per-message measurements to InfluxDB (obj)
  statsd Configure the submission of per-message measurements to statsd (obj)

influxdb

stats > influxdb  
  scheme The scheme to use when submitting metrics to the InfluxDB server. Default: http (str)
  host The hostname or ip address of the InfluxDB server. Default: localhost (str)
  port The port of the influxdb server. Default: 8086 (int)
  user An optional username to use when submitting measurements. (str)
  password An optional password to use when submitting measurements. (str)
  database The InfluxDB database to submit measurements to. Default: rejected (str)

statsd

stats > statsd  
  enabled Toggle statsd reporting off and on (bool)
  prefix An optional prefix to use when creating the statsd metric path (str)
  host The hostname or ip address of the statsd server (str)
  port The port of the statsd server. Default: 8125 (int)

Connections

Each RabbitMQ connection entry should be a nested object with a unique name with connection attributes.

Connection Name  
  host The hostname or ip address of the RabbitMQ server (str)
port The port of the RabbitMQ server (int)
vhost The virtual host to connect to (str)
user The username to connect as (str)
pass The password to use (str)
heartbeat_interval Optional: the AMQP heartbeat interval (int) default: 300 sec

Consumers

Each consumer entry should be a nested object with a unique name with consumer attributes.

Consumer Name  
  consumer The package.module.Class path to the consumer code (str)
connections The connections to connect to (list) - See Consumer Connections
qty The number of consumers per connection to run (int)
queue The RabbitMQ queue name to consume from (str)
ack Explicitly acknowledge messages (no_ack = not ack) (bool)
max_errors Number of errors encountered before restarting a consumer (int)
sentry_dsn If Sentry support is installed, set a consumer specific sentry DSN (str)
drop_exchange The exchange to publish a message to when it is dropped. If not specified, dropped messages are not republished anywhere.
drop_invalid_messages Drop a message if the type property doesn’t match the specified message type (str)
message_type Used to validate the message type of a message before processing. This attribute can be set to a string that is matched against the AMQP message type or a list of acceptable message types. (str, array)
error_exchange The exchange to publish messages that raise ProcessingException to (str)
error_max_retry The number of ProcessingException raised on a message before a message is dropped. If not specified messages will never be dropped (int)
influxdb_measurement When using InfluxDB, the measurement name for per-message measurements. Defaults to the consumer name. (str)
config Free-form key-value configuration section for the consumer (obj)

Consumer Connections

The consumer connections configuration allows for one or more connections to be made by a single consumer. This configuration section supports two formats. If a list of connection names are specified, the consumer will connect to and consume from the all of the specified connections.

Consumer Name:
    connections:
      - connection1
      - connection2

If the connections list include structured values, additional settings can be set. For example, you may want to consume from one RabbitMQ broker and publish to another, as is illustrated below:

Consumer Name:
    connections:
      - name: connection1
        consume: True
        publisher_confirmation: False
      - name: connection2
        consume: False
        publisher_confirmation: True

In the above example, the consumer will have two connections, connection1 and connection2. It will only consume from connection1 but can publish messages connection2 by specifying the connection name in the publish_message() method.

Structured Connections

When specifying a structured consumer connection, the following attributes are available.

Consumer Name > connections  
  name The connection name, as specified in the Connections section of the application configuration.
consume Specify if the connection should consume on the connection. (bool)
  publisher_confirmation Enable publisher confirmations. (bool)

Daemon

This section contains the settings required to run the application as a daemon. They are as follows:

user The username to run as when the process is daemonized (bool)
group Optional The group name to switch to when the process is daemonized (str)
pidfile The pidfile to write when the process is daemonized (str)

Logging

rejected uses logging.config.dictConfig to create a flexible method for configuring the python standard logging module. If rejected is being run in Python 2.6, logutils.dictconfig.dictConfig is used instead.

The following basic example illustrates all of the required sections in the dictConfig format, implemented in YAML:

version: 1
formatters: []
verbose:
  format: '%(levelname) -10s %(asctime)s %(process)-6d %(processName) -15s %(name) -10s %(funcName) -20s: %(message)s'
  datefmt: '%Y-%m-%d %H:%M:%S'
handlers:
  console:
    class: logging.StreamHandler
    formatter: verbose
    debug_only: True
loggers:
  rejected:
    handlers: [console]
    level: INFO
    propagate: true
  myconsumer:
    handlers: [console]
    level: DEBUG
    propagate: true
disable_existing_loggers: true
incremental: false

Note

The debug_only node of the Logging > handlers > console section is not part of the standard dictConfig format. Please see the Logging Caveats section below for more information.

Logging Caveats

In order to allow for customizable console output when running in the foreground and no console output when daemonized, a debug_only node has been added to the standard dictConfig format in the handler section. This method is evaluated when logging is configured and if present, it is removed prior to passing the dictionary to dictConfig if present.

If the value is set to true and the application is not running in the foreground, the configuration for the handler and references to it will be removed from the configuration dictionary.

Troubleshooting

If you find that your application is not logging anything or sending output to the terminal, ensure that you have created a logger section in your configuration for your consumer package. For example if your Consumer instance is named myconsumer.MyConsumer make sure there is a myconsumer logger in the logging configuration.