Parsers

What’s a parser?

All parsers are functions that take a string value and return a parsed instance.

For example:

int takes a string value and returns an int.
parse_class takes a string value that’s a dotted Python path and returns the class object
ListOf(str) takes a string value that uses a comma delimiter and returns a list of strings

Note

When specifying configuration options, the default value must always be a string. When Everett can’t find a value for a requested key, it will take the default value and pass it through the parser. Because parsers always take a string as input, the default value must always be a string.

Good:

debug = config("debug", parser=bool, default="false")
                                             ^^^^^^^

Bad:

debug = config("debug", parser=bool, default=False)
                                             ^^^^^ Not a string

Available parsers 

Python types like str, int, float, pathlib.Path 

Python types can convert strings to Python values. You can use these as parsers:

str
int
float
decimal
pathlib.Path

bools 

Everett provides a special bool parser that handles more descriptive values for “true” and “false”:

true: t, true, yes, y, on, 1 (and uppercase versions)
false: f, false, no, n, off, 0 (and uppercase versions)

everett.manager.parse_bool(val)

Parse a bool value.

Handles a series of values, but you should probably standardize on “true” and “false”.

>>> from everett.manager import parse_bool
>>> parse_bool("y")
True
>>> parse_bool("FALSE")
False

Parameters:: val (str) –
Return type:: bool

classes 

Everett provides a everett.manager.parse_class that takes a string specifying a module and class and returns the class.

everett.manager.parse_class(val)

Parse a string, imports the module and returns the class.

>>> from everett.manager import parse_class
>>> parse_class("everett.manager.Option")
<class 'everett.manager.Option'>

Parameters:: val (str) –
Return type:: Any

data size 

Everett provides a everett.manager.parse_data_size that takes a string specifying an amount and a data size metric (e.g. kb, kib, tb, etc) and returns the amount of bytes that represents.

everett.manager.parse_data_size(val)

Parse a string denoting a data size into an int of bytes.

This allows you to parse data sizes with a number and then the metric. Examples:

10b - 10 bytes
100kb - 100 kilobytes = 100 * 1000
40gb - 40 gigabytes = 40 * 1000^3
23gib - 40 gibibytes = 23 * 1024^3

Supported metrics:

b - bytes
decimal:
- kb - kilobytes
- mb - megabytes
- gb - gigabytes
- tb - terabytes
- pb - petabytes
binary:
- kib - kibibytes
- mib - mebibytes
- gib - gibibytes
- tib - tebibytes
- pib - pebibytes

The metrics are not case sensitive–it supports upper, lower, and mixed case.

>>> from everett.manager import parse_data_size
>>> parse_data_size("40_000_000")
40000000
>>> parse_data_size("40gb")
40000000000
>>> parse_data_size("20KiB")
20480

Parameters:: val (str) –
Return type:: Any

time period 

Everett provides a everett.manager.parse_time_period that takes a string specifying a period of time and returns the total number of seconds represented by that period.

everett.manager.parse_data_size(val)

Parse a string denoting a data size into an int of bytes.

This allows you to parse data sizes with a number and then the metric. Examples:

10b - 10 bytes
100kb - 100 kilobytes = 100 * 1000
40gb - 40 gigabytes = 40 * 1000^3
23gib - 40 gibibytes = 23 * 1024^3

Supported metrics:

b - bytes
decimal:
- kb - kilobytes
- mb - megabytes
- gb - gigabytes
- tb - terabytes
- pb - petabytes
binary:
- kib - kibibytes
- mib - mebibytes
- gib - gibibytes
- tib - tebibytes
- pib - pebibytes

The metrics are not case sensitive–it supports upper, lower, and mixed case.

>>> from everett.manager import parse_data_size
>>> parse_data_size("40_000_000")
40000000
>>> parse_data_size("40gb")
40000000000
>>> parse_data_size("20KiB")
20480

Parameters:: val (str) –
Return type:: Any

ListOf(parser)

Everett provides a special everett.manager.ListOf parser which parses a list of some other type. For example:

ListOf(str)  # comma-delimited list of strings
ListOf(int)  # comma-delimited list of ints

everett.manager.ListOf(parser, delimiter=',', allow_empty=True)

Parse a delimiter-separated list of things.

After delimiting items, this strips the whitespace at the beginning and end of each string. Then it passes each string into the parser to get the final value.

>>> from everett.manager import ListOf
>>> ListOf(str)('')
[]
>>> ListOf(str)('a,b,c,d')
['a', 'b', 'c', 'd']
>>> ListOf(int)('1,2,3,4')
[1, 2, 3, 4]
>>> ListOf(str)('1, 2  ,3,4')
['1', '2', '3', '4']

ListOf defaults to using a comma as a delimiter, but supports other delimiters:

>>> ListOf(str, delimiter=":")("/path/a/:/path/b/")
['/path/a/', '/path/b/']

ListOf supports raising a configuration error when one of the values is an empty string:

>>> ListOf(str, allow_empty=False)("a,,b")
Traceback (most recent call last):
  ...
ValueError: 'a,,b' can not have empty values

The user will get a configuration error like this:

ValueError: 'a,,b' can not have empty values
NAMES requires a value parseable by <ListOf(str, delimiter=',', allow_empty=False)>

Note: This doesn’t handle quotes or backslashes or any complicated string parsing.

For example:

>>> ListOf(str)('"a,b",c,d')
['"a', 'b"', 'c', 'd']

Parameters:

parser (Callable) –
delimiter (str) –
allow_empty (bool) –

ChoiceOf(parser, list-of-choices)

Everett provides a everett.manager.ChoiceOf parser which can enforce that configuration values belong to a specificed value domain.

everett.manager.ChoiceOf(parser, choices)

Parser that enforces values are in a specified value domain.

Choices can be a list of string values that are parseable by the sub parser. For example, say you only supported two cloud providers and need the configuration value to be one of “aws” or “gcp”:

>>> from everett.manager import ChoiceOf
>>> ChoiceOf(str, choices=["aws", "gcp"])("aws")
'aws'

Choices works with the int sub-parser:

>>> from everett.manager import ChoiceOf
>>> ChoiceOf(int, choices=["1", "2", "3"])("1")
1

Choices works with any sub-parser:

>>> from everett.manager import ChoiceOf, parse_data_size
>>> ChoiceOf(parse_data_size, choices=["1kb", "1mb", "1gb"])("1mb")
1000000

Note: The choices list is a list of strings–these are values before being parsed. This makes it easier for people who are doing configuration to know what the values they put in their configuration files need to look like.

Parameters:

parser (Callable) –
choices (list[str]) –

dj_database_url 

Everett works with dj-database-url. The dj_database_url.parse function takes a string and returns a Django database connection value.

For example:

import dj_database_url
from everett.manager import ConfigManager

config = ConfigManager.basic_config()
DATABASES = {
    "default": config("DATABASE_URL", parser=dj_database_url.parse)
}

That’ll pull the DATABASE_URL value from the environment (it throws an error if it’s not there) and runs it through dj_database_url which parses it and returns what Django needs.

With a default:

import dj_database_url
from everett.manager import ConfigManager

config = ConfigManager.basic_config()
DATABASES = {
    "default": config(
        "DATABASE_URL", default="sqlite:///my.db", parser=dj_database_url.parse
    )
}

Note

To use dj-database-url, you’ll need to install it separately. Everett doesn’t depend on it or require it to be installed.

django-cache-url 

Everett works with django-cache-url.

For example:

import django_cache_url
from everett.manager import ConfigManager

config = ConfigManager.basic_config()
CACHES = {
    "default": config("CACHE_URL", parser=django_cache_url.parse)
}

That’ll pull the CACHE_URL value from the environment (it throws an error if it’s not there) and runs it through django_cache_url which parses it and returns what Django needs.

With a default:

import django_cache_url
from everett.manager import ConfigManager

config = ConfigManager.basic_config()
CACHES = {
    "default": config(
        "CACHE_URL", default="locmem://myapp", parser=django_cache_url.parse
    )
}

Note

To use django-cache-url, you’ll need to install it separately. Everett doesn’t require it to be installed.

Implementing your own parsers 

Implementing your own parser should be straight-forward. Parsing functions always take a string and return the Python value you need.

If the value is not parseable, the parsing function should raise a ValueError.

For example, say we wanted to implement a parser that returned yes/no/no-answer or a parser class that’s line delimited:

# parser_examples.py

from everett.manager import ConfigManager, get_parser


def parse_ynm(val):
    """Returns True, False or None (empty string)"""
    val = val.strip().lower()
    if not val:
        return None

    return val[0] == "y"


config = ConfigManager.from_dict(
    {"NO_ANSWER": "", "YES": "yes", "ALSO_YES": "y", "NO": "no"}
)

assert config("no_answer", parser=parse_ynm) is None
assert config("yes", parser=parse_ynm) is True
assert config("also_yes", parser=parse_ynm) is True
assert config("no", parser=parse_ynm) is False


class Pairs(object):
    def __init__(self, val_parser):
        self.val_parser = val_parser

    def __call__(self, val):
        val_parser = get_parser(self.val_parser)
        out = []
        for part in val.split(","):
            k, v = part.split(":")
            out.append((k, val_parser(v)))
        return out


config = ConfigManager.from_dict({"FOO": "a:1,b:2,c:3"})

assert config("FOO", parser=Pairs(int)) == [("a", 1), ("b", 2), ("c", 3)]