API

This is the API of functions and classes in Everett.

Configuration things:

Utility functions:

Testing utility functions:

Configuration environments:

Errors:

Parsers:

everett

Everett is a Python library for configuration.

everett.NO_VALUE = NO_VALUE

Singleton indicating a non-value.

exception everett.ConfigurationError

Configuration error base class.

exception everett.DetailedConfigurationError(msg, namespace, key, parser)

Base class for configuration errors that have a msg, namespace, key, and parser.

Parameters:
  • msg (str) –

  • namespace (List[str] | None) –

  • key (str) –

  • parser (Callable) –

exception everett.InvalidKeyError

Error that indicates the key is not valid for this component.

exception everett.ConfigurationMissingError(msg, namespace, key, parser)

Error that indicates that required configuration is missing.

Parameters:
  • msg (str) –

  • namespace (List[str] | None) –

  • key (str) –

  • parser (Callable) –

everett.manager

Contains configuration infrastructure.

This module contains the configuration classes and functions for deriving configuration values from specified sources in the order specified.

class everett.manager.ConfigDictEnv(cfg)

Source for pulling configuration out of a dict.

This is handy for testing. You might also use it if you wanted to move all your defaults values into one centralized place.

Keys are prefixed by namespaces and the whole thing is uppercased.

For example, namespace “bar” for key “foo” becomes BAR_FOO in the dict.

For example:

from everett.manager import ConfigDictEnv, ConfigManager

config = ConfigManager([
    ConfigDictEnv({
        "FOO_BAR": "someval",
        "BAT": "1",
    })
])

Keys are not case sensitive. This also works:

from everett.manager import ConfigDictEnv, ConfigManager

config = ConfigManager([
    ConfigDictEnv({
        "foo_bar": "someval",
        "bat": "1",
    })
])

print config("foo_bar")
print config("FOO_BAR")
print config.with_namespace("foo")("bar")

Also, ConfigManager has a convenience classmethod for creating a ConfigManager with just a dict environment:

from everett.manager import ConfigManager

config = ConfigManager.from_dict({
    "FOO_BAR": "bat"
})

Changed in version 0.3: Keys are no longer case-sensitive.

Parameters:

cfg (Dict) –

__init__(cfg)
Parameters:

cfg (Dict) –

get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue

class everett.manager.ConfigEnvFileEnv(possible_paths)

Source for pulling configuration out of .env files.

This source lets you specify configuration in an .env file. This is useful for local development when in production you use values in environment variables.

Keys are prefixed by namespaces and the whole thing is uppercased.

For example, key “foo” will be FOO in the file.

For example, namespace “bar” for key “foo” becomes BAR_FOO in the file.

Key and namespace can consist of alphanumeric characters and _.

To use, instantiate and toss in the source list:

from everett.manager import ConfigEnvFileEnv, ConfigManager

config = ConfigManager([
    ConfigEnvFileEnv('.env')
])

For multiple paths:

from everett.manager import ConfigEnvFileEnv, ConfigManager

config = ConfigManager([
    ConfigEnvFileEnv([
        '.env',
        'config/prod.env'
    ])
])

Here’s an example .env file:

DEBUG=true

# secrets
SECRET_KEY=ou812

# database setup
DB_HOST=localhost
DB_PORT=5432

# CSP reporting
CSP_SCRIPT_SRC="'self' www.googletagmanager.com"
Parameters:

possible_paths (str | List[str]) –

__init__(possible_paths)
Parameters:

possible_paths (str | List[str]) –

get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue

class everett.manager.ConfigManager(environments, doc='', msg_builder=<function build_msg>, with_override=True)

Manage multiple configuration environment layers.

Instantiate a ConfigManager.

Parameters:
  • environments (List[Any]) – list of configuration sources to look through in the order they should be looked through

  • doc (str) –

    help text printed to users when they encounter configuration errors

    New in version 0.6.

  • msg_builder (Callable) –

    function that takes arguments and builds an exception message intended to be printed or conveyed to the user

    For example:

    def build_msg(namespace, key, parser, msg="", option_doc="", config_doc=""):
        full_key = namespace or []
        full_key = "_".join(full_key + [key]).upper()
    
        return (
            f"{full_key} requires a value parseable by {qualname(parser)}\n"
            + option_doc + "\n"
            + config_doc + "\n"
        )
    

  • with_override (bool) – whether or not to insert the special override environment used for testing as the first environment in the list of sources

__init__(environments, doc='', msg_builder=<function build_msg>, with_override=True)

Instantiate a ConfigManager.

Parameters:
  • environments (List[Any]) – list of configuration sources to look through in the order they should be looked through

  • doc (str) –

    help text printed to users when they encounter configuration errors

    New in version 0.6.

  • msg_builder (Callable) –

    function that takes arguments and builds an exception message intended to be printed or conveyed to the user

    For example:

    def build_msg(namespace, key, parser, msg="", option_doc="", config_doc=""):
        full_key = namespace or []
        full_key = "_".join(full_key + [key]).upper()
    
        return (
            f"{full_key} requires a value parseable by {qualname(parser)}\n"
            + option_doc + "\n"
            + config_doc + "\n"
        )
    

  • with_override (bool) – whether or not to insert the special override environment used for testing as the first environment in the list of sources

classmethod basic_config(env_file='.env', doc='')

Return a basic ConfigManager.

This sets up a ConfigManager that will look for configuration in this order:

  1. environment

  2. specified env_file defaulting to .env

This is for a fast one-line opinionated setup.

Example:

from everett.manager import ConfigManager

config = ConfigManager.basic_config()

This is shorthand for:

config = ConfigManager(
    environments=[
        ConfigOSEnv(),
        ConfigEnvFileEnv(['.env'])
    ]
)
Parameters:
  • env_file (str) – the name of the env file to use

  • doc (str) – help text printed to users when they encounter configuration errors

Returns:

a everett.manager.ConfigManager

Return type:

ConfigManager

classmethod from_dict(dict_config)

Create a ConfigManager with specified configuration as a Python dict.

This is shorthand for:

config = ConfigManager([ConfigDictEnv(dict_config)])

This is handy for writing tests for the app you’re using Everett in.

Parameters:

dict_config (Dict) – Python dict holding the configuration for this manager

Returns:

ConfigManager with specified configuration

Return type:

ConfigManager

New in version 0.3.

get_bound_component()

Retrieve the bound component for this config object.

Returns:

component or None

Return type:

Any

get_namespace()

Retrieve the complete namespace for this config object.

Returns:

namespace as a list of strings

Return type:

List[str]

with_namespace(namespace)

Apply a namespace to this configuration.

Namespaces accumulate as you add them.

Parameters:
  • namepace – namespace as a string or list of strings

  • namespace (List[str] | str) –

Returns:

a clone of the ConfigManager instance with the namespace applied

Return type:

ConfigManager

with_options(component)

Apply options component options to this configuration.

Parameters:

component (Any) – the instance or class with a Config to bind this ConfigManager to

Returns:

a clone of the ConfigManager instance bound to specified component

Return type:

ConfigManager

__call__(key, namespace=None, default=NO_VALUE, default_if_empty=True, alternate_keys=None, doc='', parser=<class 'str'>, raise_error=True, raw_value=False)

Return a parsed value from the environment.

Parameters:
  • key (str) – the key to look up

  • namespace (List[str] | str | None) – the namespace for the key–different environments use this differently

  • default (str | NoValue) –

    the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return everett.NO_VALUE depending on the value of raise_error

    If this ConfigManager is bound to a component, the default will be the default of the option in the bound component configuration.

  • default_if_empty (bool) – if True, treat empty string values as a non-value and return the specified default

  • alternate_keys (List[str] | None) –

    the list of alternate keys to look up; supports a root: key prefix which will cause this to look at the configuration root rather than the current namespace

    If this ConfigManager is bound to a component, the alternate_keys will be the alternate_keys of the option in the bound component configuration.

    New in version 0.3.

  • doc (str) –

    documentation for this config option

    If this ConfigManager is bound to a component, the doc will be the doc of the option in the bound component configuration.

    New in version 0.6.

  • parser (Callable) –

    the parser for converting this value to a Python object

    If this ConfigManager is bound to a component, the parser will be the parser of the option in the bound component configuration.

  • raise_error (bool) – True if you want a lack of value to raise a everett.ConfigurationError

  • raw_value (bool) – True if you want the raw unparsed value, False otherwise

Raises:
  • everett.ConfigurationMissingError – if the required bit of configuration is missing from all the environments

  • everett.InvalidKeyError – if the configuration key doesn’t exist for that component

  • everett.InvalidValueError – if the configuration value is invalid in some way (not an integer, not a bool, etc)

Return type:

Any

Note

The default value should always be a string that is parseable by the parser. This simplifies thinking about values since all values are strings that are parsed by the parser rather than default values do one thing and non-default values doa nother. Further, it simplifies documentation for the user since the default value is an example value.

The parser can be any callable that takes a string value and returns a parsed value.

raise_configuration_error(msg)

Convenience function for raising configuration errors.

This is helpful for situations where you need to do additional checking of configuration values and need to raise a configuration error for the user that includes the configuration documentation.

For example:

from everett.manager import ConfigManager

config = ConfigManager.basic_config()
host = config("host")
port = config("port")

if host is None or port is None:
    config.raise_configuration_error(
        "Both HOST and PORT must be specified."
    )
Parameters:

msg (str) – the configuration error message

Return type:

None

class everett.manager.ConfigOSEnv

Source for pulling configuration out of the environment.

This source lets you specify configuration in the environment. This is useful for infrastructure related configuration like usernames and ports and secret configuration like passwords.

Keys are prefixed by namespaces and the whole thing is uppercased.

For example, key “foo” will be FOO in the environment.

For example, namespace “bar” for key “foo” becomes BAR_FOO in the environment.

Key and namespace can consist of alphanumeric characters and _.

Note

Unlike other config environments, this one is case sensitive in that keys defined in the environment must be all uppercase.

For example, these are good:

FOO=bar
FOO_BAR=bar
FOO_BAR1=bar

This is bad:

foo=bar

To use, instantiate and toss in the source list:

from everett.manager import ConfigOSEnv, ConfigManager

config = ConfigManager([
    ConfigOSEnv()
])
get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue

class everett.manager.ConfigObjEnv(obj, force_lower=True)

Source for pulling configuration values out of a Python object.

This is handy for a few weird situations. For example, you can use this to “bridge” Everett configuration with command line arguments. The argparse Namespace works fine here.

Namespace (the Everett one–not the argparse one) is prefixed. So key “foo” in namespace “bar” is “foo_bar”.

For example:

import argparse

from everett.manager import ConfigObjEnv, ConfigManager

parser = argparse.ArgumentParser()
parser.add_argument(
    "--debug", help="to debug or not to debug"
)
parsed_vals = parser.parse_known_args()[0]

config = ConfigManager([
    ConfigObjEnv(parsed_vals)
])

print config("debug", parser=bool)

Keys are not case-sensitive–everything is converted to lowercase before pulling it from the object.

Note

ConfigObjEnv has nothing to do with the library configobj.

New in version 0.6.

Parameters:
  • obj (Any) –

  • force_lower (bool) –

__init__(obj, force_lower=True)
Parameters:
  • obj (Any) –

  • force_lower (bool) –

get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue

class everett.manager.ListOf(parser, delimiter=',')

Parse a comma-separated list of things.

After delimiting items, this strips the whitespace at the beginning and end of each string. Then it passes each string into the parser to get the final value.

>>> from everett.manager import ListOf
>>> ListOf(str)('')
[]
>>> ListOf(str)('a,b,c,d')
['a', 'b', 'c', 'd']
>>> ListOf(int)('1,2,3,4')
[1, 2, 3, 4]
>>> ListOf(str)('1, 2  ,3,4')
['1', '2', '3', '4']

Note: This doesn’t handle quotes or backslashes or any complicated string parsing.

For example:

>>> ListOf(str)('"a,b",c,d')
['"a', 'b"', 'c', 'd']
Parameters:
  • parser (Callable) –

  • delimiter (str) –

__init__(parser, delimiter=',')
Parameters:
  • parser (Callable) –

  • delimiter (str) –

__call__(value)

Call self as a function.

Parameters:

value (str) –

Return type:

List[Any]

class everett.manager.Option(default=NO_VALUE, alternate_keys=None, doc='', parser=<class 'str'>, meta=None)

Settings for a single configuration option.

Use this when creating Everett configuration components.

Example:

from everett.manager import Option

class MyService:
    # Note: The Config class has to be called "Config".
    class Config:
        host = Option(default="localhost")
        port = Option(default="8000", parser=int)
Parameters:
  • default (str | NoValue) – the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return everett.NO_VALUE depending on the value of raise_error

  • alternate_keys (List[str] | None) –

    the list of alternate keys to look up; supports a root: key prefix which will cause this to look at the configuration root rather than the current namespace

    New in version 0.3.

  • doc (str) –

    documentation for this config option

    New in version 0.6.

  • parser (Callable) – the parser for converting this value to a Python object

  • meta (Any) – any meta information that’s tied to this option; useful for noting which options are related in some way or which are secrets that shouldn’t be logged

__init__(default=NO_VALUE, alternate_keys=None, doc='', parser=<class 'str'>, meta=None)
Parameters:
  • default (str | NoValue) – the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return everett.NO_VALUE depending on the value of raise_error

  • alternate_keys (List[str] | None) –

    the list of alternate keys to look up; supports a root: key prefix which will cause this to look at the configuration root rather than the current namespace

    New in version 0.3.

  • doc (str) –

    documentation for this config option

    New in version 0.6.

  • parser (Callable) – the parser for converting this value to a Python object

  • meta (Any | None) – any meta information that’s tied to this option; useful for noting which options are related in some way or which are secrets that shouldn’t be logged

everett.manager.config_override(**cfg)

Allow you to override config for writing tests.

This can be used as a class decorator:

@config_override(FOO="bar", BAZ="bat")
class FooTestClass(object):
    ...

This can be used as a function decorator:

@config_override(FOO="bar")
def test_foo():
    ...

This can also be used as a context manager:

def test_foo():
    with config_override(FOO="bar"):
        ...
Parameters:

cfg (str) –

Return type:

ConfigOverride

everett.manager.get_config_for_class(cls)

Roll up configuration options for this class and parent classes.

This handles subclasses overriding configuration options in parent classes.

Parameters:

cls (Type) – the component class to return configuration options for

Returns:

final dict of configuration options for this class in key -> (option, cls) form

Return type:

Dict[str, Tuple[Option, Type]]

everett.manager.get_runtime_config(config, component, traverse=<function traverse_tree>)

Returns configuration specification and values for a component tree

For example, if you had a tree of components instantiated, you could traverse the tree and log the configuration:

from everett.manager import (
    ConfigManager,
    generate_uppercase_key,
    get_runtime_config,
    Option,
    parse_class,
)

class App:
    class Config:
        debug = Option(default="False", parser=bool)
        reader = Option(parser=parse_class)
        writer = Option(parser=parse_class)

    def __init__(self, config):
        self.config = config.with_options(self)

        # App has a reader and a writer each of which has configuration
        # options
        self.reader = self.config("reader")(config.with_namespace("reader"))
        self.writer = self.config("writer")(config.with_namespace("writer"))

class Reader:
    class Config:
        input_file = Option()

    def __init__(self, config):
        self.config = config.with_options(self)

class Writer:
    class Config:
        output_file = Option()

    def __init__(self, config):
        self.config = config.with_options(self)

cm = ConfigManager.from_dict(
    {
        # This specifies which reader component to use. Because we
        # specified this one, we need to define a READER_INPUT_FILE
        # value.
        "READER": "__main__.Reader",
        "READER_INPUT_FILE": "input.txt",

        # Same thing for the writer component.
        "WRITER": "__main__.Writer",
        "WRITER_OUTPUT_FILE": "output.txt",
    }
)

my_app = App(cm)

# This traverses the component tree starting with my_app and then
# traversing .reader and .writer attributes.
for namespace, key, value, option in get_runtime_config(cm, my_app):
    full_key = generate_uppercase_key(key, namespace)
    print(f"{full_key.upper()}={value or ''}")

# This should print out:
# DEBUG=False
# READER=__main__.Reader
# READER_INPUT_FILE=input.txt
# WRITER=__main__.Writer
# WRITER_OUTPUT_FILE=output.txt
Parameters:
  • config (ConfigManager) – a configuration manager instance

  • component (Any) – a component or tree of components

  • traverse (Callable) – the function for traversing the component tree; see everett.manager.traverse_tree() for signature

Returns:

a list of (namespace, key, value, option) tuples

Return type:

List[Tuple[List[str], str, Any, Option]]

everett.manager.parse_bool(val)

Parse a bool value.

Handles a series of values, but you should probably standardize on “true” and “false”.

>>> from everett.manager import parse_bool
>>> parse_bool("y")
True
>>> parse_bool("FALSE")
False
Parameters:

val (str) –

Return type:

bool

everett.manager.parse_class(val)

Parse a string, imports the module and returns the class.

>>> from everett.manager import parse_class
>>> parse_class("everett.manager.Option")
<class 'everett.manager.Option'>
Parameters:

val (str) –

Return type:

Any

everett.manager.parse_data_size(val)

Parse a string denoting a data size into an int of bytes.

This allows you to parse data sizes with a number and then the metric. Examples:

  • 10b - 10 bytes

  • 100kb - 100 kilobytes = 100 * 1000

  • 40gb - 40 gigabytes = 40 * 1000^3

  • 23gib - 40 gibibytes = 23 * 1024^3

Supported metrics:

  • b - bytes

  • decimal:

    • kb - kilobytes

    • mb - megabytes

    • gb - gigabytes

    • tb - terabytes

    • pb - petabytes

  • binary:

    • kib - kibibytes

    • mib - mebibytes

    • gib - gibibytes

    • tib - tebibytes

    • pib - pebibytes

The metrics are not case sensitive–it supports upper, lower, and mixed case.

>>> from everett.manager import parse_data_size
>>> parse_data_size("40_000_000")
40000000
>>> parse_data_size("40gb")
40000000000
>>> parse_data_size("20KiB")
20480
Parameters:

val (str) –

Return type:

Any

everett.manager.parse_time_period(val)

Parse a string denoting a time period into a number of seconds.

Units:

  • w - week

  • d - day

  • h - hour

  • m - minute

  • s - second

>>> from everett.manager import parse_time_period
>>> parse_time_period("103")
103
>>> parse_time_period("1_000m")
60000
>>> parse_time_period("15m4s")
904
Parameters:

val (str) –

Return type:

Any

everett.manager.parse_env_file(envfile)

Parse the content of an iterable of lines as .env.

Return a dict of config variables.

>>> from everett.manager import parse_env_file
>>> parse_env_file(["DUDE=Abides"])
{'DUDE': 'Abides'}
Parameters:

envfile (Iterable[str]) –

Return type:

Dict

everett.ext.inifile

Holds the ConfigIniEnv environment.

To use this, you must install the optional requirements:

$ pip install 'everett[ini]'
class everett.ext.inifile.ConfigIniEnv(possible_paths)

Source for pulling configuration from INI files.

This requires optional dependencies. You can install them with:

$ pip install 'everett[ini]'

Takes a path or list of possible paths to look for a INI file. It uses the first INI file it can find.

If it finds no INI files in the possible paths, then this configuration source will be a no-op.

This will expand ~ as well as work relative to the current working directory.

This example looks just for the INI file specified in the environment:

from everett.manager import ConfigManager
from everett.ext.inifile import ConfigIniEnv

config = ConfigManager([
    ConfigIniEnv(possible_paths=os.environ.get("FOO_INI"))
])

If there’s no FOO_INI in the environment, then the path will be ignored.

Here’s an example that looks for the INI file specified in the environment variable FOO_INI and failing that will look for .antenna.ini in the user’s home directory:

from everett.manager import ConfigManager
from everett.ext.inifile import ConfigIniEnv

config = ConfigManager([
    ConfigIniEnv(
        possible_paths=[
            os.environ.get("FOO_INI"),
            "~/.antenna.ini"
        ]
    )
])

This example looks for a config/local.ini file which overrides values in a config/base.ini file both are relative to the current working directory:

from everett.manager import ConfigManager
from everett.ext.inifile import ConfigIniEnv

config = ConfigManager([
    ConfigIniEnv(possible_paths="config/local.ini"),
    ConfigIniEnv(possible_paths="config/base.ini")
])

Note how you can have multiple ConfigIniEnv files and this is how you can set Everett up to have values in one INI file override values in another INI file.

INI files must have a “main” section. This is where keys that aren’t in a namespace are placed.

Minimal INI file:

[main]

In the INI file, namespace is a section. So key “user” in namespace “foo” is:

[foo]
user=someval

Everett uses configobj, so it supports nested sections like this:

[main]
foo=bar

[namespace]
foo2=bar2

  [[namespace2]]
  foo3=bar3

Which gives you these:

  • FOO

  • NAMESPACE_FOO2

  • NAMESPACE_NAMESPACE2_FOO3

See more details here: http://configobj.readthedocs.io/en/latest/configobj.html#the-config-file-format

Parameters:

possible_paths (str | List[str]) – either a single string with a file path (e.g. "/etc/project.ini" or a list of strings with file paths

parse_ini_file(path)

Parse ini file at path and return dict.

Parameters:

path (str) –

Return type:

Dict

get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue

everett.ext.yamlfile

Holds the ConfigYamlEnv environment.

To use this, you must install the optional requirements:

$ pip install 'everett[yaml]'
class everett.ext.yamlfile.ConfigYamlEnv(possible_paths)

Source for pulling configuration from YAML files.

This requires optional dependencies. You can install them with:

$ pip install 'everett[yaml]'

Takes a path or list of possible paths to look for a YAML file. It uses the first YAML file it can find.

If it finds no YAML files in the possible paths, then this configuration source will be a no-op.

This will expand ~ as well as work relative to the current working directory.

This example looks just for the YAML file specified in the environment:

from everett.manager import ConfigManager
from everett.ext.yamlfile import ConfigYamlEnv

config = ConfigManager([
    ConfigYamlEnv(os.environ.get('FOO_YAML'))
])

If there’s no FOO_YAML in the environment, then the path will be ignored.

Here’s an example that looks for the YAML file specified in the environment variable FOO_YAML and failing that will look for .antenna.yaml in the user’s home directory:

from everett.manager import ConfigManager
from everett.ext.yamlfile import ConfigYamlEnv

config = ConfigManager([
    ConfigYamlEnv([
        os.environ.get('FOO_YAML'),
        '~/.antenna.yaml'
    ])
])

This example looks for a config/local.yaml file which overrides values in a config/base.yaml file both are relative to the current working directory:

from everett.manager import ConfigManager
from everett.ext.yamlfile import ConfigYamlEnv

config = ConfigManager([
    ConfigYamlEnv('config/local.yaml'),
    ConfigYamlEnv('config/base.yaml')
])

Note how you can have multiple ConfigYamlEnv files. This is how you can set Everett up to have values in one YAML file override values in another YAML file.

Everett looks for keys and values in YAML files. YAML files can be split into multiple documents, but Everett only looks at the first one.

Keys are case-insensitive. You can do namespaces either in the key itself using _ as a separator or as nested mappings.

All values should be double-quoted.

Here’s an example:

foo: "bar"
FOO2: "bar"
namespace_foo: "bar"
namespace:
    namespace2:
        foo: "bar"

Giving you these namespaced keys:

  • FOO

  • FOO2

  • NAMESPACE_FOO

  • NAMESPACE_NAMEPSACE2_FOO

Parameters:

possible_paths (str | List[str]) – either a single string with a file path (e.g. "/etc/project.yaml" or a list of strings with file paths

parse_yaml_file(path)

Parse yaml file at path and return a dict.

Parameters:

path (str) –

Return type:

Dict

get(key, namespace=None)

Retrieve value for key.

Parameters:
  • key (str) –

  • namespace (List[str] | None) –

Return type:

str | NoValue