API
This is the API of functions and classes in Everett.
Configuration things:
Utility functions:
Testing utility functions:
Configuration environments:
Errors:
everett.InvalidValueError
Parsers:
everett
Everett is a Python library for configuration.
- everett.NO_VALUE = NO_VALUE
Singleton indicating a non-value.
- exception everett.ConfigurationError
Configuration error base class.
- exception everett.DetailedConfigurationError(msg, namespace, key, parser)
Base class for configuration errors that have a msg, namespace, key, and parser.
- Parameters:
msg (str) –
namespace (List[str] | None) –
key (str) –
parser (Callable) –
- exception everett.InvalidKeyError
Error that indicates the key is not valid for this component.
- exception everett.ConfigurationMissingError(msg, namespace, key, parser)
Error that indicates that required configuration is missing.
- Parameters:
msg (str) –
namespace (List[str] | None) –
key (str) –
parser (Callable) –
everett.manager
Contains configuration infrastructure.
This module contains the configuration classes and functions for deriving configuration values from specified sources in the order specified.
- class everett.manager.ConfigDictEnv(cfg)
Source for pulling configuration out of a dict.
This is handy for testing. You might also use it if you wanted to move all your defaults values into one centralized place.
Keys are prefixed by namespaces and the whole thing is uppercased.
For example, namespace “bar” for key “foo” becomes
BAR_FOO
in the dict.For example:
from everett.manager import ConfigDictEnv, ConfigManager config = ConfigManager([ ConfigDictEnv({ "FOO_BAR": "someval", "BAT": "1", }) ])
Keys are not case sensitive. This also works:
from everett.manager import ConfigDictEnv, ConfigManager config = ConfigManager([ ConfigDictEnv({ "foo_bar": "someval", "bat": "1", }) ]) print config("foo_bar") print config("FOO_BAR") print config.with_namespace("foo")("bar")
Also,
ConfigManager
has a convenience classmethod for creating aConfigManager
with just a dict environment:from everett.manager import ConfigManager config = ConfigManager.from_dict({ "FOO_BAR": "bat" })
Changed in version 0.3: Keys are no longer case-sensitive.
- Parameters:
cfg (Dict) –
- __init__(cfg)
- Parameters:
cfg (Dict) –
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue
- class everett.manager.ConfigEnvFileEnv(possible_paths)
Source for pulling configuration out of
.env
files.This source lets you specify configuration in an .env file. This is useful for local development when in production you use values in environment variables.
Keys are prefixed by namespaces and the whole thing is uppercased.
For example, key “foo” will be
FOO
in the file.For example, namespace “bar” for key “foo” becomes
BAR_FOO
in the file.Key and namespace can consist of alphanumeric characters and
_
.To use, instantiate and toss in the source list:
from everett.manager import ConfigEnvFileEnv, ConfigManager config = ConfigManager([ ConfigEnvFileEnv('.env') ])
For multiple paths:
from everett.manager import ConfigEnvFileEnv, ConfigManager config = ConfigManager([ ConfigEnvFileEnv([ '.env', 'config/prod.env' ]) ])
Here’s an example .env file:
DEBUG=true # secrets SECRET_KEY=ou812 # database setup DB_HOST=localhost DB_PORT=5432 # CSP reporting CSP_SCRIPT_SRC="'self' www.googletagmanager.com"
- Parameters:
possible_paths (str | List[str]) –
- __init__(possible_paths)
- Parameters:
possible_paths (str | List[str]) –
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue
- class everett.manager.ConfigManager(environments, doc='', msg_builder=<function build_msg>, with_override=True)
Manage multiple configuration environment layers.
Instantiate a ConfigManager.
- Parameters:
environments (List[Any]) – list of configuration sources to look through in the order they should be looked through
doc (str) –
help text printed to users when they encounter configuration errors
New in version 0.6.
msg_builder (Callable) –
function that takes arguments and builds an exception message intended to be printed or conveyed to the user
For example:
def build_msg(namespace, key, parser, msg="", option_doc="", config_doc=""): full_key = namespace or [] full_key = "_".join(full_key + [key]).upper() return ( f"{full_key} requires a value parseable by {qualname(parser)}\n" + option_doc + "\n" + config_doc + "\n" )
with_override (bool) – whether or not to insert the special override environment used for testing as the first environment in the list of sources
- __init__(environments, doc='', msg_builder=<function build_msg>, with_override=True)
Instantiate a ConfigManager.
- Parameters:
environments (List[Any]) – list of configuration sources to look through in the order they should be looked through
doc (str) –
help text printed to users when they encounter configuration errors
New in version 0.6.
msg_builder (Callable) –
function that takes arguments and builds an exception message intended to be printed or conveyed to the user
For example:
def build_msg(namespace, key, parser, msg="", option_doc="", config_doc=""): full_key = namespace or [] full_key = "_".join(full_key + [key]).upper() return ( f"{full_key} requires a value parseable by {qualname(parser)}\n" + option_doc + "\n" + config_doc + "\n" )
with_override (bool) – whether or not to insert the special override environment used for testing as the first environment in the list of sources
- classmethod basic_config(env_file='.env', doc='')
Return a basic ConfigManager.
This sets up a ConfigManager that will look for configuration in this order:
environment
specified
env_file
defaulting to.env
This is for a fast one-line opinionated setup.
Example:
from everett.manager import ConfigManager config = ConfigManager.basic_config()
This is shorthand for:
config = ConfigManager( environments=[ ConfigOSEnv(), ConfigEnvFileEnv(['.env']) ] )
- Parameters:
env_file (str) – the name of the env file to use
doc (str) – help text printed to users when they encounter configuration errors
- Returns:
- Return type:
- classmethod from_dict(dict_config)
Create a ConfigManager with specified configuration as a Python dict.
This is shorthand for:
config = ConfigManager([ConfigDictEnv(dict_config)])
This is handy for writing tests for the app you’re using Everett in.
- Parameters:
dict_config (Dict) – Python dict holding the configuration for this manager
- Returns:
ConfigManager with specified configuration
- Return type:
New in version 0.3.
- get_bound_component()
Retrieve the bound component for this config object.
- Returns:
component or None
- Return type:
Any
- get_namespace()
Retrieve the complete namespace for this config object.
- Returns:
namespace as a list of strings
- Return type:
List[str]
- with_namespace(namespace)
Apply a namespace to this configuration.
Namespaces accumulate as you add them.
- Parameters:
namepace – namespace as a string or list of strings
namespace (List[str] | str) –
- Returns:
a clone of the ConfigManager instance with the namespace applied
- Return type:
- with_options(component)
Apply options component options to this configuration.
- Parameters:
component (Any) – the instance or class with a Config to bind this ConfigManager to
- Returns:
a clone of the ConfigManager instance bound to specified component
- Return type:
- __call__(key, namespace=None, default=NO_VALUE, default_if_empty=True, alternate_keys=None, doc='', parser=<class 'str'>, raise_error=True, raw_value=False)
Return a parsed value from the environment.
- Parameters:
key (str) – the key to look up
namespace (List[str] | str | None) – the namespace for the key–different environments use this differently
default (str | NoValue) –
the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return
everett.NO_VALUE
depending on the value ofraise_error
If this ConfigManager is bound to a component, the default will be the default of the option in the bound component configuration.
default_if_empty (bool) – if True, treat empty string values as a non-value and return the specified default
alternate_keys (List[str] | None) –
the list of alternate keys to look up; supports a
root:
key prefix which will cause this to look at the configuration root rather than the current namespaceIf this ConfigManager is bound to a component, the alternate_keys will be the alternate_keys of the option in the bound component configuration.
New in version 0.3.
doc (str) –
documentation for this config option
If this ConfigManager is bound to a component, the doc will be the doc of the option in the bound component configuration.
New in version 0.6.
parser (Callable) –
the parser for converting this value to a Python object
If this ConfigManager is bound to a component, the parser will be the parser of the option in the bound component configuration.
raise_error (bool) – True if you want a lack of value to raise a
everett.ConfigurationError
raw_value (bool) – True if you want the raw unparsed value, False otherwise
- Raises:
everett.ConfigurationMissingError – if the required bit of configuration is missing from all the environments
everett.InvalidKeyError – if the configuration key doesn’t exist for that component
everett.InvalidValueError – if the configuration value is invalid in some way (not an integer, not a bool, etc)
- Return type:
Any
Note
The default value should always be a string that is parseable by the parser. This simplifies thinking about values since all values are strings that are parsed by the parser rather than default values do one thing and non-default values doa nother. Further, it simplifies documentation for the user since the default value is an example value.
The parser can be any callable that takes a string value and returns a parsed value.
- raise_configuration_error(msg)
Convenience function for raising configuration errors.
This is helpful for situations where you need to do additional checking of configuration values and need to raise a configuration error for the user that includes the configuration documentation.
For example:
from everett.manager import ConfigManager config = ConfigManager.basic_config() host = config("host") port = config("port") if host is None or port is None: config.raise_configuration_error( "Both HOST and PORT must be specified." )
- Parameters:
msg (str) – the configuration error message
- Return type:
None
- class everett.manager.ConfigOSEnv
Source for pulling configuration out of the environment.
This source lets you specify configuration in the environment. This is useful for infrastructure related configuration like usernames and ports and secret configuration like passwords.
Keys are prefixed by namespaces and the whole thing is uppercased.
For example, key “foo” will be
FOO
in the environment.For example, namespace “bar” for key “foo” becomes
BAR_FOO
in the environment.Key and namespace can consist of alphanumeric characters and
_
.Note
Unlike other config environments, this one is case sensitive in that keys defined in the environment must be all uppercase.
For example, these are good:
FOO=bar FOO_BAR=bar FOO_BAR1=bar
This is bad:
foo=bar
To use, instantiate and toss in the source list:
from everett.manager import ConfigOSEnv, ConfigManager config = ConfigManager([ ConfigOSEnv() ])
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue
- class everett.manager.ConfigObjEnv(obj, force_lower=True)
Source for pulling configuration values out of a Python object.
This is handy for a few weird situations. For example, you can use this to “bridge” Everett configuration with command line arguments. The argparse Namespace works fine here.
Namespace (the Everett one–not the argparse one) is prefixed. So key “foo” in namespace “bar” is “foo_bar”.
For example:
import argparse from everett.manager import ConfigObjEnv, ConfigManager parser = argparse.ArgumentParser() parser.add_argument( "--debug", help="to debug or not to debug" ) parsed_vals = parser.parse_known_args()[0] config = ConfigManager([ ConfigObjEnv(parsed_vals) ]) print config("debug", parser=bool)
Keys are not case-sensitive–everything is converted to lowercase before pulling it from the object.
Note
ConfigObjEnv has nothing to do with the library configobj.
New in version 0.6.
- Parameters:
obj (Any) –
force_lower (bool) –
- __init__(obj, force_lower=True)
- Parameters:
obj (Any) –
force_lower (bool) –
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue
- class everett.manager.ListOf(parser, delimiter=',')
Parse a comma-separated list of things.
After delimiting items, this strips the whitespace at the beginning and end of each string. Then it passes each string into the parser to get the final value.
>>> from everett.manager import ListOf >>> ListOf(str)('') [] >>> ListOf(str)('a,b,c,d') ['a', 'b', 'c', 'd'] >>> ListOf(int)('1,2,3,4') [1, 2, 3, 4] >>> ListOf(str)('1, 2 ,3,4') ['1', '2', '3', '4']
Note: This doesn’t handle quotes or backslashes or any complicated string parsing.
For example:
>>> ListOf(str)('"a,b",c,d') ['"a', 'b"', 'c', 'd']
- Parameters:
parser (Callable) –
delimiter (str) –
- __init__(parser, delimiter=',')
- Parameters:
parser (Callable) –
delimiter (str) –
- __call__(value)
Call self as a function.
- Parameters:
value (str) –
- Return type:
List[Any]
- class everett.manager.Option(default=NO_VALUE, alternate_keys=None, doc='', parser=<class 'str'>, meta=None)
Settings for a single configuration option.
Use this when creating Everett configuration components.
Example:
from everett.manager import Option class MyService: # Note: The Config class has to be called "Config". class Config: host = Option(default="localhost") port = Option(default="8000", parser=int)
- Parameters:
default (str | NoValue) – the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return
everett.NO_VALUE
depending on the value ofraise_error
alternate_keys (List[str] | None) –
the list of alternate keys to look up; supports a
root:
key prefix which will cause this to look at the configuration root rather than the current namespaceNew in version 0.3.
doc (str) –
documentation for this config option
New in version 0.6.
parser (Callable) – the parser for converting this value to a Python object
meta (Any) – any meta information that’s tied to this option; useful for noting which options are related in some way or which are secrets that shouldn’t be logged
- __init__(default=NO_VALUE, alternate_keys=None, doc='', parser=<class 'str'>, meta=None)
- Parameters:
default (str | NoValue) – the default value (if any); this must be a string that is parseable by the specified parser; if no default is provided, this will raise an error or return
everett.NO_VALUE
depending on the value ofraise_error
alternate_keys (List[str] | None) –
the list of alternate keys to look up; supports a
root:
key prefix which will cause this to look at the configuration root rather than the current namespaceNew in version 0.3.
doc (str) –
documentation for this config option
New in version 0.6.
parser (Callable) – the parser for converting this value to a Python object
meta (Any | None) – any meta information that’s tied to this option; useful for noting which options are related in some way or which are secrets that shouldn’t be logged
- everett.manager.config_override(**cfg)
Allow you to override config for writing tests.
This can be used as a class decorator:
@config_override(FOO="bar", BAZ="bat") class FooTestClass(object): ...
This can be used as a function decorator:
@config_override(FOO="bar") def test_foo(): ...
This can also be used as a context manager:
def test_foo(): with config_override(FOO="bar"): ...
- Parameters:
cfg (str) –
- Return type:
ConfigOverride
- everett.manager.get_config_for_class(cls)
Roll up configuration options for this class and parent classes.
This handles subclasses overriding configuration options in parent classes.
- Parameters:
cls (Type) – the component class to return configuration options for
- Returns:
final dict of configuration options for this class in
key -> (option, cls)
form- Return type:
Dict[str, Tuple[Option, Type]]
- everett.manager.get_runtime_config(config, component, traverse=<function traverse_tree>)
Returns configuration specification and values for a component tree
For example, if you had a tree of components instantiated, you could traverse the tree and log the configuration:
from everett.manager import ( ConfigManager, generate_uppercase_key, get_runtime_config, Option, parse_class, ) class App: class Config: debug = Option(default="False", parser=bool) reader = Option(parser=parse_class) writer = Option(parser=parse_class) def __init__(self, config): self.config = config.with_options(self) # App has a reader and a writer each of which has configuration # options self.reader = self.config("reader")(config.with_namespace("reader")) self.writer = self.config("writer")(config.with_namespace("writer")) class Reader: class Config: input_file = Option() def __init__(self, config): self.config = config.with_options(self) class Writer: class Config: output_file = Option() def __init__(self, config): self.config = config.with_options(self) cm = ConfigManager.from_dict( { # This specifies which reader component to use. Because we # specified this one, we need to define a READER_INPUT_FILE # value. "READER": "__main__.Reader", "READER_INPUT_FILE": "input.txt", # Same thing for the writer component. "WRITER": "__main__.Writer", "WRITER_OUTPUT_FILE": "output.txt", } ) my_app = App(cm) # This traverses the component tree starting with my_app and then # traversing .reader and .writer attributes. for namespace, key, value, option in get_runtime_config(cm, my_app): full_key = generate_uppercase_key(key, namespace) print(f"{full_key.upper()}={value or ''}") # This should print out: # DEBUG=False # READER=__main__.Reader # READER_INPUT_FILE=input.txt # WRITER=__main__.Writer # WRITER_OUTPUT_FILE=output.txt
- Parameters:
config (ConfigManager) – a configuration manager instance
component (Any) – a component or tree of components
traverse (Callable) – the function for traversing the component tree; see
everett.manager.traverse_tree()
for signature
- Returns:
a list of (namespace, key, value, option) tuples
- Return type:
List[Tuple[List[str], str, Any, Option]]
- everett.manager.parse_bool(val)
Parse a bool value.
Handles a series of values, but you should probably standardize on “true” and “false”.
>>> from everett.manager import parse_bool >>> parse_bool("y") True >>> parse_bool("FALSE") False
- Parameters:
val (str) –
- Return type:
bool
- everett.manager.parse_class(val)
Parse a string, imports the module and returns the class.
>>> from everett.manager import parse_class >>> parse_class("everett.manager.Option") <class 'everett.manager.Option'>
- Parameters:
val (str) –
- Return type:
Any
- everett.manager.parse_data_size(val)
Parse a string denoting a data size into an int of bytes.
This allows you to parse data sizes with a number and then the metric. Examples:
10b - 10 bytes
100kb - 100 kilobytes = 100 * 1000
40gb - 40 gigabytes = 40 * 1000^3
23gib - 40 gibibytes = 23 * 1024^3
Supported metrics:
b - bytes
decimal:
kb - kilobytes
mb - megabytes
gb - gigabytes
tb - terabytes
pb - petabytes
binary:
kib - kibibytes
mib - mebibytes
gib - gibibytes
tib - tebibytes
pib - pebibytes
The metrics are not case sensitive–it supports upper, lower, and mixed case.
>>> from everett.manager import parse_data_size >>> parse_data_size("40_000_000") 40000000 >>> parse_data_size("40gb") 40000000000 >>> parse_data_size("20KiB") 20480
- Parameters:
val (str) –
- Return type:
Any
- everett.manager.parse_time_period(val)
Parse a string denoting a time period into a number of seconds.
Units:
w - week
d - day
h - hour
m - minute
s - second
>>> from everett.manager import parse_time_period >>> parse_time_period("103") 103 >>> parse_time_period("1_000m") 60000 >>> parse_time_period("15m4s") 904
- Parameters:
val (str) –
- Return type:
Any
- everett.manager.parse_env_file(envfile)
Parse the content of an iterable of lines as
.env
.Return a dict of config variables.
>>> from everett.manager import parse_env_file >>> parse_env_file(["DUDE=Abides"]) {'DUDE': 'Abides'}
- Parameters:
envfile (Iterable[str]) –
- Return type:
Dict
everett.ext.inifile
Holds the ConfigIniEnv environment.
To use this, you must install the optional requirements:
$ pip install 'everett[ini]'
- class everett.ext.inifile.ConfigIniEnv(possible_paths)
Source for pulling configuration from INI files.
This requires optional dependencies. You can install them with:
$ pip install 'everett[ini]'
Takes a path or list of possible paths to look for a INI file. It uses the first INI file it can find.
If it finds no INI files in the possible paths, then this configuration source will be a no-op.
This will expand
~
as well as work relative to the current working directory.This example looks just for the INI file specified in the environment:
from everett.manager import ConfigManager from everett.ext.inifile import ConfigIniEnv config = ConfigManager([ ConfigIniEnv(possible_paths=os.environ.get("FOO_INI")) ])
If there’s no
FOO_INI
in the environment, then the path will be ignored.Here’s an example that looks for the INI file specified in the environment variable
FOO_INI
and failing that will look for.antenna.ini
in the user’s home directory:from everett.manager import ConfigManager from everett.ext.inifile import ConfigIniEnv config = ConfigManager([ ConfigIniEnv( possible_paths=[ os.environ.get("FOO_INI"), "~/.antenna.ini" ] ) ])
This example looks for a
config/local.ini
file which overrides values in aconfig/base.ini
file both are relative to the current working directory:from everett.manager import ConfigManager from everett.ext.inifile import ConfigIniEnv config = ConfigManager([ ConfigIniEnv(possible_paths="config/local.ini"), ConfigIniEnv(possible_paths="config/base.ini") ])
Note how you can have multiple
ConfigIniEnv
files and this is how you can set Everett up to have values in one INI file override values in another INI file.INI files must have a “main” section. This is where keys that aren’t in a namespace are placed.
Minimal INI file:
[main]
In the INI file, namespace is a section. So key “user” in namespace “foo” is:
[foo] user=someval
Everett uses configobj, so it supports nested sections like this:
[main] foo=bar [namespace] foo2=bar2 [[namespace2]] foo3=bar3
Which gives you these:
FOO
NAMESPACE_FOO2
NAMESPACE_NAMESPACE2_FOO3
See more details here: http://configobj.readthedocs.io/en/latest/configobj.html#the-config-file-format
- Parameters:
possible_paths (str | List[str]) – either a single string with a file path (e.g.
"/etc/project.ini"
or a list of strings with file paths
- parse_ini_file(path)
Parse ini file at
path
and return dict.- Parameters:
path (str) –
- Return type:
Dict
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue
everett.ext.yamlfile
Holds the ConfigYamlEnv environment.
To use this, you must install the optional requirements:
$ pip install 'everett[yaml]'
- class everett.ext.yamlfile.ConfigYamlEnv(possible_paths)
Source for pulling configuration from YAML files.
This requires optional dependencies. You can install them with:
$ pip install 'everett[yaml]'
Takes a path or list of possible paths to look for a YAML file. It uses the first YAML file it can find.
If it finds no YAML files in the possible paths, then this configuration source will be a no-op.
This will expand
~
as well as work relative to the current working directory.This example looks just for the YAML file specified in the environment:
from everett.manager import ConfigManager from everett.ext.yamlfile import ConfigYamlEnv config = ConfigManager([ ConfigYamlEnv(os.environ.get('FOO_YAML')) ])
If there’s no
FOO_YAML
in the environment, then the path will be ignored.Here’s an example that looks for the YAML file specified in the environment variable
FOO_YAML
and failing that will look for.antenna.yaml
in the user’s home directory:from everett.manager import ConfigManager from everett.ext.yamlfile import ConfigYamlEnv config = ConfigManager([ ConfigYamlEnv([ os.environ.get('FOO_YAML'), '~/.antenna.yaml' ]) ])
This example looks for a
config/local.yaml
file which overrides values in aconfig/base.yaml
file both are relative to the current working directory:from everett.manager import ConfigManager from everett.ext.yamlfile import ConfigYamlEnv config = ConfigManager([ ConfigYamlEnv('config/local.yaml'), ConfigYamlEnv('config/base.yaml') ])
Note how you can have multiple
ConfigYamlEnv
files. This is how you can set Everett up to have values in one YAML file override values in another YAML file.Everett looks for keys and values in YAML files. YAML files can be split into multiple documents, but Everett only looks at the first one.
Keys are case-insensitive. You can do namespaces either in the key itself using
_
as a separator or as nested mappings.All values should be double-quoted.
Here’s an example:
foo: "bar" FOO2: "bar" namespace_foo: "bar" namespace: namespace2: foo: "bar"
Giving you these namespaced keys:
FOO
FOO2
NAMESPACE_FOO
NAMESPACE_NAMEPSACE2_FOO
- Parameters:
possible_paths (str | List[str]) – either a single string with a file path (e.g.
"/etc/project.yaml"
or a list of strings with file paths
- parse_yaml_file(path)
Parse yaml file at
path
and return a dict.- Parameters:
path (str) –
- Return type:
Dict
- get(key, namespace=None)
Retrieve value for key.
- Parameters:
key (str) –
namespace (List[str] | None) –
- Return type:
str | NoValue