Author:
Tal Einat , Jelle Zijlstra <jelle.zijlstra at gmail.com>
Discussions-To:
Status:
Final
Type:
Standards Track
Created:
06-Jun-2021
Python-Version:
3.15
Post-History:
Resolution:
Table of Contents
NotGiven = object()MISSING or SentinelEllipsis sentinel valuetyping.Literal in type annotationsUnique placeholder values, commonly known as “sentinel values”, are common in programming. They have many uses, such as for:
Default values for function arguments, for when a value was not given:
def foo(value=None): ...
Return values from functions when something is not found or unavailable:
>>> "abc".find("d") -1
Missing data, such as NULL in relational databases or “N/A” (“not available”) in spreadsheets
Python has the special value None, which is intended to be used as such a sentinel value in most cases. However, sometimes an alternative sentinel value is needed, usually when it needs to be distinct from None since None is a valid value in that context. Such cases are common enough that several idioms for implementing such sentinels have arisen over the years, but uncommon enough that there hasn’t been a clear need for standardization. However, the common implementations, including some in the stdlib, suffer from several significant drawbacks.
This PEP proposes adding a built-in class for defining sentinel values, to be used in the stdlib and made publicly available to all Python code. Sentinels can be defined in Python with the sentinel() built-in class, and in C with the PySentinel_New() C API function.
Note: Changing all existing sentinels in the stdlib to be implemented this way is not deemed necessary, and whether to do so is left to the discretion of the maintainers.
In May 2021, a question was brought up on the python-dev mailing list [1] about how to better implement a sentinel value for traceback.print_exception. The existing implementation used the following common idiom:
_sentinel = object()
However, this object has an uninformative and overly verbose repr, causing the function’s signature to be overly long and hard to read:
>>> help(traceback.print_exception) Help on function print_exception in module traceback:
print_exception(exc, /, value=<object object at 0x000002825DF09650>, tb=<object object at 0x000002825DF09650>, limit=None, file=None, chain=True)
Additionally, two other drawbacks of many existing sentinels were brought up in the discussion:
is failing. Some common sentinel idioms have similar problems after being pickled and unpickled.In the ensuing discussion, Victor Stinner supplied a list of currently used sentinel values in the Python standard library [2]. This showed that the need for sentinels is fairly common, that there are various implementation methods used even within the stdlib, and that many of these suffer from at least one of the three above drawbacks.
The discussion did not lead to any clear consensus on whether a standard implementation method is needed or desirable, whether the drawbacks mentioned are significant, nor which kind of implementation would be good. The author of this PEP created an issue on bugs.python.org (now a GitHub issue [3]) suggesting options for improvement, but that focused on only a single problematic aspect of a few cases, and failed to gather any support.
A poll [4] was created on discuss.python.org to get a clearer sense of the community’s opinions. After nearly two weeks, significant further, discussion, and 39 votes, the poll’s results were not conclusive. 40% had voted for “The status-quo is fine / there’s no need for consistency in this”, but most voters had voted for one or more standardized solutions. Specifically, 37% of the voters chose “Consistent use of a new, dedicated sentinel factory / class / meta-class, also made publicly available in the stdlib”.
With such mixed opinions, this PEP was created to facilitate making a decision on the subject.
While working on this PEP, iterating on various options and implementations and continuing discussions, the author has come to the opinion that a simple, good implementation available in the standard library would be worth having, both for use in the standard library itself and elsewhere.
The criteria guiding the chosen implementation were:
is operator, it should always be considered identical to itself but never to any other object.With so many uses in the Python standard library [2], it would be useful to have an implementation available to the standard library, since the stdlib cannot use implementations of sentinel objects available elsewhere (such as the sentinels [5] or sentinel [6] PyPI packages).
After researching existing idioms and implementations, and going through many different possible implementations, the design below was chosen to meet these criteria while keeping the API and implementation small (see Reference Implementation).
A new built-in callable named sentinel will be added.
>>> MISSING = sentinel('MISSING') >>> MISSING MISSING
sentinel() takes a single positional-only argument, name, which must be a str. Passing a non-string raises TypeError. The name is used as the sentinel’s name and repr.
Sentinel objects have two public attributes:
__name__ is the sentinel’s name.__module__ is the name of the module where sentinel() was called.sentinel may not be subclassed.
Each call to sentinel(name) returns a new sentinel object. If a sentinel is needed in more than one place, it should be assigned to a variable and that same object should be reused explicitly, just as with the common MISSING = object() idiom:
MISSING = sentinel('MISSING')
def read_value(default=MISSING): ...
Checking if a value is such a sentinel should be done using the is operator, as is recommended for None. Equality checks using == will also work as expected, returning True only when the object is compared with itself. Identity checks such as if value is MISSING: should usually be used rather than boolean checks such as if value: or if not value:.
Sentinel objects are “truthy”, i.e. boolean evaluation will result in True. This parallels the default for arbitrary classes, as well as the boolean value of Ellipsis. This is unlike None, which is “falsy”.
Creating a copy of a sentinel object, such as by using copy.copy() or by copy.deepcopy(), will return the same object.
Sentinels importable from their defining module by name preserve their identity when pickled and unpickled, using the standard pickle mechanism for named singletons. When sentinel() creates a sentinel, it records the calling module as the sentinel’s __module__ attribute. Pickling records the sentinel by module and name. Unpickling then imports the module and retrieves the sentinel by name, so the following round trip preserves identity:
MISSING = sentinel('MISSING') assert pickle.loads(pickle.dumps(MISSING)) is MISSING
Sentinels that are not importable by module and name, such as sentinels created in a local scope and not assigned to a matching module global or class attribute, are not picklable.
The repr of a sentinel object is the name passed to sentinel(). No implicit module qualification is added. If a qualified repr is desired, the qualified name should be passed explicitly:
>>> MyClass_NotGiven = sentinel('MyClass.NotGiven') >>> MyClass_NotGiven MyClass.NotGiven
Ordering comparisons are undefined for sentinel objects. Sentinels do not support weakrefs.
To make usage of sentinels clear and simple in typed Python code, we propose to amend the type system with a special case for sentinel objects.
Sentinel objects may be used in type expressions, representing themselves. This is similar to how None is handled in the existing type system. For example:
MISSING = sentinel('MISSING')
def foo(value: int | MISSING = MISSING) -> int: ...
More formally, type checkers should recognize sentinel creations of the form NAME = sentinel('NAME') as creating a new sentinel object. If the name passed to sentinel() does not match the name the object is assigned to, type checkers should emit an error.
Sentinels defined using this syntax may be used in type expressions. They represent a fully static type that has a single member, the sentinel object itself.
Type checkers should support narrowing union types involving sentinels using the is and is not operators:
from typing import assert_type
MISSING = sentinel('MISSING')
def foo(value: int | MISSING) -> None: if value is MISSING: assert_type(value, MISSING) else: assert_type(value, int)
To support usage in type expressions, the runtime implementation of sentinel objects should have the __or__ and __ror__ methods, returning typing.Union objects.
The Typing Council supports this part of the proposal.
Sentinels can also be useful in C extensions. We propose two new C API functions:
PyObject *PySentinel_New(const char *name, const char *module_name) creates a new sentinel object.bool PySentinel_Check(PyObject *obj) checks if an object is a sentinel.C code can use the == operator to check if an object is a specific sentinel.
Adding a new builtin means that code which currently relies on the bare name sentinel raising NameError will instead see the new builtin. This is the usual compatibility consideration for new builtins. Existing local, global, and imported names called sentinel are unaffected. Code that already uses the name sentinel will have to be adapted to use the new builtin and may receive new linter errors from linters that warn about collisions with builtin names.
The normal types of documentation of new builtins and features, namely docstrings, library docs and a section in “What’s New”, should suffice.
This proposal should have no security implications.
A reference implementation is available as a CPython pull request [10]. A previous reference implementation is found in a dedicated GitHub repo [7]. A sketch of the intended behavior follows:
class sentinel: """Unique sentinel values."""
\_\_slots\_\_ \= ("\_\_name\_\_", "\_module\_name")
def \_\_init\_subclass\_\_(cls):
raise TypeError("type 'sentinel' is not an acceptable base type")
def \_\_init\_\_(self, name, /):
if not isinstance(name, str):
raise TypeError("sentinel name must be a string")
self.\_\_name\_\_ \= name
self.\_module\_name \= sys.\_getframemodulename(1)
@property
def \_\_module\_\_(self):
return self.\_module\_name
def \_\_repr\_\_(self):
return self.\_\_name\_\_
def \_\_reduce\_\_(self):
return self.\_\_name\_\_
def \_\_copy\_\_(self):
return self
def \_\_deepcopy\_\_(self, memo):
return self
def \_\_or\_\_(self, other):
return typing.Union\[self, other\]
def \_\_ror\_\_(self, other):
return typing.Union\[other, self\]
A backport exists in the typing-extensions module, though its behavior does not precisely match the current iteration of this PEP.
NotGiven = object()This suffers from all of the drawbacks mentioned in the Rationale section.
MISSING or SentinelSince such a value could be used for various things in various places, one could not always be confident that it would never be a valid value in some use cases. On the other hand, a dedicated and distinct sentinel value can be used with confidence without needing to consider potential edge-cases.
Additionally, it is useful to be able to provide a meaningful name and repr for a sentinel value, specific to the context where it is used.
Finally, this was a very unpopular option in the poll [4], with only 12% of the votes voting for it.
Ellipsis sentinel valueThis is not the original intended use of Ellipsis, though it has become increasingly common to use it to define empty class or function blocks instead of using pass.
Also, similar to a potential new single sentinel value, Ellipsis can’t be as confidently used in all cases, unlike a dedicated, distinct value.
The suggested idiom is:
class NotGivenType(Enum): NotGiven = 'NotGiven' NotGiven = NotGivenType.NotGiven
Besides the excessive repetition, the repr is overly long: <NotGivenType.NotGiven: 'NotGiven'>. A shorter repr can be defined, at the expense of a bit more code and yet more repetition.
Finally, this option was the least popular among the nine options in the poll [4], being the only option to receive no votes.
The suggested idiom is:
@sentinel class NotGivenType: pass NotGiven = NotGivenType()
While this allows for a very simple and clear implementation of the decorator, the idiom is too verbose, repetitive, and difficult to remember.
Since classes are inherently singletons, using a class as a sentinel value makes sense and allows for a simple implementation.
The simplest version of this is:
class NotGiven: pass
To have a clear repr, one would need to use a meta-class:
class NotGiven(metaclass=SentinelMeta): pass
… or a class decorator:
@Sentinel class NotGiven: pass
Using classes this way is unusual and could be confusing. The intention of code would be hard to understand without comments. It would also cause such sentinels to have some unexpected and undesirable behavior, such as being callable.
Most common existing idioms have significant drawbacks. So far, no idiom has been found that is clear and concise while avoiding these drawbacks.
Also, in the poll [4] on this subject, the options for recommending an idiom were unpopular, with the highest-voted option being voted for by only 25% of the voters.
Earlier drafts proposed adding a Sentinel class to a new sentinels or sentinellib module. However, adding a new module for a single public callable is unnecessary, and using a module makes the feature less convenient than the existing object() idiom. The Steering Council also specifically encouraged making the feature a builtin so that it is at least as easy to use as object().
Using the name sentinels would also conflict with an existing, actively used PyPI package. While other module names are possible, making the feature a builtin avoids the naming problem entirely.
Earlier drafts proposed making sentinel names unique within each module. Under that design, repeated calls such as sentinel("MISSING") from the same module would return the same object, using a process-global registry keyed by module name and sentinel name.
This was rejected because the behavior is too implicit. Code that needs a shared sentinel can define one explicitly and reuse it by name, just as code already does with MISSING = object(). Code in a local scope may also want a fresh sentinel for each call or iteration, and repeated calls to sentinel(name) should behave like repeated calls to object() by creating distinct objects.
Removing the registry also keeps the implementation and mental model simpler: sentinel(name) creates a new unique object whose repr is name.
Earlier drafts proposed an optional module_name argument to support the registry-based design.
With the registry removed, a public module_name argument is no longer needed for the core proposal. The implementation still records the calling module internally, as TypeVar and similar helpers do, so that pickle can serialize importable sentinels by module and name. This internal module name does not affect the sentinel’s repr. If users want a repr that includes a module or class name, they can include it in the single name argument explicitly, e.g. sentinel("mymodule.MISSING").
This was desirable to allow using this for existing sentinel values without changing their repr. However, this was eventually dropped as it wasn’t considered worth the added complexity.
Discussions considered allowing sentinels to be explicitly truthy, falsy, or not convertible to bool. Some existing third-party sentinels expose falsy behavior as part of their public API, and several participants argued that raising in boolean contexts would better enforce identity checks.
This PEP keeps the initial proposal simpler by giving sentinels the default truthy behavior of ordinary objects and by recommending identity checks. Custom boolean behavior may be considered later if the added API and typing complexity is judged worthwhile.
typing.Literal in type annotationsThis was suggested by several people in discussions and is what this PEP first went with. However, it was pointed out that this would cause potential confusion, due to e.g. Literal["MISSING"] referring to the string value "MISSING" rather than being a forward-reference to a sentinel value MISSING. Using the bare name was also suggested often in discussions. This follows the precedent and well-known pattern set by None, and has the advantages of not requiring an import and being much shorter.
For sentinels defined in a class scope, to avoid potential name clashes, or when a qualified repr would be clearer, one should pass the desired qualified name explicitly. For example:
>>> class MyClass: ... NotGiven = sentinel('MyClass.NotGiven') >>> MyClass.NotGiven MyClass.NotGiven
Creating sentinels in a function or method is allowed. Each call to sentinel() creates a distinct object, so a sentinel created in a local scope behaves like one created by calling object() in that scope.
The boolean value of NotImplemented is True, but using this is deprecated since Python 3.9 (doing so generates a deprecation warning.) This deprecation is due to issues specific to NotImplemented, as described in bpo-35712 [8].
To define multiple, related sentinel values, possibly with a defined ordering among them, one should instead use Enum or something similar.
There was a discussion on the typing-sig mailing list [9] about the typing for these sentinels, where different options were discussed.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.