diff options
Diffstat (limited to 'Doc/reference')
| -rw-r--r-- | Doc/reference/compound_stmts.rst | 26 | ||||
| -rw-r--r-- | Doc/reference/datamodel.rst | 183 | ||||
| -rw-r--r-- | Doc/reference/expressions.rst | 166 | ||||
| -rw-r--r-- | Doc/reference/import.rst | 37 | ||||
| -rw-r--r-- | Doc/reference/lexical_analysis.rst | 163 | ||||
| -rw-r--r-- | Doc/reference/simple_stmts.rst | 56 |
6 files changed, 560 insertions, 71 deletions
diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 88b94eaa95..4b425a4820 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -471,10 +471,10 @@ A function definition defines a user-defined function object (see section decorators: `decorator`+ decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE dotted_name: `identifier` ("." `identifier`)* - parameter_list: (`defparameter` ",")* - : | "*" [`parameter`] ("," `defparameter`)* ["," "**" `parameter`] - : | "**" `parameter` - : | `defparameter` [","] ) + parameter_list: `defparameter` ("," `defparameter`)* ["," [`parameter_list_starargs`]] + : | `parameter_list_starargs` + parameter_list_starargs: "*" [`parameter`] ("," `defparameter`)* ["," ["**" `parameter` [","]]] + : | "**" `parameter` [","] parameter: `identifier` [":" `expression`] defparameter: `parameter` ["=" `expression`] funcname: `identifier` @@ -546,11 +546,12 @@ Function call semantics are described in more detail in section :ref:`calls`. A function call always assigns values to all parameters mentioned in the parameter list, either from position arguments, from keyword arguments, or from default values. If the form "``*identifier``" is present, it is initialized to a tuple -receiving any excess positional parameters, defaulting to the empty tuple. If -the form "``**identifier``" is present, it is initialized to a new dictionary -receiving any excess keyword arguments, defaulting to a new empty dictionary. -Parameters after "``*``" or "``*identifier``" are keyword-only parameters and -may only be passed used keyword arguments. +receiving any excess positional parameters, defaulting to the empty tuple. +If the form "``**identifier``" is present, it is initialized to a new +ordered mapping receiving any excess keyword arguments, defaulting to a +new empty mapping of the same type. Parameters after "``*``" or +"``*identifier``" are keyword-only parameters and may only be passed +used keyword arguments. .. index:: pair: function; annotations @@ -632,6 +633,11 @@ list for the base classes and the saved local namespace for the attribute dictionary. The class name is bound to this class object in the original local namespace. +The order in which attributes are defined in the class body is preserved +in the new class's ``__dict__``. Note that this is reliable only right +after the class is created and only for classes that were defined using +the definition syntax. + Class creation can be customized heavily using :ref:`metaclasses <metaclasses>`. Classes can also be decorated: just like when decorating functions, :: @@ -691,7 +697,7 @@ coroutine bodies. Functions defined with ``async def`` syntax are always coroutine functions, even if they do not contain ``await`` or ``async`` keywords. -It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in +It is a :exc:`SyntaxError` to use ``yield from`` expressions in ``async def`` coroutines. An example of a coroutine function:: diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index cd28c147b2..095a2380b3 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -627,6 +627,25 @@ Callable types as well as :keyword:`async with` and :keyword:`async for` statements. See also the :ref:`coroutine-objects` section. + Asynchronous generator functions + .. index:: + single: asynchronous generator; function + single: asynchronous generator; asynchronous iterator + + A function or method which is defined using :keyword:`async def` and + which uses the :keyword:`yield` statement is called a + :dfn:`asynchronous generator function`. Such a function, when called, + returns an asynchronous iterator object which can be used in an + :keyword:`async for` statement to execute the body of the function. + + Calling the asynchronous iterator's :meth:`aiterator.__anext__` method + will return an :term:`awaitable` which when awaited + will execute until it provides a value using the :keyword:`yield` + expression. When the function executes an empty :keyword:`return` + statement or falls off the end, a :exc:`StopAsyncIteration` exception + is raised and the asynchronous iterator will have reached the end of + the set of values to be yielded. + Built-in functions .. index:: object: built-in function @@ -686,33 +705,36 @@ Modules Attribute assignment updates the module's namespace dictionary, e.g., ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``. - .. index:: single: __dict__ (module attribute) - - Special read-only attribute: :attr:`~object.__dict__` is the module's namespace as a - dictionary object. - - .. impl-detail:: - - Because of the way CPython clears module dictionaries, the module - dictionary will be cleared when the module falls out of scope even if the - dictionary still has live references. To avoid this, copy the dictionary - or keep the module around while using its dictionary directly. - .. index:: single: __name__ (module attribute) single: __doc__ (module attribute) single: __file__ (module attribute) + single: __annotations__ (module attribute) pair: module; namespace Predefined (writable) attributes: :attr:`__name__` is the module's name; :attr:`__doc__` is the module's documentation string, or ``None`` if - unavailable; :attr:`__file__` is the pathname of the file from which the + unavailable; :attr:`__annotations__` (optional) is a dictionary containing + :term:`variable annotations <variable annotation>` collected during module + body execution; :attr:`__file__` is the pathname of the file from which the module was loaded, if it was loaded from a file. The :attr:`__file__` attribute may be missing for certain types of modules, such as C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file. + .. index:: single: __dict__ (module attribute) + + Special read-only attribute: :attr:`~object.__dict__` is the module's + namespace as a dictionary object. + + .. impl-detail:: + + Because of the way CPython clears module dictionaries, the module + dictionary will be cleared when the module falls out of scope even if the + dictionary still has live references. To avoid this, copy the dictionary + or keep the module around while using its dictionary directly. + Custom classes Custom class types are typically created by class definitions (see section :ref:`class`). A class has a namespace implemented by a dictionary object. @@ -761,13 +783,16 @@ Custom classes single: __dict__ (class attribute) single: __bases__ (class attribute) single: __doc__ (class attribute) + single: __annotations__ (class attribute) Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is the module name in which the class was defined; :attr:`~object.__dict__` is the dictionary containing the class's namespace; :attr:`~class.__bases__` is a tuple containing the base classes, in the order of their occurrence in the - base class list; :attr:`__doc__` is the class's documentation string, or - ``None`` if undefined. + base class list; :attr:`__doc__` is the class's documentation string, + or ``None`` if undefined; :attr:`__annotations__` (optional) is a dictionary + containing :term:`variable annotations <variable annotation>` collected during + class body execution. Class instances .. index:: @@ -1063,6 +1088,12 @@ to ``type(x).__getitem__(x, i)``. Except where mentioned, attempts to execute a operation raise an exception when no appropriate method is defined (typically :exc:`AttributeError` or :exc:`TypeError`). +Setting a special method to ``None`` indicates that the corresponding +operation is not available. For example, if a class sets +:meth:`__iter__` to ``None``, the class is not iterable, so calling +:func:`iter` on its instances will raise a :exc:`TypeError` (without +falling back to :meth:`__getitem__`). [#]_ + When implementing a class that emulates any built-in type, it is important that the emulation only be implemented to the degree that it makes sense for the object being modelled. For example, some sequences may work well with retrieval @@ -1233,8 +1264,9 @@ Basic customization .. method:: object.__format__(self, format_spec) - Called by the :func:`format` built-in function (and by extension, the - :meth:`str.format` method of class :class:`str`) to produce a "formatted" + Called by the :func:`format` built-in function, + and by extension, evaluation of :ref:`formatted string literals + <f-strings>` and the :meth:`str.format` method, to produce a "formatted" string representation of an object. The ``format_spec`` argument is a string that contains a description of the formatting options desired. The interpretation of the ``format_spec`` argument is up to the type @@ -1494,6 +1526,14 @@ class' :attr:`~object.__dict__`. Called to delete the attribute on an instance *instance* of the owner class. +.. method:: object.__set_name__(self, owner, name) + + Called at the time the owning class *owner* is created. The + descriptor has been assigned to *name*. + + .. versionadded:: 3.6 + + The attribute :attr:`__objclass__` is interpreted by the :mod:`inspect` module as specifying the class where this object was defined (setting this appropriately can assist in runtime introspection of dynamic class attributes). @@ -1631,11 +1671,60 @@ Notes on using *__slots__* * *__class__* assignment works only if both classes have the same *__slots__*. -.. _metaclasses: +.. _class-customization: Customizing class creation -------------------------- +Whenever a class inherits from another class, *__init_subclass__* is +called on that class. This way, it is possible to write classes which +change the behavior of subclasses. This is closely related to class +decorators, but where class decorators only affect the specific class they're +applied to, ``__init_subclass__`` solely applies to future subclasses of the +class defining the method. + +.. classmethod:: object.__init_subclass__(cls) + + This method is called whenever the containing class is subclassed. + *cls* is then the new subclass. If defined as a normal instance method, + this method is implicitly converted to a class method. + + Keyword arguments which are given to a new class are passed to + the parent's class ``__init_subclass__``. For compatibility with + other classes using ``__init_subclass__``, one should take out the + needed keyword arguments and pass the others over to the base + class, as in:: + + class Philosopher: + def __init_subclass__(cls, default_name, **kwargs): + super().__init_subclass__(**kwargs) + cls.default_name = default_name + + class AustralianPhilosopher(Philosopher, default_name="Bruce"): + pass + + The default implementation ``object.__init_subclass__`` does + nothing, but raises an error if it is called with any arguments. + + .. note:: + + The metaclass hint ``metaclass`` is consumed by the rest of the type + machinery, and is never passed to ``__init_subclass__`` implementations. + The actual metaclass (rather than the explicit hint) can be accessed as + ``type(cls)``. + + .. versionadded:: 3.6 + + +.. _metaclasses: + +Metaclasses +^^^^^^^^^^^ + +.. index:: + single: metaclass + builtin: type + By default, classes are constructed using :func:`type`. The class body is executed in a new namespace and the class name is bound locally to the result of ``type(name, bases, namespace)``. @@ -1666,6 +1755,8 @@ When a class definition is executed, the following steps occur: Determining the appropriate metaclass ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. index:: + single: metaclass hint The appropriate metaclass for a class definition is determined as follows: @@ -1687,13 +1778,16 @@ that criterion, then the class definition will fail with ``TypeError``. Preparing the class namespace ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. index:: + single: __prepare__ (metaclass method) + Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a ``__prepare__`` attribute, it is called as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the additional keyword arguments, if any, come from the class definition). If the metaclass has no ``__prepare__`` attribute, then the class namespace -is initialised as an empty :func:`dict` instance. +is initialised as an empty ordered mapping. .. seealso:: @@ -1704,6 +1798,9 @@ is initialised as an empty :func:`dict` instance. Executing the class body ^^^^^^^^^^^^^^^^^^^^^^^^ +.. index:: + single: class; body + The class body is executed (approximately) as ``exec(body, globals(), namespace)``. The key difference from a normal call to :func:`exec` is that lexical scoping allows the class body (including @@ -1713,12 +1810,19 @@ class definition occurs inside a function. However, even when the class definition occurs inside the function, methods defined inside the class still cannot see names defined at the class scope. Class variables must be accessed through the first parameter of instance or -class methods, and cannot be accessed at all from static methods. +class methods, or through the implicit lexically scoped ``__class__`` reference +described in the next section. +.. _class-object-creation: Creating the class object ^^^^^^^^^^^^^^^^^^^^^^^^^ +.. index:: + single: __class__ (method cell) + single: __classcell__ (class namespace entry) + + Once the class namespace has been populated by executing the class body, the class object is created by calling ``metaclass(name, bases, namespace, **kwds)`` (the additional keywords @@ -1732,14 +1836,34 @@ created by the compiler if any methods in a class body refer to either lexical scoping, while the class or instance that was used to make the current call is identified based on the first argument passed to the method. +.. impl-detail:: + + In CPython 3.6 and later, the ``__class__`` cell is passed to the metaclass + as a ``__classcell__`` entry in the class namespace. If present, this must + be propagated up to the ``type.__new__`` call in order for the class to be + initialised correctly. + Failing to do so will result in a :exc:`DeprecationWarning` in Python 3.6, + and a :exc:`RuntimeWarning` in the future. + +When using the default metaclass :class:`type`, or any metaclass that ultimately +calls ``type.__new__``, the following additional customisation steps are +invoked after creating the class object: + +* first, ``type.__new__`` collects all of the descriptors in the class + namespace that define a :meth:`~object.__set_name__` method; +* second, all of these ``__set_name__`` methods are called with the class + being defined and the assigned name of that particular descriptor; and +* finally, the :meth:`~object.__init_subclass__` hook is called on the + immediate parent of the new class in its method resolution order. + After the class object is created, it is passed to the class decorators included in the class definition (if any) and the resulting object is bound in the local namespace as the defined class. When a new class is created by ``type.__new__``, the object provided as the -namespace parameter is copied to a standard Python dictionary and the original -object is discarded. The new copy becomes the :attr:`~object.__dict__` attribute -of the class object. +namespace parameter is copied to a new ordered mapping and the original +object is discarded. The new copy is wrapped in a read-only proxy, which +becomes the :attr:`~object.__dict__` attribute of the class object. .. seealso:: @@ -2062,7 +2186,7 @@ left undefined. (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected (swapped) operands. These functions are only called if the left operand does - not support the corresponding operation and the operands are of different + not support the corresponding operation [#]_ and the operands are of different types. [#]_ For instance, to evaluate the expression ``x - y``, where *y* is an instance of a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if ``x.__sub__(y)`` returns *NotImplemented*. @@ -2478,6 +2602,17 @@ An example of an asynchronous context manager class:: controlled conditions. It generally isn't a good idea though, since it can lead to some very strange behaviour if it is handled incorrectly. +.. [#] The :meth:`__hash__`, :meth:`__iter__`, :meth:`__reversed__`, and + :meth:`__contains__` methods have special handling for this; others + will still raise a :exc:`TypeError`, but may do so by relying on + the behavior that ``None`` is not callable. + +.. [#] "Does not support" here means that the class has no such method, or + the method returns ``NotImplemented``. Do not set the method to + ``None`` if you want to force fallback to the right operand's reflected + method—that will instead have the opposite effect of explicitly + *blocking* such fallback. + .. [#] For operands of the same type, it is assumed that if the non-reflected method (such as :meth:`__add__`) fails the operation is not supported, which is why the reflected method is not called. diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ea89486128..3a4b80557c 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -172,7 +172,7 @@ Common syntax elements for comprehensions are: .. productionlist:: comprehension: `expression` `comp_for` - comp_for: "for" `target_list` "in" `or_test` [`comp_iter`] + comp_for: [ASYNC] "for" `target_list` "in" `or_test` [`comp_iter`] comp_iter: `comp_for` | `comp_if` comp_if: "if" `expression_nocond` [`comp_iter`] @@ -186,6 +186,17 @@ each time the innermost block is reached. Note that the comprehension is executed in a separate scope, so names assigned to in the target list don't "leak" into the enclosing scope. +Since Python 3.6, in an :keyword:`async def` function, an :keyword:`async for` +clause may be used to iterate over a :term:`asynchronous iterator`. +A comprehension in an :keyword:`async def` function may consist of either a +:keyword:`for` or :keyword:`async for` clause following the leading +expression, may contan additonal :keyword:`for` or :keyword:`async for` +clauses, and may also use :keyword:`await` expressions. +If a comprehension contains either :keyword:`async for` clauses +or :keyword:`await` expressions it is called an +:dfn:`asynchronous comprehension`. An asynchronous comprehension may +suspend the execution of the coroutine function in which it appears. +See also :pep:`530`. .. _lists: @@ -315,6 +326,14 @@ range(10) for y in bar(x))``. The parentheses can be omitted on calls with only one argument. See section :ref:`calls` for details. +Since Python 3.6, if the generator appears in an :keyword:`async def` function, +then :keyword:`async for` clauses and :keyword:`await` expressions are permitted +as with an asynchronous comprehension. If a generator expression +contains either :keyword:`async for` clauses or :keyword:`await` expressions +it is called an :dfn:`asynchronous generator expression`. +An asynchronous generator expression yields a new asynchronous +generator object, which is an asynchronous iterator +(see :ref:`async-iterators`). .. _yieldexpr: @@ -330,9 +349,22 @@ Yield expressions yield_atom: "(" `yield_expression` ")" yield_expression: "yield" [`expression_list` | "from" `expression`] -The yield expression is only used when defining a :term:`generator` function and +The yield expression is used when defining a :term:`generator` function +or an :term:`asynchronous generator` function and thus can only be used in the body of a function definition. Using a yield -expression in a function's body causes that function to be a generator. +expression in a function's body causes that function to be a generator, +and using it in an :keyword:`async def` function's body causes that +coroutine function to be an asynchronous generator. For example:: + + def gen(): # defines a generator function + yield 123 + + async def agen(): # defines an asynchronous generator function (PEP 525) + yield 123 + +Generator functions are described below, while asynchronous generator +functions are described separately in section +:ref:`asynchronous-generator-functions`. When a generator function is called, it returns an iterator known as a generator. That generator then controls the execution of the generator function. @@ -496,6 +528,134 @@ generator functions:: For examples using ``yield from``, see :ref:`pep-380` in "What's New in Python." +.. _asynchronous-generator-functions: + +Asynchronous generator functions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The presence of a yield expression in a function or method defined using +:keyword:`async def` further defines the function as a +:term:`asynchronous generator` function. + +When an asynchronous generator function is called, it returns an +asynchronous iterator known as an asynchronous generator object. +That object then controls the execution of the generator function. +An asynchronous generator object is typically used in an +:keyword:`async for` statement in a coroutine function analogously to +how a generator object would be used in a :keyword:`for` statement. + +Calling one of the asynchronous generator's methods returns an +:term:`awaitable` object, and the execution starts when this object +is awaited on. At that time, the execution proceeds to the first yield +expression, where it is suspended again, returning the value of +:token:`expression_list` to the awaiting coroutine. As with a generator, +suspension means that all local state is retained, including the +current bindings of local variables, the instruction pointer, the internal +evaluation stack, and the state of any exception handling. When the execution +is resumed by awaiting on the next object returned by the asynchronous +generator's methods, the function can proceed exactly as if the yield +expression were just another external call. The value of the yield expression +after resuming depends on the method which resumed the execution. If +:meth:`~agen.__anext__` is used then the result is :const:`None`. Otherwise, if +:meth:`~agen.asend` is used, then the result will be the value passed in to +that method. + +In an asynchronous generator function, yield expressions are allowed anywhere +in a :keyword:`try` construct. However, if an asynchronous generator is not +resumed before it is finalized (by reaching a zero reference count or by +being garbage collected), then a yield expression within a :keyword:`try` +construct could result in a failure to execute pending :keyword:`finally` +clauses. In this case, it is the responsibility of the event loop or +scheduler running the asynchronous generator to call the asynchronous +generator-iterator's :meth:`~agen.aclose` method and run the resulting +coroutine object, thus allowing any pending :keyword:`finally` clauses +to execute. + +To take care of finalization, an event loop should define +a *finalizer* function which takes an asynchronous generator-iterator +and presumably calls :meth:`~agen.aclose` and executes the coroutine. +This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`. +When first iterated over, an asynchronous generator-iterator will store the +registered *finalizer* to be called upon finalization. For a reference example +of a *finalizer* method see the implementation of +``asyncio.Loop.shutdown_asyncgens`` in :source:`Lib/asyncio/base_events.py`. + +The expression ``yield from <expr>`` is a syntax error when used in an +asynchronous generator function. + +.. index:: object: asynchronous-generator +.. _asynchronous-generator-methods: + +Asynchronous generator-iterator methods +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This subsection describes the methods of an asynchronous generator iterator, +which are used to control the execution of a generator function. + + +.. index:: exception: StopAsyncIteration + +.. coroutinemethod:: agen.__anext__() + + Returns an awaitable which when run starts to execute the asynchronous + generator or resumes it at the last executed yield expression. When an + asynchronous generator function is resumed with a :meth:`~agen.__anext__` + method, the current yield expression always evaluates to :const:`None` in + the returned awaitable, which when run will continue to the next yield + expression. The value of the :token:`expression_list` of the yield + expression is the value of the :exc:`StopIteration` exception raised by + the completing coroutine. If the asynchronous generator exits without + yielding another value, the awaitable instead raises an + :exc:`StopAsyncIteration` exception, signalling that the asynchronous + iteration has completed. + + This method is normally called implicitly by a :keyword:`async for` loop. + + +.. coroutinemethod:: agen.asend(value) + + Returns an awaitable which when run resumes the execution of the + asynchronous generator. As with the :meth:`~generator.send()` method for a + generator, this "sends" a value into the asynchronous generator function, + and the *value* argument becomes the result of the current yield expression. + The awaitable returned by the :meth:`asend` method will return the next + value yielded by the generator as the value of the raised + :exc:`StopIteration`, or raises :exc:`StopAsyncIteration` if the + asynchronous generator exits without yielding another value. When + :meth:`asend` is called to start the asynchronous + generator, it must be called with :const:`None` as the argument, + because there is no yield expression that could receive the value. + + +.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) + + Returns an awaitable that raises an exception of type ``type`` at the point + where the asynchronous generator was paused, and returns the next value + yielded by the generator function as the value of the raised + :exc:`StopIteration` exception. If the asynchronous generator exits + without yielding another value, an :exc:`StopAsyncIteration` exception is + raised by the awaitable. + If the generator function does not catch the passed-in exception, or + raises a different exception, then when the awaitalbe is run that exception + propagates to the caller of the awaitable. + +.. index:: exception: GeneratorExit + + +.. coroutinemethod:: agen.aclose() + + Returns an awaitable that when run will throw a :exc:`GeneratorExit` into + the asynchronous generator function at the point where it was paused. + If the asynchronous generator function then exits gracefully, is already + closed, or raises :exc:`GeneratorExit` (by not catching the exception), + then the returned awaitable will raise a :exc:`StopIteration` exception. + Any further awaitables returned by subsequent calls to the asynchronous + generator will raise a :exc:`StopAsyncIteration` exception. If the + asynchronous generator yields a value, a :exc:`RuntimeError` is raised + by the awaitable. If the asynchronous generator raises any other exception, + it is propagated to the caller of the awaitable. If the asynchronous + generator has already exited due to an exception or normal exit, then + further calls to :meth:`aclose` will return an awaitable that does nothing. .. _primaries: diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst index 64302b89a8..5e2c1c8b07 100644 --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -36,7 +36,7 @@ implement import semantics. When a module is first imported, Python searches for the module and if found, it creates a module object [#fnmo]_, initializing it. If the named module -cannot be found, an :exc:`ImportError` is raised. Python implements various +cannot be found, an :exc:`ModuleNotFoundError` is raised. Python implements various strategies to search for the named module when the import machinery is invoked. These strategies can be modified and extended by using various hooks described in the sections below. @@ -167,7 +167,7 @@ arguments to the :keyword:`import` statement, or from the parameters to the This name will be used in various phases of the import search, and it may be the dotted path to a submodule, e.g. ``foo.bar.baz``. In this case, Python first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``. -If any of the intermediate imports fail, an :exc:`ImportError` is raised. +If any of the intermediate imports fail, an :exc:`ModuleNotFoundError` is raised. The module cache @@ -186,7 +186,7 @@ object. During import, the module name is looked up in :data:`sys.modules` and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is ``None``, then an -:exc:`ImportError` is raised. If the module name is missing, Python will +:exc:`ModuleNotFoundError` is raised. If the module name is missing, Python will continue searching for the module. :data:`sys.modules` is writable. Deleting a key may not destroy the @@ -194,7 +194,7 @@ associated module (as other modules may hold references to it), but it will invalidate the cache entry for the named module, causing Python to search anew for the named module upon its next import. The key can also be assigned to ``None``, forcing the next import -of the module to result in an :exc:`ImportError`. +of the module to result in an :exc:`ModuleNotFoundError`. Beware though, as if you keep a reference to the module object, invalidate its cache entry in :data:`sys.modules`, and then re-import the @@ -288,8 +288,8 @@ the named module or not. If the meta path finder knows how to handle the named module, it returns a spec object. If it cannot handle the named module, it returns ``None``. If :data:`sys.meta_path` processing reaches the end of its list without returning -a spec, then an :exc:`ImportError` is raised. Any other exceptions raised -are simply propagated up, aborting the import process. +a spec, then a :exc:`ModuleNotFoundError` is raised. Any other exceptions +raised are simply propagated up, aborting the import process. The :meth:`~importlib.abc.MetaPathFinder.find_spec()` method of meta path finders is called with two or three arguments. The first is the fully @@ -298,9 +298,9 @@ The second argument is the path entries to use for the module search. For top-level modules, the second argument is ``None``, but for submodules or subpackages, the second argument is the value of the parent package's ``__path__`` attribute. If the appropriate ``__path__`` attribute cannot -be accessed, an :exc:`ImportError` is raised. The third argument is an -existing module object that will be the target of loading later. The -import system passes in a target module only during reload. +be accessed, an :exc:`ModuleNotFoundError` is raised. The third argument +is an existing module object that will be the target of loading later. +The import system passes in a target module only during reload. The meta path may be traversed multiple times for a single import request. For example, assuming none of the modules involved has already been cached, @@ -554,19 +554,30 @@ the module. details. This attribute is used instead of ``__name__`` to calculate explicit - relative imports for main modules, as defined in :pep:`366`. + relative imports for main modules, as defined in :pep:`366`. It is + expected to have the same value as ``__spec__.parent``. + + .. versionchanged:: 3.6 + The value of ``__package__`` is expected to be the same as + ``__spec__.parent``. .. attribute:: __spec__ The ``__spec__`` attribute must be set to the module spec that was - used when importing the module. This is used primarily for - introspection and during reloading. Setting ``__spec__`` + used when importing the module. Setting ``__spec__`` appropriately applies equally to :ref:`modules initialized during interpreter startup <programs>`. The one exception is ``__main__``, where ``__spec__`` is :ref:`set to None in some cases <main_spec>`. + When ``__package__`` is not defined, ``__spec__.parent`` is used as + a fallback. + .. versionadded:: 3.4 + .. versionchanged:: 3.6 + ``__spec__.parent`` is used as a fallback when ``__package__`` is + not defined. + .. attribute:: __path__ If the module is a package (either regular or namespace), the module @@ -876,7 +887,7 @@ import statements within that module. To selectively prevent import of some modules from a hook early on the meta path (rather than disabling the standard import system entirely), -it is sufficient to raise :exc:`ImportError` directly from +it is sufficient to raise :exc:`ModuleNoFoundError` directly from :meth:`~importlib.abc.MetaPathFinder.find_spec` instead of returning ``None``. The latter indicates that the meta path search should continue, while raising an exception terminates it immediately. diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 37f25f1def..da7017afff 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -405,7 +405,8 @@ String literals are described by the following lexical definitions: .. productionlist:: stringliteral: [`stringprefix`](`shortstring` | `longstring`) - stringprefix: "r" | "u" | "R" | "U" + stringprefix: "r" | "u" | "R" | "U" | "f" | "F" + : | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF" shortstring: "'" `shortstringitem`* "'" | '"' `shortstringitem`* '"' longstring: "'''" `longstringitem`* "'''" | '"""' `longstringitem`* '"""' shortstringitem: `shortstringchar` | `stringescapeseq` @@ -464,6 +465,11 @@ is not supported. to simplify the maintenance of dual Python 2.x and 3.x codebases. See :pep:`414` for more information. +A string literal with ``'f'`` or ``'F'`` in its prefix is a +:dfn:`formatted string literal`; see :ref:`f-strings`. The ``'f'`` may be +combined with ``'r'``, but not with ``'b'`` or ``'u'``, therefore raw +formatted strings are possible, but formatted bytes literals are not. + In triple-quoted literals, unescaped newlines and quotes are allowed (and are retained), except that three unescaped quotes in a row terminate the literal. (A "quote" is the character used to open the literal, i.e. either ``'`` or ``"``.) @@ -554,6 +560,10 @@ is more easily recognized as broken.) It is also important to note that the escape sequences only recognized in string literals fall into the category of unrecognized escapes for bytes literals. + .. versionchanged:: 3.6 + Unrecognized escape sequences produce a DeprecationWarning. In + some future version of Python they will be a SyntaxError. + Even in a raw literal, quotes can be escaped with a backslash, but the backslash remains in the result; for example, ``r"\""`` is a valid string literal consisting of two characters: a backslash and a double quote; ``r"\"`` @@ -583,7 +593,111 @@ comments to parts of strings, for example:: Note that this feature is defined at the syntactical level, but implemented at compile time. The '+' operator must be used to concatenate string expressions at run time. Also note that literal concatenation can use different quoting -styles for each component (even mixing raw strings and triple quoted strings). +styles for each component (even mixing raw strings and triple quoted strings), +and formatted string literals may be concatenated with plain string literals. + + +.. index:: + single: formatted string literal + single: interpolated string literal + single: string; formatted literal + single: string; interpolated literal + single: f-string +.. _f-strings: + +Formatted string literals +------------------------- + +.. versionadded:: 3.6 + +A :dfn:`formatted string literal` or :dfn:`f-string` is a string literal +that is prefixed with ``'f'`` or ``'F'``. These strings may contain +replacement fields, which are expressions delimited by curly braces ``{}``. +While other string literals always have a constant value, formatted strings +are really expressions evaluated at run time. + +Escape sequences are decoded like in ordinary string literals (except when +a literal is also marked as a raw string). After decoding, the grammar +for the contents of the string is: + +.. productionlist:: + f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* + replacement_field: "{" `f_expression` ["!" `conversion`] [":" `format_spec`] "}" + f_expression: (`conditional_expression` | "*" `or_expr`) + : ("," `conditional_expression` | "," "*" `or_expr`)* [","] + : | `yield_expression` + conversion: "s" | "r" | "a" + format_spec: (`literal_char` | NULL | `replacement_field`)* + literal_char: <any code point except "{", "}" or NULL> + +The parts of the string outside curly braces are treated literally, +except that any doubled curly braces ``'{{'`` or ``'}}'`` are replaced +with the corresponding single curly brace. A single opening curly +bracket ``'{'`` marks a replacement field, which starts with a +Python expression. After the expression, there may be a conversion field, +introduced by an exclamation point ``'!'``. A format specifier may also +be appended, introduced by a colon ``':'``. A replacement field ends +with a closing curly bracket ``'}'``. + +Expressions in formatted string literals are treated like regular +Python expressions surrounded by parentheses, with a few exceptions. +An empty expression is not allowed, and a :keyword:`lambda` expression +must be surrounded by explicit parentheses. Replacement expressions +can contain line breaks (e.g. in triple-quoted strings), but they +cannot contain comments. Each expression is evaluated in the context +where the formatted string literal appears, in order from left to right. + +If a conversion is specified, the result of evaluating the expression +is converted before formatting. Conversion ``'!s'`` calls :func:`str` on +the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. + +The result is then formatted using the :func:`format` protocol. The +format specifier is passed to the :meth:`__format__` method of the +expression or conversion result. An empty string is passed when the +format specifier is omitted. The formatted result is then included in +the final value of the whole string. + +Top-level format specifiers may include nested replacement fields. +These nested fields may include their own conversion fields and +format specifiers, but may not include more deeply-nested replacement fields. + +Formatted string literals may be concatenated, but replacement fields +cannot be split across literals. + +Some examples of formatted string literals:: + + >>> name = "Fred" + >>> f"He said his name is {name!r}." + "He said his name is 'Fred'." + >>> f"He said his name is {repr(name)}." # repr() is equivalent to !r + "He said his name is 'Fred'." + >>> width = 10 + >>> precision = 4 + >>> value = decimal.Decimal("12.34567") + >>> f"result: {value:{width}.{precision}}" # nested fields + 'result: 12.35' + +A consequence of sharing the same syntax as regular string literals is +that characters in the replacement fields must not conflict with the +quoting used in the outer formatted string literal:: + + f"abc {a["x"]} def" # error: outer string literal ended prematurely + f"abc {a['x']} def" # workaround: use different quoting + +Backslashes are not allowed in format expressions and will raise +an error:: + + f"newline: {ord('\n')}" # raises SyntaxError + +To include a value in which a backslash escape is required, create +a temporary variable. + + >>> newline = ord('\n') + >>> f"newline: {newline}" + 'newline: 10' + +See also :pep:`498` for the proposal that added formatted string literals, +and :meth:`str.format`, which uses a related format string mechanism. .. _numbers: @@ -612,20 +726,24 @@ Integer literals Integer literals are described by the following lexical definitions: .. productionlist:: - integer: `decimalinteger` | `octinteger` | `hexinteger` | `bininteger` - decimalinteger: `nonzerodigit` `digit`* | "0"+ + integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` + decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* + bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ + octinteger: "0" ("o" | "O") (["_"] `octdigit`)+ + hexinteger: "0" ("x" | "X") (["_"] `hexdigit`)+ nonzerodigit: "1"..."9" digit: "0"..."9" - octinteger: "0" ("o" | "O") `octdigit`+ - hexinteger: "0" ("x" | "X") `hexdigit`+ - bininteger: "0" ("b" | "B") `bindigit`+ + bindigit: "0" | "1" octdigit: "0"..."7" hexdigit: `digit` | "a"..."f" | "A"..."F" - bindigit: "0" | "1" There is no limit for the length of integer literals apart from what can be stored in available memory. +Underscores are ignored for determining the numeric value of the literal. They +can be used to group digits for enhanced readability. One underscore can occur +between digits, and after base specifiers like ``0x``. + Note that leading zeros in a non-zero decimal number are not allowed. This is for disambiguation with C-style octal literals, which Python used before version 3.0. @@ -634,6 +752,10 @@ Some examples of integer literals:: 7 2147483647 0o177 0b100110111 3 79228162514264337593543950336 0o377 0xdeadbeef + 100_000_000_000 0b_1110_0101 + +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. .. _floating: @@ -645,23 +767,28 @@ Floating point literals are described by the following lexical definitions: .. productionlist:: floatnumber: `pointfloat` | `exponentfloat` - pointfloat: [`intpart`] `fraction` | `intpart` "." - exponentfloat: (`intpart` | `pointfloat`) `exponent` - intpart: `digit`+ - fraction: "." `digit`+ - exponent: ("e" | "E") ["+" | "-"] `digit`+ + pointfloat: [`digitpart`] `fraction` | `digitpart` "." + exponentfloat: (`digitpart` | `pointfloat`) `exponent` + digitpart: `digit` (["_"] `digit`)* + fraction: "." `digitpart` + exponent: ("e" | "E") ["+" | "-"] `digitpart` Note that the integer and exponent parts are always interpreted using radix 10. For example, ``077e010`` is legal, and denotes the same number as ``77e10``. The -allowed range of floating point literals is implementation-dependent. Some -examples of floating point literals:: +allowed range of floating point literals is implementation-dependent. As in +integer literals, underscores are supported for digit grouping. - 3.14 10. .001 1e100 3.14e-10 0e0 +Some examples of floating point literals:: + + 3.14 10. .001 1e100 3.14e-10 0e0 3.14_15_93 Note that numeric literals do not include a sign; a phrase like ``-1`` is actually an expression composed of the unary operator ``-`` and the literal ``1``. +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. + .. _imaginary: @@ -671,7 +798,7 @@ Imaginary literals Imaginary literals are described by the following lexical definitions: .. productionlist:: - imagnumber: (`floatnumber` | `intpart`) ("j" | "J") + imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex numbers are represented as a pair of floating point numbers and have the same @@ -679,7 +806,7 @@ restrictions on their range. To create a complex number with a nonzero real part, add a floating point number to it, e.g., ``(3+4j)``. Some examples of imaginary literals:: - 3.14j 10.j 10j .001j 1e100j 3.14e-10j + 3.14j 10.j 10j .001j 1e100j 3.14e-10j 3.14_15_93j .. _operators: diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index d403c4d546..e152b16ee3 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -16,6 +16,7 @@ simple statements is: : | `assert_stmt` : | `assignment_stmt` : | `augmented_assignment_stmt` + : | `annotated_assignment_stmt` : | `pass_stmt` : | `del_stmt` : | `return_stmt` @@ -84,7 +85,7 @@ attributes or items of mutable objects: assignment_stmt: (`target_list` "=")+ (`starred_expression` | `yield_expression`) target_list: `target` ("," `target`)* [","] target: `identifier` - : | "(" `target_list` ")" + : | "(" [`target_list`] ")" : | "[" [`target_list`] "]" : | `attributeref` : | `subscription` @@ -312,6 +313,50 @@ For targets which are attribute references, the same :ref:`caveat about class and instance attributes <attr-target-note>` applies as for regular assignments. +.. _annassign: + +Annotated assignment statements +------------------------------- + +.. index:: + pair: annotated; assignment + single: statement; assignment, annotated + +Annotation assignment is the combination, in a single statement, +of a variable or attribute annotation and an optional assignment statement: + +.. productionlist:: + annotated_assignment_stmt: `augtarget` ":" `expression` ["=" `expression`] + +The difference from normal :ref:`assignment` is that only single target and +only single right hand side value is allowed. + +For simple names as assignment targets, if in class or module scope, +the annotations are evaluated and stored in a special class or module +attribute :attr:`__annotations__` +that is a dictionary mapping from variable names (mangled if private) to +evaluated annotations. This attribute is writable and is automatically +created at the start of class or module body execution, if annotations +are found statically. + +For expressions as assignment targets, the annotations are evaluated if +in class or module scope, but not stored. + +If a name is annotated in a function scope, then this name is local for +that scope. Annotations are never evaluated and stored in function scopes. + +If the right hand side is present, an annotated +assignment performs the actual assignment before evaluating annotations +(where applicable). If the right hand side is not present for an expression +target, then the interpreter evaluates the target except for the last +:meth:`__setitem__` or :meth:`__setattr__` call. + +.. seealso:: + + :pep:`526` - Variable and attribute annotation syntax + :pep:`484` - Type hints + + .. _assert: The :keyword:`assert` statement @@ -447,6 +492,10 @@ generator is done and will cause :exc:`StopIteration` to be raised. The returned value (if any) is used as an argument to construct :exc:`StopIteration` and becomes the :attr:`StopIteration.value` attribute. +In an asynchronous generator function, an empty :keyword:`return` statement +indicates that the asynchronous generator is done and will cause +:exc:`StopAsyncIteration` to be raised. A non-empty :keyword:`return` +statement is a syntax error in an asynchronous generator function. .. _yield: @@ -859,11 +908,12 @@ block textually preceding that :keyword:`global` statement. Names listed in a :keyword:`global` statement must not be defined as formal parameters or in a :keyword:`for` loop control target, :keyword:`class` -definition, function definition, or :keyword:`import` statement. +definition, function definition, :keyword:`import` statement, or variable +annotation. .. impl-detail:: - The current implementation does not enforce the two restrictions, but + The current implementation does not enforce some of these restriction, but programs should not abuse this freedom, as future implementations may enforce them or silently change the meaning of the program. |