diff options
Diffstat (limited to 'Doc/library')
61 files changed, 1250 insertions, 327 deletions
diff --git a/Doc/library/asynchat.rst b/Doc/library/asynchat.rst index ae72d26122..e02360c883 100644 --- a/Doc/library/asynchat.rst +++ b/Doc/library/asynchat.rst @@ -58,13 +58,13 @@ connection requests. The asynchronous output buffer size (default ``4096``). Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to - define a first-in-first-out queue (fifo) of *producers*. A producer need + define a :abbr:`FIFO (first-in, first-out)` queue of *producers*. A producer need have only one method, :meth:`more`, which should return data to be transmitted on the channel. The producer indicates exhaustion (*i.e.* that it contains no more data) by having its :meth:`more` method return the empty bytes object. At this point - the :class:`async_chat` object removes the producer from the fifo and starts - using the next producer, if any. When the producer fifo is empty the + the :class:`async_chat` object removes the producer from the queue and starts + using the next producer, if any. When the producer queue is empty the :meth:`handle_write` method does nothing. You use the channel object's :meth:`set_terminator` method to describe how to recognize the end of, or an important breakpoint in, an incoming transmission from the remote @@ -78,8 +78,8 @@ connection requests. .. method:: async_chat.close_when_done() - Pushes a ``None`` on to the producer fifo. When this producer is popped off - the fifo it causes the channel to be closed. + Pushes a ``None`` on to the producer queue. When this producer is popped off + the queue it causes the channel to be closed. .. method:: async_chat.collect_incoming_data(data) @@ -92,7 +92,7 @@ connection requests. .. method:: async_chat.discard_buffers() In emergencies this method will discard any data held in the input and/or - output buffers and the producer fifo. + output buffers and the producer queue. .. method:: async_chat.found_terminator() @@ -110,7 +110,7 @@ connection requests. .. method:: async_chat.push(data) - Pushes data on to the channel's fifo to ensure its transmission. + Pushes data on to the channel's queue to ensure its transmission. This is all you need to do to have the channel write the data out to the network, although it is possible to use your own producers in more complex schemes to implement encryption and chunking, for example. @@ -118,7 +118,7 @@ connection requests. .. method:: async_chat.push_with_producer(producer) - Takes a producer object and adds it to the producer fifo associated with + Takes a producer object and adds it to the producer queue associated with the channel. When all currently-pushed producers have been exhausted the channel will consume this producer's data by calling its :meth:`more` method and send the data to the remote endpoint. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 7ec3aa106e..809a3e77d5 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -101,8 +101,9 @@ keywords to your callback, use :func:`functools.partial`. For example, called after :meth:`call_soon` returns, when control returns to the event loop. - This operates as a FIFO queue, callbacks are called in the order in - which they are registered. Each callback will be called exactly once. + This operates as a :abbr:`FIFO (first-in, first-out)` queue, callbacks + are called in the order in which they are registered. Each callback + will be called exactly once. Any positional arguments after the callback will be passed to the callback when it is called. diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst index 878d8db53c..4f5f0f26ad 100644 --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -53,13 +53,14 @@ The :mod:`binascii` module defines the following functions: than one line may be passed at a time. -.. function:: b2a_base64(data) +.. function:: b2a_base64(data, \*, newline=True) Convert binary data to a line of ASCII characters in base64 coding. The return - value is the converted line, including a newline char. The newline is - added because the original use case for this function was to feed it a - series of 57 byte input lines to get output lines that conform to the - MIME-base64 standard. Otherwise the output conforms to :rfc:`3548`. + value is the converted line, including a newline char if *newline* is + true. The output of this function conforms to :rfc:`3548`. + + .. versionchanged:: 3.6 + Added the *newline* parameter. .. function:: a2b_qp(data, header=False) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 61682ccd37..5b0fb38732 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -41,12 +41,13 @@ ABC Inherits from Abstract Methods Mixin :class:`Hashable` ``__hash__`` :class:`Iterable` ``__iter__`` :class:`Iterator` :class:`Iterable` ``__next__`` ``__iter__`` +:class:`Reversible` :class:`Iterable` ``__reversed__`` :class:`Generator` :class:`Iterator` ``send``, ``throw`` ``close``, ``__iter__``, ``__next__`` :class:`Sized` ``__len__`` :class:`Callable` ``__call__`` :class:`Sequence` :class:`Sized`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``, - :class:`Iterable`, ``__len__`` ``index``, and ``count`` + :class:`Reversible`, ``__len__`` ``index``, and ``count`` :class:`Container` :class:`MutableSequence` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods and @@ -108,6 +109,12 @@ ABC Inherits from Abstract Methods Mixin :meth:`~iterator.__next__` methods. See also the definition of :term:`iterator`. +.. class:: Reversible + + ABC for classes that provide the :meth:`__reversed__` method. + + .. versionadded:: 3.6 + .. class:: Generator ABC for generator classes that implement the protocol defined in diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index 4936b3a0f8..1758f32577 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -964,6 +964,9 @@ customize a prototype instance: constructor that is convenient for use cases where named tuples are being subclassed. + * :meth:`types.SimpleNamespace` for a mutable namespace based on an underlying + dictionary instead of a tuple. + :class:`OrderedDict` objects ---------------------------- @@ -985,8 +988,9 @@ the items are returned in the order their keys were first added. .. method:: popitem(last=True) The :meth:`popitem` method for ordered dictionaries returns and removes a - (key, value) pair. The pairs are returned in LIFO order if *last* is true - or FIFO order if false. + (key, value) pair. The pairs are returned in + :abbr:`LIFO (last-in, first-out)` order if *last* is true + or :abbr:`FIFO (first-in, first-out)` order if false. .. method:: move_to_end(key, last=True) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index 511c581ad0..91bdd18d70 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -102,7 +102,8 @@ Public functions .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1) Recursively descend the directory tree named by *dir*, compiling all :file:`.py` - files along the way. + files along the way. Return a true value if all the files compiled successfully, + and a false value otherwise. The *maxlevels* parameter is used to limit the depth of the recursion; it defaults to ``10``. @@ -154,7 +155,8 @@ Public functions .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1) - Compile the file with path *fullname*. + Compile the file with path *fullname*. Return a true value if the file + compiled successfully, and a false value otherwise. If *ddir* is given, it is prepended to the path to the file being compiled for use in compilation time tracebacks, and is also compiled in to the @@ -190,8 +192,10 @@ Public functions .. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1) - Byte-compile all the :file:`.py` files found along ``sys.path``. If - *skip_curdir* is true (the default), the current directory is not included + Byte-compile all the :file:`.py` files found along ``sys.path``. Return a + true value if all the files compiled successfully, and a false value otherwise. + + If *skip_curdir* is true (the default), the current directory is not included in the search. All other parameters are passed to the :func:`compile_dir` function. Note that unlike the other compile functions, ``maxlevels`` defaults to ``0``. diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst index cf85fcd4b2..810cea8ba4 100644 --- a/Doc/library/contextlib.rst +++ b/Doc/library/contextlib.rst @@ -18,6 +18,18 @@ Utilities Functions and classes provided: +.. class:: AbstractContextManager + + An :term:`abstract base class` for classes that implement + :meth:`object.__enter__` and :meth:`object.__exit__`. A default + implementation for :meth:`object.__enter__` is provided which returns + ``self`` while :meth:`object.__exit__` is an abstract method which by default + returns ``None``. See also the definition of :ref:`typecontextmanager`. + + .. versionadded:: 3.6 + + + .. decorator:: contextmanager This function is a :term:`decorator` that can be used to define a factory @@ -447,9 +459,9 @@ Here's an example of doing this for a context manager that accepts resource acquisition and release functions, along with an optional validation function, and maps them to the context management protocol:: - from contextlib import contextmanager, ExitStack + from contextlib import contextmanager, AbstractContextManager, ExitStack - class ResourceManager: + class ResourceManager(AbstractContextManager): def __init__(self, acquire_resource, release_resource, check_resource_ok=None): self.acquire_resource = acquire_resource diff --git a/Doc/library/crypt.rst b/Doc/library/crypt.rst index a21c1e7a75..dbd4274038 100644 --- a/Doc/library/crypt.rst +++ b/Doc/library/crypt.rst @@ -68,7 +68,7 @@ Module Attributes A list of available password hashing algorithms, as ``crypt.METHOD_*`` objects. This list is sorted from strongest to - weakest, and is guaranteed to have at least ``crypt.METHOD_CRYPT``. + weakest. Module Functions diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst index 1eddfdc12c..ae45549a6d 100644 --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -16,3 +16,4 @@ Here's an overview: hashlib.rst hmac.rst + secrets.rst diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 9254ae8cfa..cd78750a94 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -610,7 +610,8 @@ Instance methods: .. method:: date.__format__(format) Same as :meth:`.date.strftime`. This makes it possible to specify a format - string for a :class:`.date` object when using :meth:`str.format`. For a + string for a :class:`.date` object in :ref:`formatted string + literals <f-strings>` and when using :meth:`str.format`. For a complete list of formatting directives, see :ref:`strftime-strptime-behavior`. @@ -1138,7 +1139,7 @@ Instance methods: ``self.date().isocalendar()``. -.. method:: datetime.isoformat(sep='T') +.. method:: datetime.isoformat(sep='T', timespec='auto') Return a string representing the date and time in ISO 8601 format, YYYY-MM-DDTHH:MM:SS.mmmmmm or, if :attr:`microsecond` is 0, @@ -1159,6 +1160,37 @@ Instance methods: >>> datetime(2002, 12, 25, tzinfo=TZ()).isoformat(' ') '2002-12-25 00:00:00-06:39' + The optional argument *timespec* specifies the number of additional + components of the time to include (the default is ``'auto'``). + It can be one of the following: + + - ``'auto'``: Same as ``'seconds'`` if :attr:`microsecond` is 0, + same as ``'microseconds'`` otherwise. + - ``'hours'``: Include the :attr:`hour` in the two-digit HH format. + - ``'minutes'``: Include :attr:`hour` and :attr:`minute` in HH:MM format. + - ``'seconds'``: Include :attr:`hour`, :attr:`minute`, and :attr:`second` + in HH:MM:SS format. + - ``'milliseconds'``: Include full time, but truncate fractional second + part to milliseconds. HH:MM:SS.sss format. + - ``'microseconds'``: Include full time in HH:MM:SS.mmmmmm format. + + .. note:: + + Excluded time components are truncated, not rounded. + + :exc:`ValueError` will be raised on an invalid *timespec* argument. + + + >>> from datetime import datetime + >>> datetime.now().isoformat(timespec='minutes') + '2002-12-25T00:00' + >>> dt = datetime(2015, 1, 1, 12, 30, 59, 0) + >>> dt.isoformat(timespec='microseconds') + '2015-01-01T12:30:59.000000' + + .. versionadded:: 3.6 + Added the *timespec* argument. + .. method:: datetime.__str__() @@ -1185,7 +1217,8 @@ Instance methods: .. method:: datetime.__format__(format) Same as :meth:`.datetime.strftime`. This makes it possible to specify a format - string for a :class:`.datetime` object when using :meth:`str.format`. For a + string for a :class:`.datetime` object in :ref:`formatted string + literals <f-strings>` and when using :meth:`str.format`. For a complete list of formatting directives, see :ref:`strftime-strptime-behavior`. @@ -1407,13 +1440,46 @@ Instance methods: aware :class:`.time`, without conversion of the time data. -.. method:: time.isoformat() +.. method:: time.isoformat(timespec='auto') Return a string representing the time in ISO 8601 format, HH:MM:SS.mmmmmm or, if - self.microsecond is 0, HH:MM:SS If :meth:`utcoffset` does not return ``None``, a + :attr:`microsecond` is 0, HH:MM:SS If :meth:`utcoffset` does not return ``None``, a 6-character string is appended, giving the UTC offset in (signed) hours and minutes: HH:MM:SS.mmmmmm+HH:MM or, if self.microsecond is 0, HH:MM:SS+HH:MM + The optional argument *timespec* specifies the number of additional + components of the time to include (the default is ``'auto'``). + It can be one of the following: + + - ``'auto'``: Same as ``'seconds'`` if :attr:`microsecond` is 0, + same as ``'microseconds'`` otherwise. + - ``'hours'``: Include the :attr:`hour` in the two-digit HH format. + - ``'minutes'``: Include :attr:`hour` and :attr:`minute` in HH:MM format. + - ``'seconds'``: Include :attr:`hour`, :attr:`minute`, and :attr:`second` + in HH:MM:SS format. + - ``'milliseconds'``: Include full time, but truncate fractional second + part to milliseconds. HH:MM:SS.sss format. + - ``'microseconds'``: Include full time in HH:MM:SS.mmmmmm format. + + .. note:: + + Excluded time components are truncated, not rounded. + + :exc:`ValueError` will be raised on an invalid *timespec* argument. + + + >>> from datetime import time + >>> time(hour=12, minute=34, second=56, microsecond=123456).isoformat(timespec='minutes') + '12:34' + >>> dt = time(hour=12, minute=34, second=56, microsecond=0) + >>> dt.isoformat(timespec='microseconds') + '12:34:56.000000' + >>> dt.isoformat(timespec='auto') + '12:34:56' + + .. versionadded:: 3.6 + Added the *timespec* argument. + .. method:: time.__str__() @@ -1430,7 +1496,8 @@ Instance methods: .. method:: time.__format__(format) Same as :meth:`.time.strftime`. This makes it possible to specify a format string - for a :class:`.time` object when using :meth:`str.format`. For a + for a :class:`.time` object in :ref:`formatted string + literals <f-strings>` and when using :meth:`str.format`. For a complete list of formatting directives, see :ref:`strftime-strptime-behavior`. @@ -1741,10 +1808,7 @@ made to civil time. otherwise :exc:`ValueError` is raised. The *name* argument is optional. If specified it must be a string that - is used as the value returned by the ``tzname(dt)`` method. Otherwise, - ``tzname(dt)`` returns a string 'UTCsHH:MM', where s is the sign of - *offset*, HH and MM are two digits of ``offset.hours`` and - ``offset.minutes`` respectively. + will be used as the value returned by the :meth:`datetime.tzname` method. .. versionadded:: 3.2 @@ -1757,11 +1821,19 @@ made to civil time. .. method:: timezone.tzname(dt) - Return the fixed value specified when the :class:`timezone` instance is - constructed or a string 'UTCsHH:MM', where s is the sign of - *offset*, HH and MM are two digits of ``offset.hours`` and + Return the fixed value specified when the :class:`timezone` instance + is constructed. If *name* is not provided in the constructor, the + name returned by ``tzname(dt)`` is generated from the value of the + ``offset`` as follows. If *offset* is ``timedelta(0)``, the name + is "UTC", otherwise it is a string 'UTC±HH:MM', where ± is the sign + of ``offset``, HH and MM are two digits of ``offset.hours`` and ``offset.minutes`` respectively. + .. versionchanged:: 3.6 + Name generated from ``offset=timedelta(0)`` is now plain 'UTC', not + 'UTC+00:00'. + + .. method:: timezone.dst(dt) Always returns ``None``. @@ -1911,6 +1983,34 @@ format codes. | ``%%`` | A literal ``'%'`` character. | % | | +-----------+--------------------------------+------------------------+-------+ +Several additional directives not required by the C89 standard are included for +convenience. These parameters all correspond to ISO 8601 date values. These +may not be available on all platforms when used with the :meth:`strftime` +method. The ISO 8601 year and ISO 8601 week directives are not interchangeable +with the year and week number directives above. Calling :meth:`strptime` with +incomplete or ambiguous ISO 8601 directives will raise a :exc:`ValueError`. + ++-----------+--------------------------------+------------------------+-------+ +| Directive | Meaning | Example | Notes | ++===========+================================+========================+=======+ +| ``%G`` | ISO 8601 year with century | 0001, 0002, ..., 2013, | \(8) | +| | representing the year that | 2014, ..., 9998, 9999 | | +| | contains the greater part of | | | +| | the ISO week (``%V``). | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%u`` | ISO 8601 weekday as a decimal | 1, 2, ..., 7 | | +| | number where 1 is Monday. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%V`` | ISO 8601 week as a decimal | 01, 02, ..., 53 | \(8) | +| | number with Monday as | | | +| | the first day of the week. | | | +| | Week 01 is the week containing | | | +| | Jan 4. | | | ++-----------+--------------------------------+------------------------+-------+ + +.. versionadded:: 3.6 + ``%G``, ``%u`` and ``%V`` were added. + Notes: (1) @@ -1975,7 +2075,14 @@ Notes: (7) When used with the :meth:`strptime` method, ``%U`` and ``%W`` are only used - in calculations when the day of the week and the year are specified. + in calculations when the day of the week and the calendar year (``%Y``) + are specified. + +(8) + Similar to ``%U`` and ``%W``, ``%V`` is only used in calculations when the + day of the week and the ISO year (``%G``) are specified in a + :meth:`strptime` format string. Also note that ``%G`` and ``%Y`` are not + interchangeable. .. rubric:: Footnotes diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index 528f97b1ab..9edc696b79 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -447,6 +447,19 @@ Decimal objects ``Decimal('321e+5').adjusted()`` returns seven. Used for determining the position of the most significant digit with respect to the decimal point. + .. method:: as_integer_ratio() + + Return a pair ``(n, d)`` of integers that represent the given + :class:`Decimal` instance as a fraction, in lowest terms and + with a positive denominator:: + + >>> Decimal('-3.14').as_integer_ratio() + (-157, 50) + + The conversion is exact. Raise OverflowError on infinities and ValueError + on NaNs. + + .. versionadded:: 3.6 .. method:: as_tuple() diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index d2d8ac7839..245b4d2075 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -31,9 +31,9 @@ the following command can be used to display the disassembly of >>> dis.dis(myfunc) 2 0 LOAD_GLOBAL 0 (len) - 3 LOAD_FAST 0 (alist) - 6 CALL_FUNCTION 1 - 9 RETURN_VALUE + 2 LOAD_FAST 0 (alist) + 4 CALL_FUNCTION 1 + 6 RETURN_VALUE (The "2" is a line number). @@ -682,8 +682,7 @@ iterations of the loop. .. XXX explain the WHY stuff! -All of the following opcodes expect arguments. An argument is two bytes, with -the more significant byte last. +All of the following opcodes use their arguments. .. opcode:: STORE_NAME (namei) @@ -769,6 +768,15 @@ the more significant byte last. to hold *count* entries. +.. opcode:: BUILD_CONST_KEY_MAP (count) + + The version of :opcode:`BUILD_MAP` specialized for constant keys. *count* + values are consumed from the stack. The top element on the stack contains + a tuple of keys. + + .. versionadded:: 3.6 + + .. opcode:: LOAD_ATTR (namei) Replaces TOS with ``getattr(TOS, co_names[namei])``. @@ -929,27 +937,16 @@ the more significant byte last. .. opcode:: MAKE_FUNCTION (argc) Pushes a new function object on the stack. From bottom to top, the consumed - stack must consist of - - * ``argc & 0xFF`` default argument objects in positional order - * ``(argc >> 8) & 0xFF`` pairs of name and default argument, with the name - just below the object on the stack, for keyword-only parameters - * ``(argc >> 16) & 0x7FFF`` parameter annotation objects - * a tuple listing the parameter names for the annotations (only if there are - ony annotation objects) + stack must consist of values if the argument carries a specified flag value + + * ``0x01`` a tuple of default argument objects in positional order + * ``0x02`` a dictionary of keyword-only parameters' default values + * ``0x04`` an annotation dictionary + * ``0x08`` a tuple containing cells for free variables, making a closure * the code associated with the function (at TOS1) * the :term:`qualified name` of the function (at TOS) -.. opcode:: MAKE_CLOSURE (argc) - - Creates a new function object, sets its *__closure__* slot, and pushes it on - the stack. TOS is the :term:`qualified name` of the function, TOS1 is the - code associated with the function, and TOS2 is the tuple containing cells for - the closure's free variables. *argc* is interpreted as in ``MAKE_FUNCTION``; - the annotations and defaults are also in the same order below TOS2. - - .. opcode:: BUILD_SLICE (argc) .. index:: builtin: slice @@ -989,6 +986,28 @@ the more significant byte last. arguments. +.. opcode:: FORMAT_VALUE (flags) + + Used for implementing formatted literal strings (f-strings). Pops + an optional *fmt_spec* from the stack, then a required *value*. + *flags* is interpreted as follows: + + * ``(flags & 0x03) == 0x00``: *value* is formatted as-is. + * ``(flags & 0x03) == 0x01``: call :func:`str` on *value* before + formatting it. + * ``(flags & 0x03) == 0x02``: call :func:`repr` on *value* before + formatting it. + * ``(flags & 0x03) == 0x03``: call :func:`ascii` on *value* before + formatting it. + * ``(flags & 0x04) == 0x04``: pop *fmt_spec* from the stack and use + it, else use an empty *fmt_spec*. + + Formatting is performed using :c:func:`PyObject_Format`. The + result is pushed on the stack. + + .. versionadded:: 3.6 + + .. opcode:: HAVE_ARGUMENT This is not really an opcode. It identifies the dividing line between diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index a3d5afcb9e..2111d1c04e 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -558,7 +558,8 @@ Some rules: 4. %-style formatting: `%s` and `%r` call the :class:`Enum` class's :meth:`__str__` and :meth:`__repr__` respectively; other codes (such as `%i` or `%h` for IntEnum) treat the enum member as its mixed-in type. -5. :meth:`str.format` (or :func:`format`) will use the mixed-in +5. :ref:`Formatted string literals <f-strings>`, :meth:`str.format`, + and :func:`format` will use the mixed-in type's :meth:`__format__`. If the :class:`Enum` class's :func:`str` or :func:`repr` is desired, use the `!s` or `!r` format codes. @@ -747,6 +748,15 @@ besides the :class:`Enum` member you looking for:: .. versionchanged:: 3.5 +Boolean evaluation: Enum classes that are mixed with non-Enum types (such as +:class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in +type's rules; otherwise, all members evaluate as ``True``. To make your own +Enum's boolean evaluation depend on the member's value add the following to +your class:: + + def __bool__(self): + return bool(self.value) + The :attr:`__members__` attribute is only available on the class. If you give your :class:`Enum` subclass extra methods, like the `Planet`_ diff --git a/Doc/library/faulthandler.rst b/Doc/library/faulthandler.rst index deedea1f26..d0c4cd07c5 100644 --- a/Doc/library/faulthandler.rst +++ b/Doc/library/faulthandler.rst @@ -70,6 +70,9 @@ Fault handler state .. versionchanged:: 3.5 Added support for passing file descriptor to this function. + .. versionchanged:: 3.6 + On Windows, a handler for Windows exception is also installed. + .. function:: disable() Disable the fault handler: uninstall the signal handlers installed by diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index aa4c529b2a..5881fefe29 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -72,9 +72,8 @@ The following function is the primary interface of this module: .. versionchanged:: 3.2 Can be used as a context manager. - .. versionchanged:: 3.5.2 - The *bufsize* parameter is no longer used. - + .. deprecated-removed:: 3.6 3.8 + The *bufsize* parameter. The following functions use the global state created by :func:`fileinput.input`; if there is no active state, :exc:`RuntimeError` is raised. @@ -167,8 +166,8 @@ available for subclassing as well: .. deprecated:: 3.4 The ``'rU'`` and ``'U'`` modes. - .. versionchanged:: 3.5.2 - The *bufsize* parameter is no longer used. + .. deprecated-removed:: 3.6 3.8 + The *bufsize* parameter. **Optional in-place filtering:** if the keyword argument ``inplace=True`` is @@ -195,10 +194,14 @@ The two following opening hooks are provided by this module: Usage example: ``fi = fileinput.FileInput(openhook=fileinput.hook_compressed)`` -.. function:: hook_encoded(encoding) +.. function:: hook_encoded(encoding, errors=None) Returns a hook which opens each file with :func:`open`, using the given - *encoding* to read the file. + *encoding* and *errors* to read the file. Usage example: ``fi = - fileinput.FileInput(openhook=fileinput.hook_encoded("iso-8859-1"))`` + fileinput.FileInput(openhook=fileinput.hook_encoded("utf-8", + "surrogateescape"))`` + + .. versionchanged:: 3.6 + Added the optional *errors* parameter. diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index e609463624..1f13afc1c5 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -878,11 +878,11 @@ are always available. They are listed here in alphabetical order. Open *file* and return a corresponding :term:`file object`. If the file cannot be opened, an :exc:`OSError` is raised. - *file* is either a string or bytes object giving the pathname (absolute or - relative to the current working directory) of the file to be opened or - an integer file descriptor of the file to be wrapped. (If a file descriptor - is given, it is closed when the returned I/O object is closed, unless - *closefd* is set to ``False``.) + *file* is either a string, bytes, or :class:`os.PathLike` object giving the + pathname (absolute or relative to the current working directory) of the file + to be opened or an integer file descriptor of the file to be wrapped. (If a + file descriptor is given, it is closed when the returned I/O object is + closed, unless *closefd* is set to ``False``.) *mode* is an optional string that specifies the mode in which the file is opened. It defaults to ``'r'`` which means open for reading in text mode. @@ -1077,6 +1077,9 @@ are always available. They are listed here in alphabetical order. .. versionchanged:: 3.5 The ``'namereplace'`` error handler was added. + .. versionchanged:: 3.6 + Support added to accept objects implementing :class:`os.PathLike`. + .. function:: ord(c) Given a string representing one Unicode character, return an integer diff --git a/Doc/library/grp.rst b/Doc/library/grp.rst index a30e6229db..74de3f9520 100644 --- a/Doc/library/grp.rst +++ b/Doc/library/grp.rst @@ -43,6 +43,9 @@ It defines the following items: Return the group database entry for the given numeric group ID. :exc:`KeyError` is raised if the entry asked for cannot be found. + .. deprecated:: 3.6 + Since Python 3.6 the support of non-integer arguments like floats or + strings in :func:`getgrgid` is deprecated. .. function:: getgrnam(name) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 30be3354af..93bcc91f91 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -39,8 +39,8 @@ Hash algorithms --------------- There is one constructor method named for each type of :dfn:`hash`. All return -a hash object with the same simple interface. For example: use :func:`sha1` to -create a SHA1 hash object. You can now feed this object with :term:`bytes-like +a hash object with the same simple interface. For example: use :func:`sha256` to +create a SHA-256 hash object. You can now feed this object with :term:`bytes-like objects <bytes-like object>` (normally :class:`bytes`) using the :meth:`update` method. At any point you can ask it for the :dfn:`digest` of the concatenation of the data fed to it so far using the :meth:`digest` or @@ -59,21 +59,23 @@ concatenation of the data fed to it so far using the :meth:`digest` or .. index:: single: OpenSSL; (use in module hashlib) Constructors for hash algorithms that are always present in this module are -:func:`md5`, :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, -and :func:`sha512`. Additional algorithms may also be available depending upon -the OpenSSL library that Python uses on your platform. +:func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, +and :func:`sha512`. :func:`md5` is normally available as well, though it +may be missing if you are using a rare "FIPS compliant" build of Python. +Additional algorithms may also be available depending upon the OpenSSL +library that Python uses on your platform. For example, to obtain the digest of the byte string ``b'Nobody inspects the spammish repetition'``:: >>> import hashlib - >>> m = hashlib.md5() + >>> m = hashlib.sha256() >>> m.update(b"Nobody inspects") >>> m.update(b" the spammish repetition") >>> m.digest() - b'\xbbd\x9c\x83\xdd\x1e\xa5\xc9\xd9\xde\xc9\xa1\x8d\xf0\xff\xe9' + b'\x03\x1e\xdd}Ae\x15\x93\xc5\xfe\\\x00o\xa5u+7\xfd\xdf\xf7\xbcN\x84:\xa6\xaf\x0c\x95\x0fK\x94\x06' >>> m.digest_size - 16 + 32 >>> m.block_size 64 @@ -102,7 +104,9 @@ Hashlib provides the following constant attributes: .. data:: algorithms_guaranteed A set containing the names of the hash algorithms guaranteed to be supported - by this module on all platforms. + by this module on all platforms. Note that 'md5' is in this list despite + some upstream vendors offering an odd "FIPS compliant" Python build that + excludes it. .. versionadded:: 3.2 diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst index 16c4fac947..c3584f5950 100644 --- a/Doc/library/http.server.rst +++ b/Doc/library/http.server.rst @@ -369,10 +369,9 @@ the current directory:: Handler = http.server.SimpleHTTPRequestHandler - httpd = socketserver.TCPServer(("", PORT), Handler) - - print("serving at port", PORT) - httpd.serve_forever() + with socketserver.TCPServer(("", PORT), Handler) as httpd: + print("serving at port", PORT) + httpd.serve_forever() .. _http-server-cli: diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index c25e7d8611..b9b3b91518 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -500,6 +500,17 @@ An :class:`IMAP4` instance has the following methods: M.store(num, '+FLAGS', '\\Deleted') M.expunge() + .. note:: + + Creating flags containing ']' (for example: "[test]") violates + :rfc:`3501` (the IMAP protocol). However, imaplib has historically + allowed creation of such tags, and popular IMAP servers, such as Gmail, + accept and produce such flags. There are non-Python programs which also + create such tags. Although it is an RFC violation and IMAP clients and + servers are supposed to be strict, imaplib nonetheless continues to allow + such tags to be created for backward compatibility reasons, and as of + python 3.6, handles them if they are sent from the server, since this + improves real-world compatibility. .. method:: IMAP4.subscribe(mailbox) diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst index 9828ba6fe2..ccf5f92d16 100644 --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -85,7 +85,9 @@ This module provides an interface to the mechanisms used to implement the .. deprecated:: 3.3 Use :func:`importlib.util.find_spec` instead unless Python 3.3 compatibility is required, in which case use - :func:`importlib.find_loader`. + :func:`importlib.find_loader`. For example usage of the former case, + see the :ref:`importlib-examples` section of the :mod:`importlib` + documentation. .. function:: load_module(name, file, pathname, description) @@ -112,9 +114,12 @@ This module provides an interface to the mechanisms used to implement the If previously used in conjunction with :func:`imp.find_module` then consider using :func:`importlib.import_module`, otherwise use the loader returned by the replacement you chose for :func:`imp.find_module`. If you - called :func:`imp.load_module` and related functions directly then use the - classes in :mod:`importlib.machinery`, e.g. - ``importlib.machinery.SourceFileLoader(name, path).load_module()``. + called :func:`imp.load_module` and related functions directly with file + path arguments then use a combination of + :func:`importlib.util.spec_from_file_location` and + :func:`importlib.util.module_from_spec`. See the :ref:`importlib-examples` + section of the :mod:`importlib` documentation for details of the various + approaches. .. function:: new_module(name) @@ -123,7 +128,7 @@ This module provides an interface to the mechanisms used to implement the in ``sys.modules``. .. deprecated:: 3.4 - Use :class:`types.ModuleType` instead. + Use :func:`importlib.util.module_from_spec` instead. .. function:: reload(module) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 3302446b40..873ec44a02 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -262,7 +262,7 @@ ABC hierarchy:: module and *path* will be the value of :attr:`__path__` from the parent package. If a spec cannot be found, ``None`` is returned. When passed in, ``target`` is a module object that the finder may - use to make a more educated about what spec to return. + use to make a more educated guess about what spec to return. .. versionadded:: 3.4 @@ -312,7 +312,7 @@ ABC hierarchy:: within the :term:`path entry` to which it is assigned. If a spec cannot be found, ``None`` is returned. When passed in, ``target`` is a module object that the finder may use to make a more educated - about what spec to return. + guess about what spec to return. .. versionadded:: 3.4 @@ -927,6 +927,10 @@ find and load modules. Concrete implementation of :meth:`importlib.abc.Loader.load_module` where specifying the name of the module to load is optional. + .. deprecated:: 3.6 + + Use :meth:`importlib.abc.Loader.exec_module` instead. + .. class:: SourcelessFileLoader(fullname, path) @@ -966,6 +970,10 @@ find and load modules. Concrete implementation of :meth:`importlib.abc.Loader.load_module` where specifying the name of the module to load is optional. + .. deprecated:: 3.6 + + Use :meth:`importlib.abc.Loader.exec_module` instead. + .. class:: ExtensionFileLoader(fullname, path) @@ -1306,3 +1314,120 @@ an :term:`importer`. loader = importlib.machinery.SourceFileLoader lazy_loader = importlib.util.LazyLoader.factory(loader) finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes)) + +.. _importlib-examples: + +Examples +-------- + +To programmatically import a module, use :func:`importlib.import_module`. +:: + + import importlib + + itertools = importlib.import_module('itertools') + +If you need to find out if a module can be imported without actually doing the +import, then you should use :func:`importlib.util.find_spec`. +:: + + import importlib.util + import sys + + # For illustrative purposes. + name = 'itertools' + + spec = importlib.util.find_spec(name) + if spec is None: + print("can't find the itertools module") + else: + # If you chose to perform the actual import ... + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + # Adding the module to sys.modules is optional. + sys.modules[name] = module + +To import a Python source file directly, use the following recipe +(Python 3.4 and newer only):: + + import importlib.util + import sys + + # For illustrative purposes. + import tokenize + file_path = tokenize.__file__ + module_name = tokenize.__name__ + + spec = importlib.util.spec_from_file_location(module_name, file_path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + # Optional; only necessary if you want to be able to import the module + # by name later. + sys.modules[module_name] = module + +For deep customizations of import, you typically want to implement an +:term:`importer`. This means managing both the :term:`finder` and :term:`loader` +side of things. For finders there are two flavours to choose from depending on +your needs: a :term:`meta path finder` or a :term:`path entry finder`. The +former is what you would put on :attr:`sys.meta_path` while the latter is what +you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works +with :attr:`sys.path` entries to potentially create a finder. This example will +show you how to register your own importers so that import will use them (for +creating an importer for yourself, read the documentation for the appropriate +classes defined within this package):: + + import importlib.machinery + import sys + + # For illustrative purposes only. + SpamMetaPathFinder = importlib.machinery.PathFinder + SpamPathEntryFinder = importlib.machinery.FileFinder + loader_details = (importlib.machinery.SourceFileLoader, + importlib.machinery.SOURCE_SUFFIXES) + + # Setting up a meta path finder. + # Make sure to put the finder in the proper location in the list in terms of + # priority. + sys.meta_path.append(SpamMetaPathFinder) + + # Setting up a path entry finder. + # Make sure to put the path hook in the proper location in the list in terms + # of priority. + sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details)) + +Import itself is implemented in Python code, making it possible to +expose most of the import machinery through importlib. The following +helps illustrate the various APIs that importlib exposes by providing an +approximate implementation of +:func:`importlib.import_module` (Python 3.4 and newer for the importlib usage, +Python 3.6 and newer for other parts of the code). +:: + + import importlib.util + import sys + + def import_module(name, package=None): + """An approximate implementation of import.""" + absolute_name = importlib.util.resolve_name(name, package) + try: + return sys.modules[absolute_name] + except KeyError: + pass + + path = None + if '.' in absolute_name: + parent_name, _, child_name = absolute_name.rpartition('.') + parent_module = import_module(parent_name) + path = parent_module.spec.submodule_search_locations + for finder in sys.meta_path: + spec = finder.find_spec(absolute_name, path) + if spec is not None: + break + else: + raise ImportError(f'No module named {absolute_name!r}') + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + sys.modules[absolute_name] = module + if path is not None: + setattr(parent_module, child_name, module) + return module diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 8e7ed19f29..5cb7c22adb 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -235,24 +235,6 @@ attributes: listed in the metaclass' custom :meth:`__dir__`. -.. function:: getmoduleinfo(path) - - Returns a :term:`named tuple` ``ModuleInfo(name, suffix, mode, module_type)`` - of values that describe how Python will interpret the file identified by - *path* if it is a module, or ``None`` if it would not be identified as a - module. In that tuple, *name* is the name of the module without the name of - any enclosing package, *suffix* is the trailing part of the file name (which - may not be a dot-delimited extension), *mode* is the :func:`open` mode that - would be used (``'r'`` or ``'rb'``), and *module_type* is an integer giving - the type of the module. *module_type* will have a value which can be - compared to the constants defined in the :mod:`imp` module; see the - documentation for that module for more information on module types. - - .. deprecated:: 3.3 - You may check the file path's suffix against the supported suffixes - listed in :mod:`importlib.machinery` to infer the same information. - - .. function:: getmodulename(path) Return the name of the module named by the file *path*, without including the @@ -266,8 +248,7 @@ attributes: still return ``None``. .. versionchanged:: 3.3 - This function is now based directly on :mod:`importlib` rather than the - deprecated :func:`getmoduleinfo`. + The function is based directly on :mod:`importlib`. .. function:: ismodule(object) @@ -646,6 +627,16 @@ function. The name of the parameter as a string. The name must be a valid Python identifier. + .. impl-detail:: + + CPython generates implicit parameter names of the form ``.0`` on the + code objects used to implement comprehensions and generator + expressions. + + .. versionchanged:: 3.6 + These parameter names are exposed by this module as names like + ``implicit0``. + .. attribute:: Parameter.default The default value for the parameter. If the parameter has no default @@ -854,8 +845,6 @@ Classes and functions from kwonlyargs to defaults. *annotations* is a dictionary mapping argument names to annotations. - The first four items in the tuple correspond to :func:`getargspec`. - .. versionchanged:: 3.4 This function is now based on :func:`signature`, but still ignores ``__wrapped__`` attributes and includes the already bound first @@ -884,7 +873,7 @@ Classes and functions .. function:: formatargspec(args[, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations[, formatarg, formatvarargs, formatvarkw, formatvalue, formatreturns, formatannotations]]) Format a pretty argument spec from the values returned by - :func:`getargspec` or :func:`getfullargspec`. + :func:`getfullargspec`. The first seven arguments are (``args``, ``varargs``, ``varkw``, ``defaults``, ``kwonlyargs``, ``kwonlydefaults``, ``annotations``). diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index dfc1ddc2d7..b0d0a8c8c8 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -591,7 +591,13 @@ loops that truncate the stream. .. function:: tee(iterable, n=2) - Return *n* independent iterators from a single iterable. Roughly equivalent to:: + Return *n* independent iterators from a single iterable. + + The following Python code helps explain what *tee* does (although the actual + implementation is more complex and uses only a single underlying + :abbr:`FIFO (first-in, first-out)` queue). + + Roughly equivalent to:: def tee(iterable, n=2): it = iter(iterable) diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 855adabf86..4eaea09bad 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -161,11 +161,19 @@ for this value. first call to :meth:`emit`. By default, the file grows indefinitely. + .. method:: reopenIfNeeded() + + Checks to see if the file has changed. If it has, the existing stream is + flushed and closed and the file opened again, typically as a precursor to + outputting the record to the file. + + .. versionadded:: 3.6 + + .. method:: emit(record) - Outputs the record to the file, but first checks to see if the file has - changed. If it has, the existing stream is flushed and closed and the - file opened again, before outputting the record to the file. + Outputs the record to the file, but first calls :meth:`reopenIfNeeded` to + reopen the file if it has changed. .. _base-rotating-handler: diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst index 8f538339aa..c544c809e9 100644 --- a/Doc/library/mmap.rst +++ b/Doc/library/mmap.rst @@ -264,13 +264,18 @@ To map anonymous memory, -1 should be passed as the fileno along with the length .. method:: write(bytes) Write the bytes in *bytes* into memory at the current position of the - file pointer; the file position is updated to point after the bytes that - were written. If the mmap was created with :const:`ACCESS_READ`, then + file pointer and return the number of bytes written (never less than + ``len(bytes)``, since if the write fails, a :exc:`ValueError` will be + raised). The file position is updated to point after the bytes that + were written. If the mmap was created with :const:`ACCESS_READ`, then writing to it will raise a :exc:`TypeError` exception. .. versionchanged:: 3.5 Writable :term:`bytes-like object` is now accepted. + .. versionchanged:: 3.6 + The number of bytes written is now returned. + .. method:: write_byte(byte) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index d20098ffa7..f886ecb4dd 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -647,8 +647,9 @@ primitives like locks. For passing messages one can use :func:`Pipe` (for a connection between two processes) or a queue (which allows multiple producers and consumers). -The :class:`Queue`, :class:`SimpleQueue` and :class:`JoinableQueue` types are multi-producer, -multi-consumer FIFO queues modelled on the :class:`queue.Queue` class in the +The :class:`Queue`, :class:`SimpleQueue` and :class:`JoinableQueue` types +are multi-producer, multi-consumer :abbr:`FIFO (first-in, first-out)` +queues modelled on the :class:`queue.Queue` class in the standard library. They differ in that :class:`Queue` lacks the :meth:`~queue.Queue.task_done` and :meth:`~queue.Queue.join` methods introduced into Python 2.5's :class:`queue.Queue` class. @@ -886,8 +887,13 @@ Miscellaneous .. function:: cpu_count() - Return the number of CPUs in the system. May raise - :exc:`NotImplementedError`. + Return the number of CPUs in the system. + + This number is not equivalent to the number of CPUs the current process can + use. The number of usable CPUs can be obtained with + ``len(os.sched_getaffinity(0))`` + + May raise :exc:`NotImplementedError`. .. seealso:: :func:`os.cpu_count` diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 743efb691a..62e4bdadac 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -178,6 +178,9 @@ process and user. .. versionadded:: 3.2 + .. versionchanged:: 3.6 + Support added to accept objects implementing :class:`os.PathLike`. + .. function:: fsdecode(filename) @@ -188,6 +191,36 @@ process and user. .. versionadded:: 3.2 + .. versionchanged:: 3.6 + Support added to accept objects implementing :class:`os.PathLike`. + + +.. function:: fspath(path) + + Return the file system representation of the path. + + If :class:`str` or :class:`bytes` is passed in, it is returned unchanged; + otherwise, the result of calling ``type(path).__fspath__`` is returned + (which is represented by :class:`os.PathLike`). All other types raise a + :exc:`TypeError`. + + .. versionadded:: 3.6 + + +.. class:: PathLike + + An :term:`abstract base class` for objects representing a file system path, + e.g. :class:`pathlib.PurePath`. + + .. versionadded:: 3.6 + + .. abstractmethod:: __fspath__() + + Return the file system path representation of the object. + + The method should only return a :class:`str` or :class:`bytes` object, + with the preference being for :class:`str`. + .. function:: getenv(key, default=None) @@ -1900,14 +1933,29 @@ features: :attr:`~DirEntry.path` attributes of each :class:`DirEntry` will be of the same type as *path*. + The :func:`scandir` iterator supports the :term:`context manager` protocol + and has the following method: + + .. method:: scandir.close() + + Close the iterator and free acquired resources. + + This is called automatically when the iterator is exhausted or garbage + collected, or when an error happens during iterating. However it + is advisable to call it explicitly or use the :keyword:`with` + statement. + + .. versionadded:: 3.6 + The following example shows a simple use of :func:`scandir` to display all the files (excluding directories) in the given *path* that don't start with ``'.'``. The ``entry.is_file()`` call will generally not make an additional system call:: - for entry in os.scandir(path): - if not entry.name.startswith('.') and entry.is_file(): - print(entry.name) + with os.scandir(path) as it: + for entry in it: + if not entry.name.startswith('.') and entry.is_file(): + print(entry.name) .. note:: @@ -1923,6 +1971,12 @@ features: .. versionadded:: 3.5 + .. versionadded:: 3.6 + Added support for the :term:`context manager` protocol and the + :func:`~scandir.close()` method. If a :func:`scandir` iterator is neither + exhausted nor explicitly closed a :exc:`ResourceWarning` will be emitted + in its destructor. + .. class:: DirEntry @@ -1943,6 +1997,9 @@ features: control over errors, you can catch :exc:`OSError` when calling one of the ``DirEntry`` methods and handle as appropriate. + To be directly usable as a path-like object, ``DirEntry`` implements the + :class:`os.PathLike` interface. + Attributes and methods on a ``DirEntry`` instance are as follows: .. attribute:: name @@ -2055,6 +2112,9 @@ features: .. versionadded:: 3.5 + .. versionchanged:: 3.6 + Added support for the :class:`os.PathLike` interface. + .. function:: stat(path, \*, dir_fd=None, follow_symlinks=True) @@ -3615,6 +3675,11 @@ Miscellaneous System Information Return the number of CPUs in the system. Returns None if undetermined. + This number is not equivalent to the number of CPUs the current process can + use. The number of usable CPUs can be obtained with + ``len(os.sched_getaffinity(0))`` + + .. versionadded:: 3.4 diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index f803fb6274..74c4807b34 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -35,12 +35,6 @@ Pure paths are useful in some special cases; for example: accessing the OS. In this case, instantiating one of the pure classes may be useful since those simply don't have any OS-accessing operations. -.. note:: - This module has been included in the standard library on a - :term:`provisional basis <provisional package>`. Backwards incompatible - changes (up to and including removal of the package) may occur if deemed - necessary by the core developers. - .. seealso:: :pep:`428`: The pathlib module -- object-oriented filesystem paths. @@ -111,7 +105,8 @@ we also call *flavours*: PurePosixPath('setup.py') Each element of *pathsegments* can be either a string representing a - path segment, or another path object:: + path segment, an object implementing the :class:`os.PathLike` interface + which returns a string, or another path object:: >>> PurePath('foo', 'some/path', 'bar') PurePosixPath('foo/some/path/bar') @@ -152,6 +147,12 @@ we also call *flavours*: to ``PurePosixPath('bar')``, which is wrong if ``foo`` is a symbolic link to another directory) + Pure path objects implement the :class:`os.PathLike` interface, allowing them + to be used anywhere the interface is accepted. + + .. versionchanged:: 3.6 + Added support for the :class:`os.PathLike` interface. + .. class:: PurePosixPath(*pathsegments) A subclass of :class:`PurePath`, this path flavour represents non-Windows @@ -199,7 +200,7 @@ Paths of a different flavour compare unequal and cannot be ordered:: >>> PureWindowsPath('foo') < PurePosixPath('foo') Traceback (most recent call last): File "<stdin>", line 1, in <module> - TypeError: unorderable types: PureWindowsPath() < PurePosixPath() + TypeError: '<' not supported between instances of 'PureWindowsPath' and 'PurePosixPath' Operators @@ -216,6 +217,14 @@ The slash operator helps create child paths, similarly to :func:`os.path.join`:: >>> '/usr' / q PurePosixPath('/usr/bin') +A path object can be used anywhere an object implementing :class:`os.PathLike` +is accepted:: + + >>> import os + >>> p = PurePath('/etc') + >>> os.fspath(p) + '/etc' + The string representation of a path is the raw filesystem path itself (in native form, e.g. with backslashes under Windows), which you can pass to any function taking a file path as a string:: diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index 0d64191d79..6e8430fa8e 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -492,7 +492,7 @@ methods: .. method:: object.__getnewargs_ex__() - In protocols 4 and newer, classes that implements the + In protocols 2 and newer, classes that implements the :meth:`__getnewargs_ex__` method can dictate the values passed to the :meth:`__new__` method upon unpickling. The method must return a pair ``(args, kwargs)`` where *args* is a tuple of positional arguments @@ -504,15 +504,22 @@ methods: class requires keyword-only arguments. Otherwise, it is recommended for compatibility to implement :meth:`__getnewargs__`. + .. versionchanged:: 3.6 + :meth:`__getnewargs_ex__` is now used in protocols 2 and 3. + .. method:: object.__getnewargs__() - This method serve a similar purpose as :meth:`__getnewargs_ex__` but - for protocols 2 and newer. It must return a tuple of arguments ``args`` - which will be passed to the :meth:`__new__` method upon unpickling. + This method serve a similar purpose as :meth:`__getnewargs_ex__`, but + supports only positional arguments. It must return a tuple of arguments + ``args`` which will be passed to the :meth:`__new__` method upon unpickling. + + :meth:`__getnewargs__` will not be called if :meth:`__getnewargs_ex__` is + defined. - In protocols 4 and newer, :meth:`__getnewargs__` will not be called if - :meth:`__getnewargs_ex__` is defined. + .. versionchanged:: 3.6 + Before Python 3.6, :meth:`__getnewargs__` was called instead of + :meth:`__getnewargs_ex__` in protocols 2 and 3. .. method:: object.__getstate__() diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 1cb0935377..e3eaa3f3bf 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -16,8 +16,9 @@ availability of thread support in Python; see the :mod:`threading` module. The module implements three types of queue, which differ only in the order in -which the entries are retrieved. In a FIFO queue, the first tasks added are -the first retrieved. In a LIFO queue, the most recently added entry is +which the entries are retrieved. In a :abbr:`FIFO (first-in, first-out)` +queue, the first tasks added are the first retrieved. In a +:abbr:`LIFO (last-in, first-out)` queue, the most recently added entry is the first retrieved (operating like a stack). With a priority queue, the entries are kept sorted (using the :mod:`heapq` module) and the lowest valued entry is retrieved first. @@ -27,14 +28,16 @@ The :mod:`queue` module defines the following classes and exceptions: .. class:: Queue(maxsize=0) - Constructor for a FIFO queue. *maxsize* is an integer that sets the upperbound + Constructor for a :abbr:`FIFO (first-in, first-out)` queue. *maxsize* is + an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite. .. class:: LifoQueue(maxsize=0) - Constructor for a LIFO queue. *maxsize* is an integer that sets the upperbound + Constructor for a :abbr:`LIFO (last-in, first-out)` queue. *maxsize* is + an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite. diff --git a/Doc/library/random.rst b/Doc/library/random.rst index df502a0aff..e7b81adb64 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -46,7 +46,8 @@ from sources provided by the operating system. .. warning:: The pseudo-random generators of this module should not be used for - security purposes. + security purposes. For security or cryptographic uses, see the + :mod:`secrets` module. Bookkeeping functions: diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 569b522332..dfbedd48c9 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -321,8 +321,9 @@ The special characters are: The special sequences consist of ``'\'`` and a character from the list below. -If the ordinary character is not on the list, then the resulting RE will match -the second character. For example, ``\$`` matches the character ``'$'``. +If the ordinary character is not an ASCII digit or an ASCII letter, then the +resulting RE will match the second character. For example, ``\$`` matches the +character ``'$'``. ``\number`` Matches the contents of the group of the same number. Groups are numbered @@ -442,9 +443,8 @@ three digits in length. .. versionchanged:: 3.3 The ``'\u'`` and ``'\U'`` escape sequences have been added. -.. deprecated-removed:: 3.5 3.6 - Unknown escapes consisting of ``'\'`` and ASCII letter now raise a - deprecation warning and will be forbidden in Python 3.6. +.. versionchanged:: 3.6 + Unknown escapes consisting of ``'\'`` and an ASCII letter now are errors. .. seealso:: @@ -532,11 +532,11 @@ form. current locale. The use of this flag is discouraged as the locale mechanism is very unreliable, and it only handles one "culture" at a time anyway; you should use Unicode matching instead, which is the default in Python 3 - for Unicode (str) patterns. This flag makes sense only with bytes patterns. + for Unicode (str) patterns. This flag can be used only with bytes patterns. - .. deprecated-removed:: 3.5 3.6 - Deprecated the use of :const:`re.LOCALE` with string patterns or - :const:`re.ASCII`. + .. versionchanged:: 3.6 + :const:`re.LOCALE` can be used only with bytes patterns and is + not compatible with :const:`re.ASCII`. .. data:: M @@ -742,9 +742,8 @@ form. .. versionchanged:: 3.5 Unmatched groups are replaced with an empty string. - .. deprecated-removed:: 3.5 3.6 - Unknown escapes consist of ``'\'`` and ASCII letter now raise a - deprecation warning and will be forbidden in Python 3.6. + .. versionchanged:: 3.6 + Unknown escapes consisting of ``'\'`` and an ASCII letter now are errors. .. function:: subn(pattern, repl, string, count=0, flags=0) diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst index 4d3c099ed2..37e400eceb 100644 --- a/Doc/library/readline.rst +++ b/Doc/library/readline.rst @@ -167,6 +167,20 @@ The following functions operate on a global history list: This calls :c:func:`add_history` in the underlying library. +.. function:: set_auto_history(enabled) + + Enable or disable automatic calls to :c:func:`add_history` when reading + input via readline. The *enabled* argument should be a Boolean value + that when true, enables auto history, and that when False, disables + auto history. + + .. versionadded:: 3.6 + + .. impl-detail:: + Auto history is enabled by default, and changes to this do not persist + across multiple sessions. + + Startup hooks ------------- diff --git a/Doc/library/secrets.rst b/Doc/library/secrets.rst new file mode 100644 index 0000000000..9bf848f911 --- /dev/null +++ b/Doc/library/secrets.rst @@ -0,0 +1,198 @@ +:mod:`secrets` --- Generate secure random numbers for managing secrets +====================================================================== + +.. module:: secrets + :synopsis: Generate secure random numbers for managing secrets. + +.. moduleauthor:: Steven D'Aprano <steve+python@pearwood.info> +.. sectionauthor:: Steven D'Aprano <steve+python@pearwood.info> +.. versionadded:: 3.6 + +.. testsetup:: + + from secrets import * + __name__ = '<doctest>' + +**Source code:** :source:`Lib/secrets.py` + +------------- + +The :mod:`secrets` module is used for generating cryptographically strong +random numbers suitable for managing data such as passwords, account +authentication, security tokens, and related secrets. + +In particularly, :mod:`secrets` should be used in preference to the +default pseudo-random number generator in the :mod:`random` module, which +is designed for modelling and simulation, not security or cryptography. + +.. seealso:: + + :pep:`506` + + +Random numbers +-------------- + +The :mod:`secrets` module provides access to the most secure source of +randomness that your operating system provides. + +.. class:: SystemRandom + + A class for generating random numbers using the highest-quality + sources provided by the operating system. See + :class:`random.SystemRandom` for additional details. + +.. function:: choice(sequence) + + Return a randomly-chosen element from a non-empty sequence. + +.. function:: randbelow(n) + + Return a random int in the range [0, *n*). + +.. function:: randbits(k) + + Return an int with *k* random bits. + + +Generating tokens +----------------- + +The :mod:`secrets` module provides functions for generating secure +tokens, suitable for applications such as password resets, +hard-to-guess URLs, and similar. + +.. function:: token_bytes([nbytes=None]) + + Return a random byte string containing *nbytes* number of bytes. + If *nbytes* is ``None`` or not supplied, a reasonable default is + used. + + .. doctest:: + + >>> token_bytes(16) #doctest:+SKIP + b'\xebr\x17D*t\xae\xd4\xe3S\xb6\xe2\xebP1\x8b' + + +.. function:: token_hex([nbytes=None]) + + Return a random text string, in hexadecimal. The string has *nbytes* + random bytes, each byte converted to two hex digits. If *nbytes* is + ``None`` or not supplied, a reasonable default is used. + + .. doctest:: + + >>> token_hex(16) #doctest:+SKIP + 'f9bf78b9a18ce6d46a0cd2b0b86df9da' + +.. function:: token_urlsafe([nbytes=None]) + + Return a random URL-safe text string, containing *nbytes* random + bytes. The text is Base64 encoded, so on average each byte results + in approximately 1.3 characters. If *nbytes* is ``None`` or not + supplied, a reasonable default is used. + + .. doctest:: + + >>> token_urlsafe(16) #doctest:+SKIP + 'Drmhze6EPcv0fN_81Bj-nA' + + +How many bytes should tokens use? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To be secure against +`brute-force attacks <https://en.wikipedia.org/wiki/Brute-force_attack>`_, +tokens need to have sufficient randomness. Unfortunately, what is +considered sufficient will necessarily increase as computers get more +powerful and able to make more guesses in a shorter period. As of 2015, +it is believed that 32 bytes (256 bits) of randomness is sufficient for +the typical use-case expected for the :mod:`secrets` module. + +For those who want to manage their own token length, you can explicitly +specify how much randomness is used for tokens by giving an :class:`int` +argument to the various ``token_*`` functions. That argument is taken +as the number of bytes of randomness to use. + +Otherwise, if no argument is provided, or if the argument is ``None``, +the ``token_*`` functions will use a reasonable default instead. + +.. note:: + + That default is subject to change at any time, including during + maintenance releases. + + +Other functions +--------------- + +.. function:: compare_digest(a, b) + + Return ``True`` if strings *a* and *b* are equal, otherwise ``False``, + in such a way as to reduce the risk of + `timing attacks <http://codahale.com/a-lesson-in-timing-attacks/>`_. + See :func:`hmac.compare_digest` for additional details. + + +Recipes and best practices +-------------------------- + +This section shows recipes and best practices for using :mod:`secrets` +to manage a basic level of security. + +Generate an eight-character alphanumeric password: + +.. testcode:: + + import string + alphabet = string.ascii_letters + string.digits + password = ''.join(choice(alphabet) for i in range(8)) + + +.. note:: + + Applications should not + `store passwords in a recoverable format <http://cwe.mitre.org/data/definitions/257.html>`_, + whether plain text or encrypted. They should be salted and hashed + using a cryptographically-strong one-way (irreversible) hash function. + + +Generate a ten-character alphanumeric password with at least one +lowercase character, at least one uppercase character, and at least +three digits: + +.. testcode:: + + import string + alphabet = string.ascii_letters + string.digits + while True: + password = ''.join(choice(alphabet) for i in range(10)) + if (any(c.islower() for c in password) + and any(c.isupper() for c in password) + and sum(c.isdigit() for c in password) >= 3): + break + + +Generate an `XKCD-style passphrase <http://xkcd.com/936/>`_: + +.. testcode:: + + # On standard Linux systems, use a convenient dictionary file. + # Other platforms may need to provide their own word-list. + with open('/usr/share/dict/words') as f: + words = [word.strip() for word in f] + password = ' '.join(choice(words) for i in range(4)) + + +Generate a hard-to-guess temporary URL containing a security token +suitable for password recovery applications: + +.. testcode:: + + url = 'https://mydomain.com/reset=' + token_urlsafe() + + + +.. + # This modeline must appear within the last ten lines of the file. + kate: indent-width 3; remove-trailing-space on; replace-tabs on; encoding utf-8; diff --git a/Doc/library/site.rst b/Doc/library/site.rst index 0a73f5abc6..43daf790b7 100644 --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -52,7 +52,8 @@ searched for site-packages; otherwise they won't. A path configuration file is a file whose name has the form :file:`{name}.pth` and exists in one of the four directories mentioned above; its contents are additional items (one per line) to be added to ``sys.path``. Non-existing items -are never added to ``sys.path``. No item is added to ``sys.path`` more than +are never added to ``sys.path``, and no check is made that the item refers to a +directory rather than a file. No item is added to ``sys.path`` more than once. Blank lines and lines beginning with ``#`` are skipped. Lines starting with ``import`` (followed by space or tab) are executed. diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst index 977f9a8748..ad6bd3cc06 100644 --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@ -29,7 +29,7 @@ SMTPServer Objects .. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432,\ - map=None, enable_SMTPUTF8=False, decode_data=True) + map=None, enable_SMTPUTF8=False, decode_data=False) Create a new :class:`SMTPServer` object, which binds to local address *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It @@ -45,20 +45,19 @@ SMTPServer Objects global socket map is used. *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined - in :RFC:`6531`) should be enabled. The default is ``False``. If set to - ``True``, *decode_data* must be ``False`` (otherwise an error is raised). + in :RFC:`6531`) should be enabled. The default is ``False``. When ``True``, ``SMTPUTF8`` is accepted as a parameter to the ``MAIL`` command and when present is passed to :meth:`process_message` in the - ``kwargs['mail_options']`` list. + ``kwargs['mail_options']`` list. *decode_data* and *enable_SMTPUTF8* + cannot be set to ``True`` at the same time. *decode_data* specifies whether the data portion of the SMTP transaction - should be decoded using UTF-8. The default is ``True`` for backward - compatibility reasons, but will change to ``False`` in Python 3.6; specify - the keyword value explicitly to avoid the :exc:`DeprecationWarning`. When - *decode_data* is set to ``False`` the server advertises the ``8BITMIME`` + should be decoded using UTF-8. When *decode_data* is ``False`` (the + default), the server advertises the ``8BITMIME`` extension (:rfc:`6152`), accepts the ``BODY=8BITMIME`` parameter to the ``MAIL`` command, and when present passes it to :meth:`process_message` - in the ``kwargs['mail_options']`` list. + in the ``kwargs['mail_options']`` list. *decode_data* and *enable_SMTPUTF8* + cannot be set to ``True`` at the same time. .. method:: process_message(peer, mailfrom, rcpttos, data, **kwargs) @@ -75,9 +74,8 @@ SMTPServer Objects will be a bytes object. *kwargs* is a dictionary containing additional information. It is empty - unless at least one of ``decode_data=False`` or ``enable_SMTPUTF8=True`` - was given as an init parameter, in which case it contains the following - keys: + if ``decode_data=True`` was given as an init argument, otherwise + it contains the following keys: *mail_options*: a list of all received parameters to the ``MAIL`` @@ -108,9 +106,12 @@ SMTPServer Objects *localaddr* and *remoteaddr* may now contain IPv6 addresses. .. versionadded:: 3.5 - the *decode_data* and *enable_SMTPUTF8* constructor arguments, and the - *kwargs* argument to :meth:`process_message` when one or more of these is - specified. + The *decode_data* and *enable_SMTPUTF8* constructor parameters, and the + *kwargs* parameter to :meth:`process_message` when *decode_data* is + ``False``. + + .. versionchanged:: 3.6 + *decode_data* is now ``False`` by default. DebuggingServer Objects @@ -150,7 +151,7 @@ SMTPChannel Objects ------------------- .. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432,\ - map=None, enable_SMTPUTF8=False, decode_data=True) + map=None, enable_SMTPUTF8=False, decode_data=False) Create a new :class:`SMTPChannel` object which manages the communication between the server and a single SMTP client. @@ -162,22 +163,25 @@ SMTPChannel Objects limit. *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined - in :RFC:`6531`) should be enabled. The default is ``False``. A - :exc:`ValueError` is raised if both *enable_SMTPUTF8* and *decode_data* are - set to ``True`` at the same time. + in :RFC:`6531`) should be enabled. The default is ``False``. + *decode_data* and *enable_SMTPUTF8* cannot be set to ``True`` at the same + time. A dictionary can be specified in *map* to avoid using a global socket map. *decode_data* specifies whether the data portion of the SMTP transaction - should be decoded using UTF-8. The default is ``True`` for backward - compatibility reasons, but will change to ``False`` in Python 3.6. Specify - the keyword value explicitly to avoid the :exc:`DeprecationWarning`. + should be decoded using UTF-8. The default is ``False``. + *decode_data* and *enable_SMTPUTF8* cannot be set to ``True`` at the same + time. To use a custom SMTPChannel implementation you need to override the :attr:`SMTPServer.channel_class` of your :class:`SMTPServer`. .. versionchanged:: 3.5 - the *decode_data* and *enable_SMTPUTF8* arguments were added. + The *decode_data* and *enable_SMTPUTF8* parameters were added. + + .. versionchanged:: 3.6 + *decode_data* is now ``False`` by default. The :class:`SMTPChannel` has the following instance variables: diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 48311c5820..c63c73aae1 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -329,12 +329,17 @@ Constants .. versionadded:: 3.3 -.. data:: SIO_* +.. data:: SIO_RCVALL + SIO_KEEPALIVE_VALS + SIO_LOOPBACK_FAST_PATH RCVALL_* Constants for Windows' WSAIoctl(). The constants are used as arguments to the :meth:`~socket.socket.ioctl` method of socket objects. + .. versionchanged:: 3.6 + ``SIO_LOOPBACK_FAST_PATH`` was added. + .. data:: TIPC_* @@ -872,6 +877,10 @@ to sockets. it is recommended to :meth:`close` them explicitly, or to use a :keyword:`with` statement around them. + .. versionchanged:: 3.6 + :exc:`OSError` is now raised if an error occurs when the underlying + :c:func:`close` call is made. + .. note:: :meth:`close()` releases the resource associated with a connection but @@ -992,6 +1001,12 @@ to sockets. On other platforms, the generic :func:`fcntl.fcntl` and :func:`fcntl.ioctl` functions may be used; they accept a socket object as their first argument. + Currently only the following control codes are supported: + ``SIO_RCVALL``, ``SIO_KEEPALIVE_VALS``, and ``SIO_LOOPBACK_FAST_PATH``. + + .. versionchanged:: 3.6 + ``SIO_LOOPBACK_FAST_PATH`` was added. + .. method:: socket.listen([backlog]) Enable a server to accept connections. If *backlog* is specified, it must diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst index 98d2c46701..dac928124f 100644 --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -52,11 +52,12 @@ handler class by subclassing the :class:`BaseRequestHandler` class and overriding its :meth:`~BaseRequestHandler.handle` method; this method will process incoming requests. Second, you must instantiate one of the server classes, passing it -the server's address and the request handler class. Then call the +the server's address and the request handler class. It is recommended to use +the server in a :keyword:`with` statement. Then call the :meth:`~BaseServer.handle_request` or :meth:`~BaseServer.serve_forever` method of the server object to process one or many requests. Finally, call :meth:`~BaseServer.server_close` -to close the socket. +to close the socket (unless you used a :keyword:`with` statement). When inheriting from :class:`ThreadingMixIn` for threaded connection behavior, you should explicitly declare how you want your threads to behave on an abrupt @@ -111,6 +112,8 @@ server classes. :class:`UDPServer`. Setting the various attributes also changes the behavior of the underlying server mechanism. + :class:`ForkingMixIn` and the Forking classes mentioned below are + only available on POSIX platforms that support :func:`~os.fork`. .. class:: ForkingTCPServer ForkingUDPServer @@ -304,7 +307,11 @@ Server Objects This function is called if the :meth:`~BaseRequestHandler.handle` method of a :attr:`RequestHandlerClass` instance raises an exception. The default action is to print the traceback to - standard output and continue handling further requests. + standard error and continue handling further requests. + + .. versionchanged:: 3.6 + Now only called for exceptions derived from the :exc:`Exception` + class. .. method:: handle_timeout() @@ -349,6 +356,11 @@ Server Objects default implementation always returns :const:`True`. + .. versionchanged:: 3.6 + Support for the :term:`context manager` protocol was added. Exiting the + context manager is equivalent to calling :meth:`server_close`. + + Request Handler Objects ----------------------- @@ -429,11 +441,10 @@ This is the server side:: HOST, PORT = "localhost", 9999 # Create the server, binding to localhost on port 9999 - server = socketserver.TCPServer((HOST, PORT), MyTCPHandler) - - # Activate the server; this will keep running until you - # interrupt the program with Ctrl-C - server.serve_forever() + with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server: + # Activate the server; this will keep running until you + # interrupt the program with Ctrl-C + server.serve_forever() An alternative request handler class that makes use of streams (file-like objects that simplify communication by providing the standard file interface):: @@ -521,8 +532,8 @@ This is the server side:: if __name__ == "__main__": HOST, PORT = "localhost", 9999 - server = socketserver.UDPServer((HOST, PORT), MyUDPHandler) - server.serve_forever() + with socketserver.UDPServer((HOST, PORT), MyUDPHandler) as server: + server.serve_forever() This is the client side:: @@ -581,22 +592,22 @@ An example for the :class:`ThreadingMixIn` class:: HOST, PORT = "localhost", 0 server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler) - ip, port = server.server_address + with server: + ip, port = server.server_address - # Start a thread with the server -- that thread will then start one - # more thread for each request - server_thread = threading.Thread(target=server.serve_forever) - # Exit the server thread when the main thread terminates - server_thread.daemon = True - server_thread.start() - print("Server loop running in thread:", server_thread.name) + # Start a thread with the server -- that thread will then start one + # more thread for each request + server_thread = threading.Thread(target=server.serve_forever) + # Exit the server thread when the main thread terminates + server_thread.daemon = True + server_thread.start() + print("Server loop running in thread:", server_thread.name) - client(ip, port, "Hello World 1") - client(ip, port, "Hello World 2") - client(ip, port, "Hello World 3") + client(ip, port, "Hello World 1") + client(ip, port, "Hello World 2") + client(ip, port, "Hello World 3") - server.shutdown() - server.server_close() + server.shutdown() The output of the example should look something like this:: @@ -610,3 +621,5 @@ The output of the example should look something like this:: The :class:`ForkingMixIn` class is used in the same way, except that the server will spawn a new process for each request. +Available only on POSIX platforms that support :func:`~os.fork`. + diff --git a/Doc/library/spwd.rst b/Doc/library/spwd.rst index fd3c9adee4..c6cad2a3c3 100644 --- a/Doc/library/spwd.rst +++ b/Doc/library/spwd.rst @@ -55,6 +55,9 @@ The following functions are defined: Return the shadow password database entry for the given user name. + .. versionchanged:: 3.6 + Raises a :exc:`PermissionError` instead of :exc:`KeyError` if the user + doesn't have privileges. .. function:: getspall() diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 605d8d36b7..2cd823e3ba 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -629,9 +629,16 @@ Cursor Objects .. attribute:: lastrowid This read-only attribute provides the rowid of the last modified row. It is - only set if you issued an ``INSERT`` statement using the :meth:`execute` - method. For operations other than ``INSERT`` or when :meth:`executemany` is - called, :attr:`lastrowid` is set to :const:`None`. + only set if you issued an ``INSERT`` or a ``REPLACE`` statement using the + :meth:`execute` method. For operations other than ``INSERT`` or + ``REPLACE`` or when :meth:`executemany` is called, :attr:`lastrowid` is + set to :const:`None`. + + If the ``INSERT`` or ``REPLACE`` statement failed to insert the previous + successful rowid is returned. + + .. versionchanged:: 3.6 + Added support for the ``REPLACE`` statement. .. attribute:: description diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index db1c2d0359..0ade1721e6 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1453,8 +1453,8 @@ multiple fragments. For more information on the ``str`` class and its methods, see :ref:`textseq` and the :ref:`string-methods` section below. To output - formatted strings, see the :ref:`formatstrings` section. In addition, - see the :ref:`stringservices` section. + formatted strings, see the :ref:`f-strings` and :ref:`formatstrings` + sections. In addition, see the :ref:`stringservices` section. .. index:: @@ -2056,8 +2056,8 @@ expression support in the :mod:`re` module). .. index:: single: formatting, string (%) single: interpolation, string (%) - single: string; formatting - single: string; interpolation + single: string; formatting, printf + single: string; interpolation, printf single: printf-style formatting single: sprintf-style formatting single: % formatting @@ -2067,9 +2067,10 @@ expression support in the :mod:`re` module). The formatting operations described here exhibit a variety of quirks that lead to a number of common errors (such as failing to display tuples and - dictionaries correctly). Using the newer :meth:`str.format` interface - helps avoid these errors, and also provides a generally more powerful, - flexible and extensible approach to formatting text. + dictionaries correctly). Using the newer :ref:`formatted + string literals <f-strings>` or the :meth:`str.format` interface + helps avoid these errors. These alternatives also provide more powerful, + flexible and extensible approaches to formatting text. String objects have one unique built-in operation: the ``%`` operator (modulo). This is also known as the string *formatting* or *interpolation* operator. diff --git a/Doc/library/string.rst b/Doc/library/string.rst index d5d24301b6..c421c72d75 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -188,7 +188,9 @@ Format String Syntax The :meth:`str.format` method and the :class:`Formatter` class share the same syntax for format strings (although in the case of :class:`Formatter`, -subclasses can define their own format string syntax). +subclasses can define their own format string syntax). The syntax is +related to that of :ref:`formatted string literals <f-strings>`, but +there are differences. Format strings contain "replacement fields" surrounded by curly braces ``{}``. Anything that is not contained in braces is considered literal text, which is @@ -283,7 +285,8 @@ Format Specification Mini-Language "Format specifications" are used within replacement fields contained within a format string to define how individual values are presented (see -:ref:`formatstrings`). They can also be passed directly to the built-in +:ref:`formatstrings` and :ref:`f-strings`). +They can also be passed directly to the built-in :func:`format` function. Each formattable type may define how the format specification is to be interpreted. @@ -308,7 +311,8 @@ The general form of a *standard format specifier* is: If a valid *align* value is specified, it can be preceded by a *fill* character that can be any character and defaults to a space if omitted. It is not possible to use a literal curly brace ("``{``" or "``}``") as -the *fill* character when using the :meth:`str.format` +the *fill* character in a :ref:`formatted string literal +<f-strings>` or when using the :meth:`str.format` method. However, it is possible to insert a curly brace with a nested replacement field. This limitation doesn't affect the :func:`format` function. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index f469107cba..3da4574c55 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -502,6 +502,10 @@ functions. .. versionchanged:: 3.2 Added context manager support. + .. versionchanged:: 3.6 + Popen destructor now emits a :exc:`ResourceWarning` warning if the child + process is still running. + Exceptions ^^^^^^^^^^ diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 9f70a13aa9..ae7af7c7ea 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -256,7 +256,7 @@ always available. (defaulting to zero), or another type of object. If it is an integer, zero is considered "successful termination" and any nonzero value is considered "abnormal termination" by shells and the like. Most systems require it to be - in the range 0-127, and produce undefined results otherwise. Some systems + in the range 0--127, and produce undefined results otherwise. Some systems have a convention for assigning specific meanings to specific exit codes, but these are generally underdeveloped; Unix programs generally use 2 for command line syntax errors and 1 for all other kind of errors. If another type of @@ -269,6 +269,11 @@ always available. the process when called from the main thread, and the exception is not intercepted. + .. versionchanged:: 3.6 + If an error occurs in the cleanup after the Python interpreter + has caught :exc:`SystemExit` (such as an error flushing buffered data + in the standard streams), the exit status is changed to 120. + .. data:: flags diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst index 0b0df9b375..a115beadde 100644 --- a/Doc/library/sysconfig.rst +++ b/Doc/library/sysconfig.rst @@ -164,7 +164,7 @@ Other functions .. function:: get_python_version() Return the ``MAJOR.MINOR`` Python version number as a string. Similar to - ``sys.version[:3]``. + ``'%d.%d' % sys.version_info[:2]``. .. function:: get_platform() diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index b950e41dc1..f9c5153e3f 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -43,6 +43,17 @@ Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation Begin). :exc:`EOFError` when the end of the connection is read, because they can return an empty string for other reasons. See the individual descriptions below. + A :class:`Telnet` object is a context manager and can be used in a + :keyword:`with` statement. When the :keyword:`with` block ends, the + :meth:`close` method is called:: + + >>> from telnetlib import Telnet + >>> with Telnet('localhost', 23) as tn: + ... tn.interact() + ... + + .. versionchanged:: 3.6 Context manager support added + .. seealso:: diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 2ea9c27e49..7114cdf8b4 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -582,6 +582,48 @@ The :mod:`test.support` module defines the following functions: .. versionadded:: 3.5 +.. function:: check__all__(test_case, module, name_of_module=None, extra=(), blacklist=()) + + Assert that the ``__all__`` variable of *module* contains all public names. + + The module's public names (its API) are detected automatically + based on whether they match the public name convention and were defined in + *module*. + + The *name_of_module* argument can specify (as a string or tuple thereof) what + module(s) an API could be defined in in order to be detected as a public + API. One case for this is when *module* imports part of its public API from + other modules, possibly a C backend (like ``csv`` and its ``_csv``). + + The *extra* argument can be a set of names that wouldn't otherwise be automatically + detected as "public", like objects without a proper ``__module__`` + attribute. If provided, it will be added to the automatically detected ones. + + The *blacklist* argument can be a set of names that must not be treated as part of + the public API even though their names indicate otherwise. + + Example use:: + + import bar + import foo + import unittest + from test import support + + class MiscTestCase(unittest.TestCase): + def test__all__(self): + support.check__all__(self, foo) + + class OtherTestCase(unittest.TestCase): + def test__all__(self): + extra = {'BAR_CONST', 'FOO_CONST'} + blacklist = {'baz'} # Undocumented name. + # bar imports part of its API from _bar. + support.check__all__(self, bar, ('bar', '_bar'), + extra=extra, blacklist=blacklist) + + .. versionadded:: 3.6 + + The :mod:`test.support` module defines the following classes: .. class:: TransientResource(exc, **kwargs) diff --git a/Doc/library/time.rst b/Doc/library/time.rst index e6626f262d..7c81ce7e30 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -637,11 +637,11 @@ The module defines the following functions and data items: it is possible to refer to February 29. :samp:`M{m}.{n}.{d}` - The *d*'th day (0 <= *d* <= 6) or week *n* of month *m* of the year (1 + The *d*'th day (0 <= *d* <= 6) of week *n* of month *m* of the year (1 <= *n* <= 5, 1 <= *m* <= 12, where week 5 means "the last *d* day in month *m*" which may occur in either the fourth or the fifth week). Week 1 is the first week in which the *d*'th day occurs. Day - zero is Sunday. + zero is a Sunday. ``time`` has the same format as ``offset`` except that no leading sign ('-' or '+') is allowed. The default, if time is not given, is 02:00:00. diff --git a/Doc/library/tracemalloc.rst b/Doc/library/tracemalloc.rst index 3a0b1e0797..f56f27b626 100644 --- a/Doc/library/tracemalloc.rst +++ b/Doc/library/tracemalloc.rst @@ -359,10 +359,32 @@ Functions See also the :func:`get_object_traceback` function. +DomainFilter +^^^^^^^^^^^^ + +.. class:: DomainFilter(inclusive: bool, domain: int) + + Filter traces of memory blocks by their address space (domain). + + .. versionadded:: 3.6 + + .. attribute:: inclusive + + If *inclusive* is ``True`` (include), match memory blocks allocated + in the address space :attr:`domain`. + + If *inclusive* is ``False`` (exclude), match memory blocks not allocated + in the address space :attr:`domain`. + + .. attribute:: domain + + Address space of a memory block (``int``). Read-only property. + + Filter ^^^^^^ -.. class:: Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False) +.. class:: Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False, domain: int=None) Filter on traces of memory blocks. @@ -382,9 +404,17 @@ Filter .. versionchanged:: 3.5 The ``'.pyo'`` file extension is no longer replaced with ``'.py'``. + .. versionchanged:: 3.6 + Added the :attr:`domain` attribute. + + + .. attribute:: domain + + Address space of a memory block (``int`` or ``None``). + .. attribute:: inclusive - If *inclusive* is ``True`` (include), only trace memory blocks allocated + If *inclusive* is ``True`` (include), only match memory blocks allocated in a file with a name matching :attr:`filename_pattern` at line number :attr:`lineno`. @@ -399,7 +429,7 @@ Filter .. attribute:: filename_pattern - Filename pattern of the filter (``str``). + Filename pattern of the filter (``str``). Read-only property. .. attribute:: all_frames @@ -462,14 +492,17 @@ Snapshot .. method:: filter_traces(filters) Create a new :class:`Snapshot` instance with a filtered :attr:`traces` - sequence, *filters* is a list of :class:`Filter` instances. If *filters* - is an empty list, return a new :class:`Snapshot` instance with a copy of - the traces. + sequence, *filters* is a list of :class:`DomainFilter` and + :class:`Filter` instances. If *filters* is an empty list, return a new + :class:`Snapshot` instance with a copy of the traces. All inclusive filters are applied at once, a trace is ignored if no inclusive filters match it. A trace is ignored if at least one exclusive filter matches it. + .. versionchanged:: 3.6 + :class:`DomainFilter` instances are now also accepted in *filters*. + .. classmethod:: load(filename) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 731e2eca16..3e9a006a7e 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -347,11 +347,15 @@ The module defines the following classes, functions and decorators: .. class:: Iterable(Generic[T_co]) - A generic version of the :class:`collections.abc.Iterable`. + A generic version of :class:`collections.abc.Iterable`. .. class:: Iterator(Iterable[T_co]) - A generic version of the :class:`collections.abc.Iterator`. + A generic version of :class:`collections.abc.Iterator`. + +.. class:: Reversible(Iterable[T_co]) + + A generic version of :class:`collections.abc.Reversible`. .. class:: SupportsInt @@ -371,11 +375,6 @@ The module defines the following classes, functions and decorators: An ABC with one abstract method ``__round__`` that is covariant in its return type. -.. class:: Reversible - - An ABC with one abstract method ``__reversed__`` returning - an ``Iterator[T_co]``. - .. class:: Container(Generic[T_co]) A generic version of :class:`collections.abc.Container`. @@ -396,7 +395,7 @@ The module defines the following classes, functions and decorators: A generic version of :class:`collections.abc.MutableMapping`. -.. class:: Sequence(Sized, Iterable[T_co], Container[T_co]) +.. class:: Sequence(Sized, Reversible[T_co], Container[T_co]) A generic version of :class:`collections.abc.Sequence`. @@ -451,6 +450,12 @@ The module defines the following classes, functions and decorators: A generic version of :class:`collections.abc.ValuesView`. +.. class:: ContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractContextManager`. + + .. versionadded:: 3.6 + .. class:: Dict(dict, MutableMapping[KT, VT]) A generic version of :class:`dict`. diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 5c9177a9c8..f559d71342 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -262,6 +262,34 @@ the *new_callable* argument to :func:`patch`. used to set attributes on the mock after it is created. See the :meth:`configure_mock` method for details. + .. method:: assert_called(*args, **kwargs) + + Assert that the mock was called at least once. + + >>> mock = Mock() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called() + + .. versionadded:: 3.6 + + .. method:: assert_called_once(*args, **kwargs) + + Assert that the mock was called exactly once. + + >>> mock = Mock() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called_once() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called_once() + Traceback (most recent call last): + ... + AssertionError: Expected 'method' to have been called once. Called 2 times. + + .. versionadded:: 3.6 + .. method:: assert_called_with(*args, **kwargs) @@ -339,7 +367,7 @@ the *new_callable* argument to :func:`patch`. .. versionadded:: 3.5 - .. method:: reset_mock() + .. method:: reset_mock(*, return_value=False, side_effect=False) The reset_mock method resets all the call attributes on a mock object: @@ -351,12 +379,20 @@ the *new_callable* argument to :func:`patch`. >>> mock.called False + .. versionchanged:: 3.6 + Added two keyword only argument to the reset_mock function. + This can be useful where you want to make a series of assertions that reuse the same object. Note that :meth:`reset_mock` *doesn't* clear the return value, :attr:`side_effect` or any child attributes you have - set using normal assignment. Child mocks and the return value mock + set using normal assignment by default. In case you want to reset + *return_value* or :attr:`side_effect`, then pass the corresponding + parameter as ``True``. Child mocks and the return value mock (if any) are reset as well. + .. note:: *return_value*, and :attr:`side_effect` are keyword only + argument. + .. method:: mock_add_spec(spec, spec_set=False) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 0fc02c4dbe..1d5f86944e 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1393,9 +1393,9 @@ Test cases Add a function to be called after :meth:`tearDown` to cleanup resources used during the test. Functions will be called in reverse order to the - order they are added (LIFO). They are called with any arguments and - keyword arguments passed into :meth:`addCleanup` when they are - added. + order they are added (:abbr:`LIFO (last-in, first-out)`). They + are called with any arguments and keyword arguments passed into + :meth:`addCleanup` when they are added. If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called, then any cleanup functions added will still be called. diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index c6de2303c6..d79d8f0e92 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -114,8 +114,9 @@ or on combining URL components into a URL string. | | | if present | | +------------------+-------+--------------------------+----------------------+ - See section :ref:`urlparse-result-object` for more information on the result - object. + Reading the :attr:`port` attribute will raise a :exc:`ValueError` if + an invalid port is specified in the URL. See section + :ref:`urlparse-result-object` for more information on the result object. .. versionchanged:: 3.2 Added IPv6 URL parsing capabilities. @@ -125,6 +126,10 @@ or on combining URL components into a URL string. false), in accordance with :rfc:`3986`. Previously, a whitelist of schemes that support fragments existed. + .. versionchanged:: 3.6 + Out-of-range port numbers now raise :exc:`ValueError`, instead of + returning :const:`None`. + .. function:: parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace') @@ -227,8 +232,13 @@ or on combining URL components into a URL string. | | | if present | | +------------------+-------+-------------------------+----------------------+ - See section :ref:`urlparse-result-object` for more information on the result - object. + Reading the :attr:`port` attribute will raise a :exc:`ValueError` if + an invalid port is specified in the URL. See section + :ref:`urlparse-result-object` for more information on the result object. + + .. versionchanged:: 3.6 + Out-of-range port numbers now raise :exc:`ValueError`, instead of + returning :const:`None`. .. function:: urlunsplit(parts) diff --git a/Doc/library/urllib.robotparser.rst b/Doc/library/urllib.robotparser.rst index ba701c3b68..7d31932f96 100644 --- a/Doc/library/urllib.robotparser.rst +++ b/Doc/library/urllib.robotparser.rst @@ -57,15 +57,41 @@ structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html. Sets the time the ``robots.txt`` file was last fetched to the current time. + .. method:: crawl_delay(useragent) -The following example demonstrates basic use of the RobotFileParser class. + Returns the value of the ``Crawl-delay`` parameter from ``robots.txt`` + for the *useragent* in question. If there is no such parameter or it + doesn't apply to the *useragent* specified or the ``robots.txt`` entry + for this parameter has invalid syntax, return ``None``. + + .. versionadded:: 3.6 + + .. method:: request_rate(useragent) + + Returns the contents of the ``Request-rate`` parameter from + ``robots.txt`` in the form of a :func:`~collections.namedtuple` + ``(requests, seconds)``. If there is no such parameter or it doesn't + apply to the *useragent* specified or the ``robots.txt`` entry for this + parameter has invalid syntax, return ``None``. + + .. versionadded:: 3.6 + + +The following example demonstrates basic use of the :class:`RobotFileParser` +class:: >>> import urllib.robotparser >>> rp = urllib.robotparser.RobotFileParser() >>> rp.set_url("http://www.musi-cal.com/robots.txt") >>> rp.read() + >>> rrate = rp.request_rate("*") + >>> rrate.requests + 3 + >>> rrate.seconds + 20 + >>> rp.crawl_delay("*") + 6 >>> rp.can_fetch("*", "http://www.musi-cal.com/cgi-bin/search?city=San+Francisco") False >>> rp.can_fetch("*", "http://www.musi-cal.com/") True - diff --git a/Doc/library/warnings.rst b/Doc/library/warnings.rst index 5212131764..fa47965ed9 100644 --- a/Doc/library/warnings.rst +++ b/Doc/library/warnings.rst @@ -300,7 +300,7 @@ Available Functions ------------------- -.. function:: warn(message, category=None, stacklevel=1) +.. function:: warn(message, category=None, stacklevel=1, source=None) Issue a warning, or maybe ignore it or raise an exception. The *category* argument, if given, must be a warning category class (see above); it defaults to @@ -318,8 +318,14 @@ Available Functions source of :func:`deprecation` itself (since the latter would defeat the purpose of the warning message). + *source*, if supplied, is the destroyed object which emitted a + :exc:`ResourceWarning`. -.. function:: warn_explicit(message, category, filename, lineno, module=None, registry=None, module_globals=None) + .. versionchanged:: 3.6 + Added *source* parameter. + + +.. function:: warn_explicit(message, category, filename, lineno, module=None, registry=None, module_globals=None, source=None) This is a low-level interface to the functionality of :func:`warn`, passing in explicitly the message, category, filename and line number, and optionally the @@ -335,6 +341,12 @@ Available Functions source for modules found in zipfiles or other non-filesystem import sources). + *source*, if supplied, is the destroyed object which emitted a + :exc:`ResourceWarning`. + + .. versionchanged:: 3.6 + Add the *source* parameter. + .. function:: showwarning(message, category, filename, lineno, file=None, line=None) diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index 52d591ad75..48bdf14bff 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -635,7 +635,7 @@ For more information, see `Registry Value Types .. data:: REG_DWORD_LITTLE_ENDIAN - A 32-bit number in little-endian format. + A 32-bit number in little-endian format. Equivalent to :const:`REG_DWORD`. .. data:: REG_DWORD_BIG_ENDIAN @@ -659,6 +659,18 @@ For more information, see `Registry Value Types No defined value type. +.. data:: REG_QWORD + + A 64-bit number. + + .. versionadded:: 3.6 + +.. data:: REG_QWORD_LITTLE_ENDIAN + + A 64-bit number in little-endian format. Equivalent to :const:`REG_QWORD`. + + .. versionadded:: 3.6 + .. data:: REG_RESOURCE_LIST A device-driver resource list. diff --git a/Doc/library/wsgiref.rst b/Doc/library/wsgiref.rst index aad27a88c9..a1d446902b 100644 --- a/Doc/library/wsgiref.rst +++ b/Doc/library/wsgiref.rst @@ -133,9 +133,9 @@ parameter expect a WSGI-compliant dictionary to be supplied; please see for key, value in environ.items()] return ret - httpd = make_server('', 8000, simple_app) - print("Serving on port 8000...") - httpd.serve_forever() + with make_server('', 8000, simple_app) as httpd: + print("Serving on port 8000...") + httpd.serve_forever() In addition to the environment functions above, the :mod:`wsgiref.util` module @@ -285,14 +285,14 @@ request. (E.g., using the :func:`shift_path_info` function from from wsgiref.simple_server import make_server, demo_app - httpd = make_server('', 8000, demo_app) - print("Serving HTTP on port 8000...") + with make_server('', 8000, demo_app) as httpd: + print("Serving HTTP on port 8000...") - # Respond to requests until process is killed - httpd.serve_forever() + # Respond to requests until process is killed + httpd.serve_forever() - # Alternative: serve one request, then exit - httpd.handle_request() + # Alternative: serve one request, then exit + httpd.handle_request() .. function:: demo_app(environ, start_response) @@ -432,9 +432,9 @@ Paste" library. # This is the application wrapped in a validator validator_app = validator(simple_app) - httpd = make_server('', 8000, validator_app) - print("Listening on port 8000....") - httpd.serve_forever() + with make_server('', 8000, validator_app) as httpd: + print("Listening on port 8000....") + httpd.serve_forever() :mod:`wsgiref.handlers` -- server/gateway base classes @@ -774,8 +774,8 @@ This is a working "Hello World" WSGI application:: # The returned object is going to be printed return [b"Hello World"] - httpd = make_server('', 8000, hello_world_app) - print("Serving on port 8000...") + with make_server('', 8000, hello_world_app) as httpd: + print("Serving on port 8000...") - # Serve until process is killed - httpd.serve_forever() + # Serve until process is killed + httpd.serve_forever() diff --git a/Doc/library/xmlrpc.server.rst b/Doc/library/xmlrpc.server.rst index ad1e81203b..2b9107f052 100644 --- a/Doc/library/xmlrpc.server.rst +++ b/Doc/library/xmlrpc.server.rst @@ -148,29 +148,29 @@ Server code:: rpc_paths = ('/RPC2',) # Create server - server = SimpleXMLRPCServer(("localhost", 8000), - requestHandler=RequestHandler) - server.register_introspection_functions() + with SimpleXMLRPCServer(("localhost", 8000), + requestHandler=RequestHandler) as server: + server.register_introspection_functions() - # Register pow() function; this will use the value of - # pow.__name__ as the name, which is just 'pow'. - server.register_function(pow) + # Register pow() function; this will use the value of + # pow.__name__ as the name, which is just 'pow'. + server.register_function(pow) - # Register a function under a different name - def adder_function(x,y): - return x + y - server.register_function(adder_function, 'add') + # Register a function under a different name + def adder_function(x,y): + return x + y + server.register_function(adder_function, 'add') - # Register an instance; all the methods of the instance are - # published as XML-RPC methods (in this case, just 'mul'). - class MyFuncs: - def mul(self, x, y): - return x * y + # Register an instance; all the methods of the instance are + # published as XML-RPC methods (in this case, just 'mul'). + class MyFuncs: + def mul(self, x, y): + return x * y - server.register_instance(MyFuncs()) + server.register_instance(MyFuncs()) - # Run the server's main loop - server.serve_forever() + # Run the server's main loop + server.serve_forever() The following client code will call the methods made available by the preceding server:: @@ -207,18 +207,17 @@ a server allowing dotted names and registering a multicall function. def getCurrentTime(): return datetime.datetime.now() - server = SimpleXMLRPCServer(("localhost", 8000)) - server.register_function(pow) - server.register_function(lambda x,y: x+y, 'add') - server.register_instance(ExampleService(), allow_dotted_names=True) - server.register_multicall_functions() - print('Serving XML-RPC on localhost port 8000') - try: - server.serve_forever() - except KeyboardInterrupt: - print("\nKeyboard interrupt received, exiting.") - server.server_close() - sys.exit(0) + with SimpleXMLRPCServer(("localhost", 8000)) as server: + server.register_function(pow) + server.register_function(lambda x,y: x+y, 'add') + server.register_instance(ExampleService(), allow_dotted_names=True) + server.register_multicall_functions() + print('Serving XML-RPC on localhost port 8000') + try: + server.serve_forever() + except KeyboardInterrupt: + print("\nKeyboard interrupt received, exiting.") + sys.exit(0) This ExampleService demo can be invoked from the command line:: diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst index a773769e44..c3e7aa0865 100644 --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -205,18 +205,13 @@ ZipFile Objects Return a list of archive members by name. -.. index:: - single: universal newlines; zipfile.ZipFile.open method +.. method:: ZipFile.open(name, mode='r', pwd=None, *, force_zip64=False) -.. method:: ZipFile.open(name, mode='r', pwd=None) - - Extract a member from the archive as a file-like object (ZipExtFile). *name* - is the name of the file in the archive, or a :class:`ZipInfo` object. The - *mode* parameter, if included, must be one of the following: ``'r'`` (the - default), ``'U'``, or ``'rU'``. Choosing ``'U'`` or ``'rU'`` will enable - :term:`universal newlines` support in the read-only object. *pwd* is the - password used for encrypted files. Calling :meth:`.open` on a closed - ZipFile will raise a :exc:`RuntimeError`. + Access a member of the archive as a binary file-like object. *name* + can be either the name of a file within the archive or a :class:`ZipInfo` + object. The *mode* parameter, if included, must be ``'r'`` (the default) + or ``'w'``. *pwd* is the password used to decrypt encrypted ZIP files. + Calling :meth:`.open` on a closed ZipFile will raise a :exc:`RuntimeError`. :meth:`~ZipFile.open` is also a context manager and therefore supports the :keyword:`with` statement:: @@ -225,17 +220,23 @@ ZipFile Objects with myzip.open('eggs.txt') as myfile: print(myfile.read()) - .. note:: - - The file-like object is read-only and provides the following methods: - :meth:`~io.BufferedIOBase.read`, :meth:`~io.IOBase.readline`, - :meth:`~io.IOBase.readlines`, :meth:`__iter__`, - :meth:`~iterator.__next__`. + With *mode* ``'r'`` the file-like object + (``ZipExtFile``) is read-only and provides the following methods: + :meth:`~io.BufferedIOBase.read`, :meth:`~io.IOBase.readline`, + :meth:`~io.IOBase.readlines`, :meth:`__iter__`, + :meth:`~iterator.__next__`. These objects can operate independently of + the ZipFile. - .. note:: + With ``mode='w'``, a writable file handle is returned, which supports the + :meth:`~io.BufferedIOBase.write` method. While a writable file handle is open, + attempting to read or write other files in the ZIP file will raise a + :exc:`RuntimeError`. - Objects returned by :meth:`.open` can operate independently of the - ZipFile. + When writing a file, if the file size is not known in advance but may exceed + 2 GiB, pass ``force_zip64=True`` to ensure that the header format is + capable of supporting large files. If the file size is known in advance, + construct a :class:`ZipInfo` object with :attr:`~ZipInfo.file_size` set, and + use that as the *name* parameter. .. note:: @@ -243,10 +244,14 @@ ZipFile Objects or a :class:`ZipInfo` object. You will appreciate this when trying to read a ZIP file that contains members with duplicate names. - .. deprecated-removed:: 3.4 3.6 - The ``'U'`` or ``'rU'`` mode. Use :class:`io.TextIOWrapper` for reading + .. versionchanged:: 3.6 + Removed support of ``mode='U'``. Use :class:`io.TextIOWrapper` for reading compressed text files in :term:`universal newlines` mode. + .. versionchanged:: 3.6 + :meth:`open` can now be used to write files into the archive with the + ``mode='w'`` option. + .. method:: ZipFile.extract(member, path=None, pwd=None) Extract a member from the archive to the current working directory; *member* @@ -465,7 +470,31 @@ Instances of the :class:`ZipInfo` class are returned by the :meth:`.getinfo` and :meth:`.infolist` methods of :class:`ZipFile` objects. Each object stores information about a single member of the ZIP archive. -Instances have the following attributes: +There is one classmethod to make a :class:`ZipInfo` instance for a filesystem +file: + +.. classmethod:: ZipInfo.from_file(filename, arcname=None) + + Construct a :class:`ZipInfo` instance for a file on the filesystem, in + preparation for adding it to a zip file. + + *filename* should be the path to a file or directory on the filesystem. + + If *arcname* is specified, it is used as the name within the archive. + If *arcname* is not specified, the name will be the same as *filename*, but + with any drive letter and leading path separators removed. + + .. versionadded:: 3.6 + +Instances have the following methods and attributes: + +.. method:: ZipInfo.is_dir() + + Return True if this archive member is a directory. + + This uses the entry's name: directories should always end with ``/``. + + .. versionadded:: 3.6 .. attribute:: ZipInfo.filename @@ -574,4 +603,5 @@ Instances have the following attributes: Size of the uncompressed file. + .. _PKZIP Application Note: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst index 1de7baee54..111fb9ec90 100644 --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -47,14 +47,19 @@ The available exception and functions in this module are: platforms, use ``adler32(data) & 0xffffffff``. -.. function:: compress(data[, level]) +.. function:: compress(data, level=-1) Compresses the bytes in *data*, returning a bytes object containing compressed data. - *level* is an integer from ``0`` to ``9`` controlling the level of compression; + *level* is an integer from ``0`` to ``9`` or ``-1`` controlling the level of compression; ``1`` is fastest and produces the least compression, ``9`` is slowest and - produces the most. ``0`` is no compression. The default value is ``6``. + produces the most. ``0`` is no compression. The default value is ``-1`` + (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default + compromise between speed and compression (currently equivalent to level 6). Raises the :exc:`error` exception if any error occurs. + .. versionchanged:: 3.6 + Keyword arguments are now supported. + .. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) |