diff options
Diffstat (limited to 'Doc/library')
-rw-r--r-- | Doc/library/argparse.rst | 9 | ||||
-rw-r--r-- | Doc/library/email.encoders.rst | 4 | ||||
-rw-r--r-- | Doc/library/functions.rst | 7 | ||||
-rw-r--r-- | Doc/library/imp.rst | 4 | ||||
-rw-r--r-- | Doc/library/json.rst | 8 | ||||
-rw-r--r-- | Doc/library/logging.handlers.rst | 2 | ||||
-rw-r--r-- | Doc/library/logging.rst | 21 | ||||
-rw-r--r-- | Doc/library/markup.rst | 4 | ||||
-rw-r--r-- | Doc/library/operator.rst | 2 | ||||
-rw-r--r-- | Doc/library/re.rst | 142 | ||||
-rw-r--r-- | Doc/library/signal.rst | 92 | ||||
-rw-r--r-- | Doc/library/sqlite3.rst | 16 | ||||
-rw-r--r-- | Doc/library/stdtypes.rst | 9 | ||||
-rw-r--r-- | Doc/library/subprocess.rst | 2 | ||||
-rw-r--r-- | Doc/library/syslog.rst | 3 | ||||
-rw-r--r-- | Doc/library/threading.rst | 307 | ||||
-rw-r--r-- | Doc/library/time.rst | 4 | ||||
-rw-r--r-- | Doc/library/unittest.rst | 2 | ||||
-rw-r--r-- | Doc/library/urllib.parse.rst | 7 | ||||
-rw-r--r-- | Doc/library/urllib.request.rst | 103 | ||||
-rw-r--r-- | Doc/library/webbrowser.rst | 2 | ||||
-rw-r--r-- | Doc/library/xml.dom.minidom.rst | 8 | ||||
-rw-r--r-- | Doc/library/xml.dom.pulldom.rst | 87 | ||||
-rw-r--r-- | Doc/library/xml.etree.elementtree.rst | 13 |
24 files changed, 498 insertions, 360 deletions
diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 79a98cbcfd..0123b5cbac 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1642,8 +1642,8 @@ Argument groups --bar BAR bar help - Note that any arguments not your user defined groups will end up back in the - usual "positional arguments" and "optional arguments" sections. + Note that any arguments not in your user-defined groups will end up back + in the usual "positional arguments" and "optional arguments" sections. Mutual exclusion @@ -1833,9 +1833,10 @@ A partial upgrade path from :mod:`optparse` to :mod:`argparse`: * Replace all :meth:`optparse.OptionParser.add_option` calls with :meth:`ArgumentParser.add_argument` calls. -* Replace ``options, args = parser.parse_args()`` with ``args = +* Replace ``(options, args) = parser.parse_args()`` with ``args = parser.parse_args()`` and add additional :meth:`ArgumentParser.add_argument` - calls for the positional arguments. + calls for the positional arguments. Keep in mind that what was previously + called ``options``, now in :mod:`argparse` context is called ``args``. * Replace callback actions and the ``callback_*`` keyword arguments with ``type`` or ``action`` arguments. diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 5421b9f66f..81d30961db 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -18,6 +18,10 @@ exactly one argument, the message object to encode. They usually extract the payload, encode it, and reset the payload to this newly encoded value. They should also set the :mailheader:`Content-Transfer-Encoding` header as appropriate. +Note that these functions are not meaningful for a multipart message. They +must be applied to individual subparts instead, and will raise a +:exc:`TypeError` if passed a message whose type is multipart. + Here are the encoding functions provided: diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index f835dcfc71..3fcd6941c2 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -247,6 +247,13 @@ are always available. They are listed here in alphabetical order. the function serves as a numeric conversion function like :func:`int` and :func:`float`. If both arguments are omitted, returns ``0j``. + .. note:: + + When converting from a string, the string must not contain whitespace + around the central ``+`` or ``-`` operator. For example, + ``complex('1+2j')`` is fine, but ``complex('1 + 2j')`` raises + :exc:`ValueError`. + The complex type is described in :ref:`typesnumeric`. diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst index 6e9845ed00..1345b254d6 100644 --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -64,7 +64,7 @@ This module provides an interface to the mechanisms used to implement the path and the last item in the *description* tuple is :const:`PKG_DIRECTORY`. This function does not handle hierarchical module names (names containing - dots). In order to find *P*.*M*, that is, submodule *M* of package *P*, use + dots). In order to find *P.M*, that is, submodule *M* of package *P*, use :func:`find_module` and :func:`load_module` to find and load package *P*, and then use :func:`find_module` with the *path* argument set to ``P.__path__``. When *P* itself has a dotted name, apply this recipe recursively. @@ -256,7 +256,7 @@ to indicate the search result of :func:`find_module`. .. data:: PY_FROZEN - The module was found as a frozen module (see :func:`init_frozen`). + The module was found as a frozen module. .. class:: NullImporter(path_string) diff --git a/Doc/library/json.rst b/Doc/library/json.rst index a791259831..f656700887 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -168,6 +168,14 @@ Basic Usage so trying to serialize multiple objects with repeated calls to :func:`dump` using the same *fp* will result in an invalid JSON file. + .. note:: + + Keys in key/value pairs of JSON are always of the type :class:`str`. When + a dictionary is converted into JSON, all the keys of the dictionary are + coerced to strings. As a result of this, if a dictionary is convered + into JSON and then back into a dictionary, the dictionary may not equal + the original one. That is, ``loads(dumps(x)) != x`` if x has non-string + keys. .. function:: load(fp, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw) diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index c4dd438f5b..ef65cfa559 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -654,7 +654,7 @@ event of a certain severity or greater is seen. :class:`BufferingHandler`, which is an abstract class. This buffers logging records in memory. Whenever each record is added to the buffer, a check is made by calling :meth:`shouldFlush` to see if the buffer should be flushed. If it -should, then :meth:`flush` is expected to do the needful. +should, then :meth:`flush` is expected to do the flushing. .. class:: BufferingHandler(capacity) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index c429c85985..dc97726258 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -49,9 +49,22 @@ listed below. Logger Objects -------------- -Loggers have the following attributes and methods. Note that Loggers are never +Loggers have the following attributes and methods. Note that Loggers are never instantiated directly, but always through the module-level function -``logging.getLogger(name)``. +``logging.getLogger(name)``. Multiple calls to :func:`getLogger` with the same +name will always return a reference to the same Logger object. + +The ``name`` is potentially a period-separated hierarchical value, like +``foo.bar.baz`` (though it could also be just plain ``foo``, for example). +Loggers that are further down in the hierarchical list are children of loggers +higher up in the list. For example, given a logger with a name of ``foo``, +loggers with names of ``foo.bar``, ``foo.bar.baz``, and ``foo.bam`` are all +descendants of ``foo``. The logger name hierarchy is analogous to the Python +package hierarchy, and identical to it if you organise your loggers on a +per-module basis using the recommended construction +``logging.getLogger(__name__)``. That's because in a module, ``__name__`` +is the module's name in the Python package namespace. + .. class:: Logger @@ -159,7 +172,7 @@ instantiated directly, but always through the module-level function FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s' logging.basicConfig(format=FORMAT) - d = { 'clientip' : '192.168.0.1', 'user' : 'fbloggs' } + d = {'clientip': '192.168.0.1', 'user': 'fbloggs'} logger = logging.getLogger('tcpserver') logger.warning('Protocol problem: %s', 'connection reset', extra=d) @@ -1077,7 +1090,7 @@ with the :mod:`warnings` module. If *capture* is ``True``, warnings issued by the :mod:`warnings` module will be redirected to the logging system. Specifically, a warning will be formatted using :func:`warnings.formatwarning` and the resulting string - logged to a logger named ``'py.warnings'`` with a severity of ``'WARNING'``. + logged to a logger named ``'py.warnings'`` with a severity of :const:`WARNING`. If *capture* is ``False``, the redirection of warnings to the logging system will stop, and warnings will be redirected to their original destinations diff --git a/Doc/library/markup.rst b/Doc/library/markup.rst index 49794ef707..1b4cca51a7 100644 --- a/Doc/library/markup.rst +++ b/Doc/library/markup.rst @@ -23,7 +23,7 @@ definition of the Python bindings for the DOM and SAX interfaces. html.rst html.parser.rst html.entities.rst - pyexpat.rst + xml.etree.elementtree.rst xml.dom.rst xml.dom.minidom.rst xml.dom.pulldom.rst @@ -31,4 +31,4 @@ definition of the Python bindings for the DOM and SAX interfaces. xml.sax.handler.rst xml.sax.utils.rst xml.sax.reader.rst - xml.etree.elementtree.rst + pyexpat.rst diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index b03d9df208..9ba7f41104 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -340,7 +340,7 @@ Python syntax and the functions in the :mod:`operator` module. +-----------------------+-------------------------+---------------------------------------+ | Containment Test | ``obj in seq`` | ``contains(seq, obj)`` | +-----------------------+-------------------------+---------------------------------------+ -| Division | ``a / b`` | ``div(a, b)`` | +| Division | ``a / b`` | ``truediv(a, b)`` | +-----------------------+-------------------------+---------------------------------------+ | Division | ``a // b`` | ``floordiv(a, b)`` | +-----------------------+-------------------------+---------------------------------------+ diff --git a/Doc/library/re.rst b/Doc/library/re.rst index b196a28f9d..b4e0557011 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -330,16 +330,22 @@ the second character. For example, ``\$`` matches the character ``'$'``. Matches the empty string, but only at the beginning or end of a word. A word is defined as a sequence of Unicode alphanumeric or underscore characters, so the end of a word is indicated by whitespace or a - non-alphanumeric, non-underscore Unicode character. Note that - formally, ``\b`` is defined as the boundary between a ``\w`` and a - ``\W`` character (or vice versa). By default Unicode alphanumerics - are the ones used, but this can be changed by using the :const:`ASCII` - flag. Inside a character range, ``\b`` represents the backspace - character, for compatibility with Python's string literals. + non-alphanumeric, non-underscore Unicode character. Note that formally, + ``\b`` is defined as the boundary between a ``\w`` and a ``\W`` character + (or vice versa), or between ``\w`` and the beginning/end of the string. + This means that ``r'\bfoo\b'`` matches ``'foo'``, ``'foo.'``, ``'(foo)'``, + ``'bar foo baz'`` but not ``'foobar'`` or ``'foo3'``. + + By default Unicode alphanumerics are the ones used, but this can be changed + by using the :const:`ASCII` flag. Inside a character range, ``\b`` + represents the backspace character, for compatibility with Python's string + literals. ``\B`` - Matches the empty string, but only when it is *not* at the beginning or end of a - word. This is just the opposite of ``\b``, so word characters are + Matches the empty string, but only when it is *not* at the beginning or end + of a word. This means that ``r'py\B'`` matches ``'python'``, ``'py3'``, + ``'py2'``, but not ``'py'``, ``'py.'``, or ``'py!'``. + ``\B`` is just the opposite of ``\b``, so word characters are Unicode alphanumerics or the underscore, although this can be changed by using the :const:`ASCII` flag. @@ -417,31 +423,6 @@ a group reference. As for string literals, octal escapes are always at most three digits in length. -.. _matching-searching: - -Matching vs. Searching ----------------------- - -.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> - - -Python offers two different primitive operations based on regular expressions: -**match** checks for a match only at the beginning of the string, while -**search** checks for a match anywhere in the string (this is what Perl does -by default). - -Note that match may differ from search even when using a regular expression -beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in -:const:`MULTILINE` mode also immediately following a newline. The "match" -operation succeeds only if the pattern matches at the start of the string -regardless of mode, or at the starting position given by the optional *pos* -argument regardless of whether a newline precedes it. - - >>> re.match("c", "abcdef") # No match - >>> re.search("c", "abcdef") # Match - <_sre.SRE_Match object at ...> - - .. _contents-of-module-re: Module Contents @@ -575,10 +556,11 @@ form. <match-objects>`. Return ``None`` if the string does not match the pattern; note that this is different from a zero-length match. - .. note:: + Note that even in :const:`MULTILINE` mode, :func:`re.match` will only match + at the beginning of the string and not at the beginning of each line. - If you want to locate a match anywhere in *string*, use :func:`search` - instead. + If you want to locate a match anywhere in *string*, use :func:`search` + instead (see also :ref:`search-vs-match`). .. function:: split(pattern, string, maxsplit=0, flags=0) @@ -762,16 +744,14 @@ attributes: The optional *pos* and *endpos* parameters have the same meaning as for the :meth:`~regex.search` method. - .. note:: - - If you want to locate a match anywhere in *string*, use - :meth:`~regex.search` instead. - >>> pattern = re.compile("o") >>> pattern.match("dog") # No match as "o" is not at the start of "dog". >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog". <_sre.SRE_Match object at ...> + If you want to locate a match anywhere in *string*, use + :meth:`~regex.search` instead (see also :ref:`search-vs-match`). + .. method:: regex.split(string, maxsplit=0) @@ -804,8 +784,9 @@ attributes: .. attribute:: regex.flags - The flags argument used when the RE object was compiled, or ``0`` if no flags - were provided. + The regex matching flags. This is a combination of the flags given to + :func:`.compile`, any ``(?...)`` inline flags in the pattern, and implicit + flags such as :data:`UNICODE` if the pattern is a Unicode string. .. attribute:: regex.groups @@ -964,16 +945,15 @@ support the following methods and attributes: .. attribute:: match.pos The value of *pos* which was passed to the :meth:`~regex.search` or - :meth:`~regex.match` method of a :ref:`match object <match-objects>`. This - is the index into the string at which the RE engine started looking for a - match. + :meth:`~regex.match` method of a :ref:`regex object <re-objects>`. This is + the index into the string at which the RE engine started looking for a match. .. attribute:: match.endpos The value of *endpos* which was passed to the :meth:`~regex.search` or - :meth:`~regex.match` method of a :ref:`match object <match-objects>`. This - is the index into the string beyond which the RE engine will not go. + :meth:`~regex.match` method of a :ref:`regex object <re-objects>`. This is + the index into the string beyond which the RE engine will not go. .. attribute:: match.lastindex @@ -1111,59 +1091,39 @@ The equivalent regular expression would be :: (\S+) - (\d+) errors, (\d+) warnings -Avoiding recursion -^^^^^^^^^^^^^^^^^^ - -If you create regular expressions that require the engine to perform a lot of -recursion, you may encounter a :exc:`RuntimeError` exception with the message -``maximum recursion limit exceeded``. For example, :: - - >>> s = 'Begin ' + 1000*'a very long string ' + 'end' - >>> re.match('Begin (\w| )*? end', s).end() - Traceback (most recent call last): - File "<stdin>", line 1, in ? - File "/usr/local/lib/python3.2/re.py", line 132, in match - return _compile(pattern, flags).match(string) - RuntimeError: maximum recursion limit exceeded - -You can often restructure your regular expression to avoid recursion. - -Simple uses of the ``*?`` pattern are special-cased to avoid recursion. Thus, -the above regular expression can avoid recursion by being recast as ``Begin -[a-zA-Z0-9_ ]*?end``. As a further benefit, such regular expressions will run -faster than their recursive equivalents. - +.. _search-vs-match: search() vs. match() ^^^^^^^^^^^^^^^^^^^^ -In a nutshell, :func:`match` only attempts to match a pattern at the beginning -of a string where :func:`search` will match a pattern anywhere in a string. -For example: +.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> - >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog". - >>> re.search("o", "dog") # Match as search() looks everywhere in the string. - <_sre.SRE_Match object at ...> +Python offers two different primitive operations based on regular expressions: +:func:`re.match` checks for a match only at the beginning of the string, while +:func:`re.search` checks for a match anywhere in the string (this is what Perl +does by default). -.. note:: +For example:: - The following applies only to regular expression objects like those created - with ``re.compile("pattern")``, not the primitives ``re.match(pattern, - string)`` or ``re.search(pattern, string)``. + >>> re.match("c", "abcdef") # No match + >>> re.search("c", "abcdef") # Match + <_sre.SRE_Match object at ...> -:func:`match` has an optional second parameter that gives an index in the string -where the search is to start:: +Regular expressions beginning with ``'^'`` can be used with :func:`search` to +restrict the match at the beginning of the string:: - >>> pattern = re.compile("o") - >>> pattern.match("dog") # No match as "o" is not at the start of "dog." + >>> re.match("c", "abcdef") # No match + >>> re.search("^c", "abcdef") # No match + >>> re.search("^a", "abcdef") # Match + <_sre.SRE_Match object at ...> - # Equivalent to the above expression as 0 is the default starting index: - >>> pattern.match("dog", 0) +Note however that in :const:`MULTILINE` mode :func:`match` only matches at the +beginning of the string, whereas using :func:`search` with a regular expression +beginning with ``'^'`` will match at the beginning of each line. - # Match as "o" is the 2nd character of "dog" (index 0 is the first): - >>> pattern.match("dog", 1) + >>> re.match('X', 'A\nB\nX', re.MULTILINE) # No match + >>> re.search('^X', 'A\nB\nX', re.MULTILINE) # Match <_sre.SRE_Match object at ...> - >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog." Making a Phonebook @@ -1177,7 +1137,7 @@ creates a phonebook. First, here is the input. Normally it may come from a file, here we are using triple-quoted string syntax: - >>> input = """Ross McFluff: 834.345.1254 155 Elm Street + >>> text = """Ross McFluff: 834.345.1254 155 Elm Street ... ... Ronald Heathmore: 892.345.3428 436 Finley Avenue ... Frank Burger: 925.541.7625 662 South Dogwood Way @@ -1191,7 +1151,7 @@ into a list with each nonempty line having its own entry: .. doctest:: :options: +NORMALIZE_WHITESPACE - >>> entries = re.split("\n+", input) + >>> entries = re.split("\n+", text) >>> entries ['Ross McFluff: 834.345.1254 155 Elm Street', 'Ronald Heathmore: 892.345.3428 436 Finley Avenue', diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 698b1e74f4..d1cae13d62 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -5,46 +5,58 @@ :synopsis: Set handlers for asynchronous events. -This module provides mechanisms to use signal handlers in Python. Some general -rules for working with signals and their handlers: - -* A handler for a particular signal, once set, remains installed until it is - explicitly reset (Python emulates the BSD style interface regardless of the - underlying implementation), with the exception of the handler for - :const:`SIGCHLD`, which follows the underlying implementation. - -* There is no way to "block" signals temporarily from critical sections (since - this is not supported by all Unix flavors). - -* Although Python signal handlers are called asynchronously as far as the Python - user is concerned, they can only occur between the "atomic" instructions of the - Python interpreter. This means that signals arriving during long calculations - implemented purely in C (such as regular expression matches on large bodies of - text) may be delayed for an arbitrary amount of time. - -* When a signal arrives during an I/O operation, it is possible that the I/O - operation raises an exception after the signal handler returns. This is - dependent on the underlying Unix system's semantics regarding interrupted system - calls. - -* Because the C signal handler always returns, it makes little sense to catch - synchronous errors like :const:`SIGFPE` or :const:`SIGSEGV`. - -* Python installs a small number of signal handlers by default: :const:`SIGPIPE` - is ignored (so write errors on pipes and sockets can be reported as ordinary - Python exceptions) and :const:`SIGINT` is translated into a - :exc:`KeyboardInterrupt` exception. All of these can be overridden. - -* Some care must be taken if both signals and threads are used in the same - program. The fundamental thing to remember in using signals and threads - simultaneously is: always perform :func:`signal` operations in the main thread - of execution. Any thread can perform an :func:`alarm`, :func:`getsignal`, - :func:`pause`, :func:`setitimer` or :func:`getitimer`; only the main thread - can set a new signal handler, and the main thread will be the only one to - receive signals (this is enforced by the Python :mod:`signal` module, even - if the underlying thread implementation supports sending signals to - individual threads). This means that signals can't be used as a means of - inter-thread communication. Use locks instead. +This module provides mechanisms to use signal handlers in Python. + + +General rules +------------- + +The :func:`signal.signal` function allows to define custom handlers to be +executed when a signal is received. A small number of default handlers are +installed: :const:`SIGPIPE` is ignored (so write errors on pipes and sockets +can be reported as ordinary Python exceptions) and :const:`SIGINT` is +translated into a :exc:`KeyboardInterrupt` exception. + +A handler for a particular signal, once set, remains installed until it is +explicitly reset (Python emulates the BSD style interface regardless of the +underlying implementation), with the exception of the handler for +:const:`SIGCHLD`, which follows the underlying implementation. + +There is no way to "block" signals temporarily from critical sections (since +this is not supported by all Unix flavors). + + +Execution of Python signal handlers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A Python signal handler does not get executed inside the low-level (C) signal +handler. Instead, the low-level signal handler sets a flag which tells the +:term:`virtual machine` to execute the corresponding Python signal handler +at a later point(for example at the next :term:`bytecode` instruction). +This has consequences: + +* It makes little sense to catch synchronous errors like :const:`SIGFPE` or + :const:`SIGSEGV`. + +* A long-running calculation implemented purely in C (such as regular + expression matching on a large body of text) may run uninterrupted for an + arbitrary amount of time, regardless of any signals received. The Python + signal handlers will be called when the calculation finishes. + + +Signals and threads +^^^^^^^^^^^^^^^^^^^ + +Python signal handlers are always executed in the main Python thread, +even if the signal was received in another thread. This means that signals +can't be used as a means of inter-thread communication. You can use +the synchronization primitives from the :mod:`threading` module instead. + +Besides, only the main thread is allowed to set a new signal handler. + + +Module contents +--------------- The variables defined in the :mod:`signal` module are: diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index f0fd86cb85..41db5c37a9 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -3,7 +3,7 @@ .. module:: sqlite3 :synopsis: A DB-API 2.0 implementation using SQLite 3.x. -.. sectionauthor:: Gerhard Häring <gh@ghaering.de> +.. sectionauthor:: Gerhard Häring <gh@ghaering.de> SQLite is a C library that provides a lightweight disk-based database that @@ -20,6 +20,7 @@ To use the module, you must first create a :class:`Connection` object that represents the database. Here the data will be stored in the :file:`/tmp/example` file:: + import sqlite3 conn = sqlite3.connect('/tmp/example') You can also supply the special name ``:memory:`` to create a database in RAM. @@ -56,7 +57,7 @@ example:: # Never do this -- insecure! symbol = 'IBM' - c.execute("... where symbol = '%s'" % symbol) + c.execute("select * from stocks where symbol = '%s'" % symbol) # Do this instead t = (symbol,) @@ -64,7 +65,7 @@ example:: # Larger example for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), - ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00), + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), ('2006-04-06', 'SELL', 'IBM', 500, 53.00), ]: c.execute('insert into stocks values (?,?,?,?,?)', t) @@ -271,7 +272,6 @@ Connection Objects calling the cursor method, then calls the cursor's :meth:`executemany <Cursor.executemany>` method with the parameters given. - .. method:: Connection.executescript(sql_script) This is a nonstandard shortcut that creates an intermediate cursor object by @@ -376,22 +376,22 @@ Connection Objects aggregates or whole new virtual table implementations. One well-known extension is the fulltext-search extension distributed with SQLite. + Loadable extensions are disabled by default. See [#f1]_. + .. versionadded:: 3.2 .. literalinclude:: ../includes/sqlite3/load_extension.py - Loadable extensions are disabled by default. See [#f1]_. - .. method:: Connection.load_extension(path) This routine loads a SQLite extension from a shared library. You have to enable extension loading with :meth:`enable_load_extension` before you can use this routine. - .. versionadded:: 3.2 - Loadable extensions are disabled by default. See [#f1]_. + .. versionadded:: 3.2 + .. attribute:: Connection.row_factory You can change this attribute to a callable that accepts the cursor and the diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 5f5d3b6ce7..153ee44cc9 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1437,8 +1437,13 @@ Old String Formatting Operations .. note:: - The formatting operations described here are obsolete and may go away in future - versions of Python. Use the new :ref:`string-formatting` in new code. + The formatting operations described here are modelled on C's printf() + syntax. They only support formatting of certain builtin types. The + use of a binary operator means that care may be needed in order to + format tuples and dictionaries correctly. As the new + :ref:`string-formatting` syntax is more flexible and handles tuples and + dictionaries naturally, it is recommended for new code. However, there + are no current plans to deprecate printf-style formatting. String objects have one unique built-in operation: the ``%`` operator (modulo). This is also known as the string *formatting* or *interpolation* operator. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index b7e87ab302..90a01d0a8f 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -735,7 +735,7 @@ The p1.stdout.close() call after starting the p2 is important in order for p1 to receive a SIGPIPE if p2 exits before p1. Alternatively, for trusted input, the shell's own pipeline support may still -be used directly: +be used directly:: output=`dmesg | grep hda` # becomes diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst index 795d66d99f..645c326678 100644 --- a/Doc/library/syslog.rst +++ b/Doc/library/syslog.rst @@ -78,7 +78,8 @@ Priority levels (high to low): Facilities: :const:`LOG_KERN`, :const:`LOG_USER`, :const:`LOG_MAIL`, :const:`LOG_DAEMON`, :const:`LOG_AUTH`, :const:`LOG_LPR`, :const:`LOG_NEWS`, :const:`LOG_UUCP`, - :const:`LOG_CRON` and :const:`LOG_LOCAL0` to :const:`LOG_LOCAL7`. + :const:`LOG_CRON`, :const:`LOG_SYSLOG` and :const:`LOG_LOCAL0` to + :const:`LOG_LOCAL7`. Log options: :const:`LOG_PID`, :const:`LOG_CONS`, :const:`LOG_NDELAY`, :const:`LOG_NOWAIT` diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 9b3affd979..8571104c4c 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -218,30 +218,31 @@ Thread Objects This class represents an activity that is run in a separate thread of control. There are two ways to specify the activity: by passing a callable object to the -constructor, or by overriding the :meth:`run` method in a subclass. No other -methods (except for the constructor) should be overridden in a subclass. In -other words, *only* override the :meth:`__init__` and :meth:`run` methods of -this class. +constructor, or by overriding the :meth:`~Thread.run` method in a subclass. +No other methods (except for the constructor) should be overridden in a +subclass. In other words, *only* override the :meth:`~Thread.__init__` +and :meth:`~Thread.run` methods of this class. Once a thread object is created, its activity must be started by calling the -thread's :meth:`start` method. This invokes the :meth:`run` method in a -separate thread of control. +thread's :meth:`~Thread.start` method. This invokes the :meth:`~Thread.run` +method in a separate thread of control. Once the thread's activity is started, the thread is considered 'alive'. It -stops being alive when its :meth:`run` method terminates -- either normally, or -by raising an unhandled exception. The :meth:`is_alive` method tests whether the -thread is alive. +stops being alive when its :meth:`~Thread.run` method terminates -- either +normally, or by raising an unhandled exception. The :meth:`~Thread.is_alive` +method tests whether the thread is alive. -Other threads can call a thread's :meth:`join` method. This blocks the calling -thread until the thread whose :meth:`join` method is called is terminated. +Other threads can call a thread's :meth:`~Thread.join` method. This blocks +the calling thread until the thread whose :meth:`~Thread.join` method is +called is terminated. A thread has a name. The name can be passed to the constructor, and read or -changed through the :attr:`name` attribute. +changed through the :attr:`~Thread.name` attribute. -A thread can be flagged as a "daemon thread". The significance of this flag is -that the entire Python program exits when only daemon threads are left. The -initial value is inherited from the creating thread. The flag can be set -through the :attr:`daemon` property. +A thread can be flagged as a "daemon thread". The significance of this flag +is that the entire Python program exits when only daemon threads are left. +The initial value is inherited from the creating thread. The flag can be +set through the :attr:`~Thread.daemon` property. There is a "main thread" object; this corresponds to the initial thread of control in the Python program. It is not a daemon thread. @@ -250,8 +251,8 @@ There is the possibility that "dummy thread objects" are created. These are thread objects corresponding to "alien threads", which are threads of control started outside the threading module, such as directly from C code. Dummy thread objects have limited functionality; they are always considered alive and -daemonic, and cannot be :meth:`join`\ ed. They are never deleted, since it is -impossible to detect the termination of alien threads. +daemonic, and cannot be :meth:`~Thread.join`\ ed. They are never deleted, +since it is impossible to detect the termination of alien threads. .. class:: Thread(group=None, target=None, name=None, args=(), kwargs={}) @@ -282,7 +283,8 @@ impossible to detect the termination of alien threads. Start the thread's activity. It must be called at most once per thread object. It arranges for the - object's :meth:`run` method to be invoked in a separate thread of control. + object's :meth:`~Thread.run` method to be invoked in a separate thread + of control. This method will raise a :exc:`RuntimeError` if called more than once on the same thread object. @@ -298,25 +300,27 @@ impossible to detect the termination of alien threads. .. method:: join(timeout=None) - Wait until the thread terminates. This blocks the calling thread until the - thread whose :meth:`join` method is called terminates -- either normally - or through an unhandled exception -- or until the optional timeout occurs. + Wait until the thread terminates. This blocks the calling thread until + the thread whose :meth:`~Thread.join` method is called terminates -- either + normally or through an unhandled exception --, or until the optional + timeout occurs. When the *timeout* argument is present and not ``None``, it should be a floating point number specifying a timeout for the operation in seconds - (or fractions thereof). As :meth:`join` always returns ``None``, you must - call :meth:`is_alive` after :meth:`join` to decide whether a timeout - happened -- if the thread is still alive, the :meth:`join` call timed out. + (or fractions thereof). As :meth:`~Thread.join` always returns ``None``, + you must call :meth:`~Thread.is_alive` after :meth:`~Thread.join` to + decide whether a timeout happened -- if the thread is still alive, the + :meth:`~Thread.join` call timed out. When the *timeout* argument is not present or ``None``, the operation will block until the thread terminates. - A thread can be :meth:`join`\ ed many times. + A thread can be :meth:`~Thread.join`\ ed many times. - :meth:`join` raises a :exc:`RuntimeError` if an attempt is made to join - the current thread as that would cause a deadlock. It is also an error to - :meth:`join` a thread before it has been started and attempts to do so - raises the same exception. + :meth:`~Thread.join` raises a :exc:`RuntimeError` if an attempt is made + to join the current thread as that would cause a deadlock. It is also + an error to :meth:`~Thread.join` a thread before it has been started + and attempts to do so raise the same exception. .. attribute:: name @@ -334,7 +338,7 @@ impossible to detect the termination of alien threads. The 'thread identifier' of this thread or ``None`` if the thread has not been started. This is a nonzero integer. See the - :func:`thread.get_ident()` function. Thread identifiers may be recycled + :func:`_thread.get_ident()` function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited. @@ -342,18 +346,18 @@ impossible to detect the termination of alien threads. Return whether the thread is alive. - This method returns ``True`` just before the :meth:`run` method starts - until just after the :meth:`run` method terminates. The module function - :func:`.enumerate` returns a list of all alive threads. + This method returns ``True`` just before the :meth:`~Thread.run` method + starts until just after the :meth:`~Thread.run` method terminates. The + module function :func:`.enumerate` returns a list of all alive threads. .. attribute:: daemon A boolean value indicating whether this thread is a daemon thread (True) - or not (False). This must be set before :meth:`start` is called, + or not (False). This must be set before :meth:`~Thread.start` is called, otherwise :exc:`RuntimeError` is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and - therefore all threads created in the main thread default to :attr:`daemon` - = ``False``. + therefore all threads created in the main thread default to + :attr:`~Thread.daemon` = ``False``. The entire Python program exits when no alive non-daemon threads are left. @@ -375,19 +379,22 @@ synchronization primitive available, implemented directly by the :mod:`_thread` extension module. A primitive lock is in one of two states, "locked" or "unlocked". It is created -in the unlocked state. It has two basic methods, :meth:`acquire` and -:meth:`release`. When the state is unlocked, :meth:`acquire` changes the state -to locked and returns immediately. When the state is locked, :meth:`acquire` -blocks until a call to :meth:`release` in another thread changes it to unlocked, -then the :meth:`acquire` call resets it to locked and returns. The -:meth:`release` method should only be called in the locked state; it changes the -state to unlocked and returns immediately. If an attempt is made to release an -unlocked lock, a :exc:`RuntimeError` will be raised. - -When more than one thread is blocked in :meth:`acquire` waiting for the state to -turn to unlocked, only one thread proceeds when a :meth:`release` call resets -the state to unlocked; which one of the waiting threads proceeds is not defined, -and may vary across implementations. +in the unlocked state. It has two basic methods, :meth:`~Lock.acquire` and +:meth:`~Lock.release`. When the state is unlocked, :meth:`~Lock.acquire` +changes the state to locked and returns immediately. When the state is locked, +:meth:`~Lock.acquire` blocks until a call to :meth:`~Lock.release` in another +thread changes it to unlocked, then the :meth:`~Lock.acquire` call resets it +to locked and returns. The :meth:`~Lock.release` method should only be +called in the locked state; it changes the state to unlocked and returns +immediately. If an attempt is made to release an unlocked lock, a +:exc:`RuntimeError` will be raised. + +Locks also support the :ref:`context manager protocol <with-locks>`. + +When more than one thread is blocked in :meth:`~Lock.acquire` waiting for the +state to turn to unlocked, only one thread proceeds when a :meth:`~Lock.release` +call resets the state to unlocked; which one of the waiting threads proceeds +is not defined, and may vary across implementations. All methods are executed atomically. @@ -424,13 +431,14 @@ All methods are executed atomically. .. method:: Lock.release() - Release a lock. + Release a lock. This can be called from any thread, not only the thread + which has acquired the lock. When the lock is locked, reset it to unlocked, and return. If any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. - Do not call this method when the lock is unlocked. + When invoked on an unlocked lock, a :exc:`ThreadError` is raised. There is no return value. @@ -446,12 +454,14 @@ and "recursion level" in addition to the locked/unlocked state used by primitive locks. In the locked state, some thread owns the lock; in the unlocked state, no thread owns it. -To lock the lock, a thread calls its :meth:`acquire` method; this returns once -the thread owns the lock. To unlock the lock, a thread calls its -:meth:`release` method. :meth:`acquire`/:meth:`release` call pairs may be -nested; only the final :meth:`release` (the :meth:`release` of the outermost -pair) resets the lock to unlocked and allows another thread blocked in -:meth:`acquire` to proceed. +To lock the lock, a thread calls its :meth:`~RLock.acquire` method; this +returns once the thread owns the lock. To unlock the lock, a thread calls +its :meth:`~Lock.release` method. :meth:`~Lock.acquire`/:meth:`~Lock.release` +call pairs may be nested; only the final :meth:`~Lock.release` (the +:meth:`~Lock.release` of the outermost pair) resets the lock to unlocked and +allows another thread blocked in :meth:`~Lock.acquire` to proceed. + +Reentrant locks also support the :ref:`context manager protocol <with-locks>`. .. method:: RLock.acquire(blocking=True, timeout=-1) @@ -503,63 +513,75 @@ Condition Objects ----------------- A condition variable is always associated with some kind of lock; this can be -passed in or one will be created by default. (Passing one in is useful when -several condition variables must share the same lock.) - -A condition variable has :meth:`acquire` and :meth:`release` methods that call -the corresponding methods of the associated lock. It also has a :meth:`wait` -method, and :meth:`notify` and :meth:`notify_all` methods. These three must only -be called when the calling thread has acquired the lock, otherwise a -:exc:`RuntimeError` is raised. - -The :meth:`wait` method releases the lock, and then blocks until it is awakened -by a :meth:`notify` or :meth:`notify_all` call for the same condition variable in -another thread. Once awakened, it re-acquires the lock and returns. It is also -possible to specify a timeout. - -The :meth:`notify` method wakes up one of the threads waiting for the condition -variable, if any are waiting. The :meth:`notify_all` method wakes up all threads -waiting for the condition variable. - -Note: the :meth:`notify` and :meth:`notify_all` methods don't release the lock; -this means that the thread or threads awakened will not return from their -:meth:`wait` call immediately, but only when the thread that called -:meth:`notify` or :meth:`notify_all` finally relinquishes ownership of the lock. - -Tip: the typical programming style using condition variables uses the lock to +passed in or one will be created by default. Passing one in is useful when +several condition variables must share the same lock. The lock is part of +the condition object: you don't have to track it separately. + +A condition variable obeys the :ref:`context manager protocol <with-locks>`: +using the ``with`` statement acquires the associated lock for the duration of +the enclosed block. The :meth:`~Condition.acquire` and +:meth:`~Condition.release` methods also call the corresponding methods of +the associated lock. + +Other methods must be called with the associated lock held. The +:meth:`~Condition.wait` method releases the lock, and then blocks until +another thread awakens it by calling :meth:`~Condition.notify` or +:meth:`~Condition.notify_all`. Once awakened, :meth:`~Condition.wait` +re-acquires the lock and returns. It is also possible to specify a timeout. + +The :meth:`~Condition.notify` method wakes up one of the threads waiting for +the condition variable, if any are waiting. The :meth:`~Condition.notify_all` +method wakes up all threads waiting for the condition variable. + +Note: the :meth:`~Condition.notify` and :meth:`~Condition.notify_all` methods +don't release the lock; this means that the thread or threads awakened will +not return from their :meth:`~Condition.wait` call immediately, but only when +the thread that called :meth:`~Condition.notify` or :meth:`~Condition.notify_all` +finally relinquishes ownership of the lock. + + +Usage +^^^^^ + +The typical programming style using condition variables uses the lock to synchronize access to some shared state; threads that are interested in a -particular change of state call :meth:`wait` repeatedly until they see the -desired state, while threads that modify the state call :meth:`notify` or -:meth:`notify_all` when they change the state in such a way that it could -possibly be a desired state for one of the waiters. For example, the following -code is a generic producer-consumer situation with unlimited buffer capacity:: +particular change of state call :meth:`~Condition.wait` repeatedly until they +see the desired state, while threads that modify the state call +:meth:`~Condition.notify` or :meth:`~Condition.notify_all` when they change +the state in such a way that it could possibly be a desired state for one +of the waiters. For example, the following code is a generic +producer-consumer situation with unlimited buffer capacity:: # Consume one item - cv.acquire() - while not an_item_is_available(): - cv.wait() - get_an_available_item() - cv.release() + with cv: + while not an_item_is_available(): + cv.wait() + get_an_available_item() # Produce one item - cv.acquire() - make_an_item_available() - cv.notify() - cv.release() + with cv: + make_an_item_available() + +The ``while`` loop checking for the application's condition is necessary +because :meth:`~Condition.wait` can return after an arbitrary long time, +and other threads may have exhausted the available items in between. This +is inherent to multi-threaded programming. The :meth:`~Condition.wait_for` +method can be used to automate the condition checking:: -To choose between :meth:`notify` and :meth:`notify_all`, consider whether one -state change can be interesting for only one or several waiting threads. E.g. -in a typical producer-consumer situation, adding one item to the buffer only -needs to wake up one consumer thread. + # Consume an item + with cv: + cv.wait_for(an_item_is_available) + get_an_available_item() -Note: Condition variables can be, depending on the implementation, subject -to both spurious wakeups (when :meth:`wait` returns without a :meth:`notify` -call) and stolen wakeups (when another thread acquires the lock before the -awoken thread.) For this reason, it is always necessary to verify the state -the thread is waiting for when :meth:`wait` returns and optionally repeat -the call as often as necessary. +To choose between :meth:`~Condition.notify` and :meth:`~Condition.notify_all`, +consider whether one state change can be interesting for only one or several +waiting threads. E.g. in a typical producer-consumer situation, adding one +item to the buffer only needs to wake up one consumer thread. +Interface +^^^^^^^^^ + .. class:: Condition(lock=None) If the *lock* argument is given and not ``None``, it must be a :class:`Lock` @@ -626,12 +648,6 @@ the call as often as necessary. held when called and is re-aquired on return. The predicate is evaluated with the lock held. - Using this method, the consumer example above can be written thus:: - - with cv: - cv.wait_for(an_item_is_available) - get_an_available_item() - .. versionadded:: 3.2 .. method:: notify(n=1) @@ -667,12 +683,16 @@ Semaphore Objects This is one of the oldest synchronization primitives in the history of computer science, invented by the early Dutch computer scientist Edsger W. Dijkstra (he -used :meth:`P` and :meth:`V` instead of :meth:`acquire` and :meth:`release`). +used the names ``P()`` and ``V()`` instead of :meth:`~Semaphore.acquire` and +:meth:`~Semaphore.release`). A semaphore manages an internal counter which is decremented by each -:meth:`acquire` call and incremented by each :meth:`release` call. The counter -can never go below zero; when :meth:`acquire` finds that it is zero, it blocks, -waiting until some other thread calls :meth:`release`. +:meth:`~Semaphore.acquire` call and incremented by each :meth:`~Semaphore.release` +call. The counter can never go below zero; when :meth:`~Semaphore.acquire` +finds that it is zero, it blocks, waiting until some other thread calls +:meth:`~Semaphore.release`. + +Semaphores also support the :ref:`context manager protocol <with-locks>`. .. class:: Semaphore(value=1) @@ -688,11 +708,12 @@ waiting until some other thread calls :meth:`release`. When invoked without arguments: if the internal counter is larger than zero on entry, decrement it by one and return immediately. If it is zero on entry, block, waiting until some other thread has called - :meth:`release` to make it larger than zero. This is done with proper - interlocking so that if multiple :meth:`acquire` calls are blocked, - :meth:`release` will wake exactly one of them up. The implementation may - pick one at random, so the order in which blocked threads are awakened - should not be relied on. Returns true (or blocks indefinitely). + :meth:`~Semaphore.release` to make it larger than zero. This is done + with proper interlocking so that if multiple :meth:`acquire` calls are + blocked, :meth:`~Semaphore.release` will wake exactly one of them up. + The implementation may pick one at random, so the order in which + blocked threads are awakened should not be relied on. Returns + true (or blocks indefinitely). When invoked with *blocking* set to false, do not block. If a call without an argument would block, return false immediately; otherwise, @@ -729,11 +750,12 @@ main thread would initialize the semaphore:: Once spawned, worker threads call the semaphore's acquire and release methods when they need to connect to the server:: - pool_sema.acquire() - conn = connectdb() - ... use connection ... - conn.close() - pool_sema.release() + with pool_sema: + conn = connectdb() + try: + ... use connection ... + finally: + conn.close() The use of a bounded semaphore reduces the chance that a programming error which causes the semaphore to be released more than it's acquired will go undetected. @@ -748,8 +770,8 @@ This is one of the simplest mechanisms for communication between threads: one thread signals an event and other threads wait for it. An event object manages an internal flag that can be set to true with the -:meth:`~Event.set` method and reset to false with the :meth:`clear` method. The -:meth:`wait` method blocks until the flag is true. +:meth:`~Event.set` method and reset to false with the :meth:`~Event.clear` +method. The :meth:`~Event.wait` method blocks until the flag is true. .. class:: Event() @@ -776,7 +798,7 @@ An event object manages an internal flag that can be set to true with the Block until the internal flag is true. If the internal flag is true on entry, return immediately. Otherwise, block until another thread calls - :meth:`set` to set the flag to true, or until the optional timeout occurs. + :meth:`.set` to set the flag to true, or until the optional timeout occurs. When the timeout argument is present and not ``None``, it should be a floating point number specifying a timeout for the operation in seconds @@ -832,8 +854,8 @@ Barrier Objects This class provides a simple synchronization primitive for use by a fixed number of threads that need to wait for each other. Each of the threads tries to pass -the barrier by calling the :meth:`wait` method and will block until all of the -threads have made the call. At this points, the threads are released +the barrier by calling the :meth:`~Barrier.wait` method and will block until +all of the threads have made the call. At this points, the threads are released simultanously. The barrier can be reused any number of times for the same number of threads. @@ -934,19 +956,24 @@ Using locks, conditions, and semaphores in the :keyword:`with` statement All of the objects provided by this module that have :meth:`acquire` and :meth:`release` methods can be used as context managers for a :keyword:`with` -statement. The :meth:`acquire` method will be called when the block is entered, -and :meth:`release` will be called when the block is exited. +statement. The :meth:`acquire` method will be called when the block is +entered, and :meth:`release` will be called when the block is exited. Hence, +the following snippet:: -Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`, -:class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as -:keyword:`with` statement context managers. For example:: + with some_lock: + # do something... - import threading +is equivalent to:: - some_rlock = threading.RLock() + some_lock.acquire() + try: + # do something... + finally: + some_lock.release() - with some_rlock: - print("some_rlock is locked while this executes") +Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`, +:class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as +:keyword:`with` statement context managers. .. _threaded-imports: diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 7c464ac245..7854fbd4da 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -430,8 +430,8 @@ The module defines the following functions and data items: .. function:: time() - Return the time as a floating point number expressed in seconds since the epoch, - in UTC. Note that even though the time is always returned as a floating point + Return the time in seconds since the epoch as a floating point number. + Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index bdf07a40cb..b130a8b088 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -640,7 +640,7 @@ This is the output of running the example above in verbose mode: :: Classes can be skipped just like methods: :: - @skip("showing class skipping") + @unittest.skip("showing class skipping") class MySkippedTestCase(unittest.TestCase): def test_not_run(self): pass diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index aece7149ef..b33e8fe1ed 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -512,9 +512,10 @@ task isn't already covered by the URL parsing functions above. Convert a mapping object or a sequence of two-element tuples, which may either be a :class:`str` or a :class:`bytes`, to a "percent-encoded" - string. The resultant string must be converted to bytes using the - user-specified encoding before it is sent to :func:`urlopen` as the optional - *data* argument. + string. If the resultant string is to be used as a *data* for POST + operation with :func:`urlopen` function, then it should be properly encoded + to bytes, otherwise it would result in a :exc:`TypeError`. + The resulting string is a series of ``key=value`` pairs separated by ``'&'`` characters, where both *key* and *value* are quoted using :func:`quote_plus` above. When a sequence of two-element tuples is used as the *query* diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index cc759a4465..58f33e3a31 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -2,9 +2,10 @@ ============================================================= .. module:: urllib.request - :synopsis: Next generation URL opening library. + :synopsis: Extensible library for opening URLs. .. moduleauthor:: Jeremy Hylton <jeremy@alum.mit.edu> .. sectionauthor:: Moshe Zadka <moshez@users.sourceforge.net> +.. sectionauthor:: Senthil Kumaran <senthil@uthcode.com> The :mod:`urllib.request` module defines functions and classes which help in @@ -20,16 +21,26 @@ The :mod:`urllib.request` module defines the following functions: Open the URL *url*, which can be either a string or a :class:`Request` object. - *data* may be a bytes object specifying additional data to send to the + *data* must be a bytes object specifying additional data to be sent to the server, or ``None`` if no such data is needed. *data* may also be an iterable object and in that case Content-Length value must be specified in the headers. Currently HTTP requests are the only ones that use *data*; the HTTP request will be a POST instead of a GET when the *data* parameter is - provided. *data* should be a buffer in the standard + provided. + + *data* should be a buffer in the standard :mimetype:`application/x-www-form-urlencoded` format. The :func:`urllib.parse.urlencode` function takes a mapping or sequence of - 2-tuples and returns a string in this format. urllib.request module uses - HTTP/1.1 and includes ``Connection:close`` header in its HTTP requests. + 2-tuples and returns a string in this format. It should be encoded to bytes + before being used as the *data* parameter. The charset parameter in + ``Content-Type`` header may be used to specify the encoding. If charset + parameter is not sent with the Content-Type header, the server following the + HTTP 1.1 recommendation may assume that the data is encoded in ISO-8859-1 + encoding. It is advisable to use charset parameter with encoding used in + ``Content-Type`` header with the :class:`Request`. + + urllib.request module uses HTTP/1.1 and includes ``Connection:close`` header + in its HTTP requests. The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, @@ -46,8 +57,8 @@ The :mod:`urllib.request` module defines the following functions: If neither *cafile* nor *capath* is specified, an HTTPS request will not do any verification of the server's certificate. - This function returns a file-like object with two additional methods from - the :mod:`urllib.response` module + This function returns a file-like object that works as a :term:`context manager`, + with two additional methods from the :mod:`urllib.response` module * :meth:`geturl` --- return the URL of the resource retrieved, commonly used to determine if a redirect was followed @@ -66,9 +77,10 @@ The :mod:`urllib.request` module defines the following functions: are handled through the proxy when they are set. The legacy ``urllib.urlopen`` function from Python 2.6 and earlier has been - discontinued; :func:`urlopen` corresponds to the old ``urllib2.urlopen``. - Proxy handling, which was done by passing a dictionary parameter to - ``urllib.urlopen``, can be obtained by using :class:`ProxyHandler` objects. + discontinued; :func:`urllib.request.urlopen` corresponds to the old + ``urllib2.urlopen``. Proxy handling, which was done by passing a dictionary + parameter to ``urllib.urlopen``, can be obtained by using + :class:`ProxyHandler` objects. .. versionchanged:: 3.2 *cafile* and *capath* were added. @@ -83,10 +95,11 @@ The :mod:`urllib.request` module defines the following functions: .. function:: install_opener(opener) Install an :class:`OpenerDirector` instance as the default global opener. - Installing an opener is only necessary if you want urlopen to use that opener; - otherwise, simply call :meth:`OpenerDirector.open` instead of :func:`urlopen`. - The code does not check for a real :class:`OpenerDirector`, and any class with - the appropriate interface will work. + Installing an opener is only necessary if you want urlopen to use that + opener; otherwise, simply call :meth:`OpenerDirector.open` instead of + :func:`~urllib.request.urlopen`. The code does not check for a real + :class:`OpenerDirector`, and any class with the appropriate interface will + work. .. function:: build_opener([handler, ...]) @@ -138,14 +151,21 @@ The following classes are provided: *url* should be a string containing a valid URL. - *data* may be a string specifying additional data to send to the - server, or ``None`` if no such data is needed. Currently HTTP - requests are the only ones that use *data*; the HTTP request will - be a POST instead of a GET when the *data* parameter is provided. - *data* should be a buffer in the standard - :mimetype:`application/x-www-form-urlencoded` format. The - :func:`urllib.parse.urlencode` function takes a mapping or sequence - of 2-tuples and returns a string in this format. + *data* must be a bytes object specifying additional data to send to the + server, or ``None`` if no such data is needed. Currently HTTP requests are + the only ones that use *data*; the HTTP request will be a POST instead of a + GET when the *data* parameter is provided. *data* should be a buffer in the + standard :mimetype:`application/x-www-form-urlencoded` format. + + The :func:`urllib.parse.urlencode` function takes a mapping or sequence of + 2-tuples and returns a string in this format. It should be encoded to bytes + before being used as the *data* parameter. The charset parameter in + ``Content-Type`` header may be used to specify the encoding. If charset + parameter is not sent with the Content-Type header, the server following the + HTTP 1.1 recommendation may assume that the data is encoded in ISO-8859-1 + encoding. It is advisable to use charset parameter with encoding used in + ``Content-Type`` header with the :class:`Request`. + *headers* should be a dictionary, and will be treated as if :meth:`add_header` was called with each key and value as arguments. @@ -157,6 +177,9 @@ The following classes are provided: :mod:`urllib`'s default user agent string is ``"Python-urllib/2.6"`` (on Python 2.6). + An example of using ``Content-Type`` header with *data* argument would be + sending a dictionary like ``{"Content-Type":" application/x-www-form-urlencoded;charset=utf-8"}`` + The final two arguments are only of interest for correct handling of third-party HTTP cookies: @@ -967,8 +990,17 @@ The following W3C document, http://www.w3.org/International/O-charset , lists the various ways in which a (X)HTML or a XML document could have specified its encoding information. -As python.org website uses *utf-8* encoding as specified in it's meta tag, we -will use same for decoding the bytes object. :: +As the python.org website uses *utf-8* encoding as specified in it's meta tag, we +will use the same for decoding the bytes object. :: + + >>> with urllib.request.urlopen('http://www.python.org/') as f: + ... print(f.read(100).decode('utf-8')) + ... + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtm + +It is also possible to achieve the same result without using the +:term:`context manager` approach. :: >>> import urllib.request >>> f = urllib.request.urlopen('http://www.python.org/') @@ -976,7 +1008,6 @@ will use same for decoding the bytes object. :: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtm - In the following example, we are sending a data-stream to the stdin of a CGI and reading the data it returns to us. Note that this example will only work when the Python installation supports SSL. :: @@ -1045,8 +1076,9 @@ every :class:`Request`. To change this:: opener.open('http://www.example.com/') Also, remember that a few standard headers (:mailheader:`Content-Length`, -:mailheader:`Content-Type` and :mailheader:`Host`) are added when the -:class:`Request` is passed to :func:`urlopen` (or :meth:`OpenerDirector.open`). +:mailheader:`Content-Type` without charset parameter and :mailheader:`Host`) +are added when the :class:`Request` is passed to :func:`urlopen` (or +:meth:`OpenerDirector.open`). .. _urllib-examples: @@ -1064,9 +1096,12 @@ from urlencode is encoded to bytes before it is sent to urlopen as data:: >>> import urllib.request >>> import urllib.parse - >>> params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) - >>> params = params.encode('utf-8') - >>> f = urllib.request.urlopen("http://www.musi-cal.com/cgi-bin/query", params) + >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) + >>> data = data.encode('utf-8') + >>> request = urllib.request.Request("http://requestb.in/xrbl82xr") + >>> # adding charset parameter to the Content-Type header. + >>> request.add_header("Content-Type","application/x-www-form-urlencoded;charset=utf-8") + >>> f = urllib.request.urlopen(request, data) >>> print(f.read().decode('utf-8')) The following example uses an explicitly specified HTTP proxy, overriding @@ -1114,10 +1149,10 @@ some point in the future. size in response to a retrieval request. If the *url* uses the :file:`http:` scheme identifier, the optional *data* - argument may be given to specify a ``POST`` request (normally the request type - is ``GET``). The *data* argument must in standard - :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode` - function below. + argument may be given to specify a ``POST`` request (normally the request + type is ``GET``). The *data* argument must be a bytes object in standard + :mimetype:`application/x-www-form-urlencoded` format; see the + :func:`urlencode` function below. :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that the amount of data available was less than the expected amount (which is the diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst index 23ba6c5471..dfb09ee40e 100644 --- a/Doc/library/webbrowser.rst +++ b/Doc/library/webbrowser.rst @@ -137,6 +137,8 @@ for the controller classes, all defined in this module. +-----------------------+-----------------------------------------+-------+ | ``'macosx'`` | :class:`MacOSX('default')` | \(4) | +-----------------------+-----------------------------------------+-------+ +| ``'safari'`` | :class:`MacOSX('safari')` | \(4) | ++-----------------------+-----------------------------------------+-------+ Notes: diff --git a/Doc/library/xml.dom.minidom.rst b/Doc/library/xml.dom.minidom.rst index ab5476d91f..ae286b0ad9 100644 --- a/Doc/library/xml.dom.minidom.rst +++ b/Doc/library/xml.dom.minidom.rst @@ -15,6 +15,14 @@ Model interface. It is intended to be simpler than the full DOM and also significantly smaller. +.. note:: + + The :mod:`xml.dom.minidom` module provides an implementation of the W3C-DOM, + with an API similar to that in other programming languages. Users who are + unfamiliar with the W3C-DOM interface or who would like to write less code + for processing XML files should consider using the + :mod:`xml.etree.ElementTree` module instead. + DOM applications typically start by parsing some XML into a DOM. With :mod:`xml.dom.minidom`, this is done through the parse functions:: diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst index 4a5ef4c135..eb16a0933d 100644 --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -9,34 +9,72 @@ -------------- -:mod:`xml.dom.pulldom` allows building only selected portions of a Document -Object Model representation of a document from SAX events. +The :mod:`xml.dom.pulldom` module provides a "pull parser" which can also be +asked to produce DOM-accessible fragments of the document where necessary. The +basic concept involves pulling "events" from a stream of incoming XML and +processing them. In contrast to SAX which also employs an event-driven +processing model together with callbacks, the user of a pull parser is +responsible for explicitly pulling events from the stream, looping over those +events until either processing is finished or an error condition occurs. +Example:: -.. class:: PullDOM(documentFactory=None) + from xml.dom import pulldom - :class:`xml.sax.handler.ContentHandler` implementation that ... + doc = pulldom.parse('sales_items.xml') + for event, node in doc: + if event == pulldom.START_ELEMENT and node.tagName == 'item': + if int(node.getAttribute('price')) > 50: + doc.expandNode(node) + print(node.toxml()) +``event`` is a constant and can be one of: -.. class:: DOMEventStream(stream, parser, bufsize) +* :data:`START_ELEMENT` +* :data:`END_ELEMENT` +* :data:`COMMENT` +* :data:`START_DOCUMENT` +* :data:`END_DOCUMENT` +* :data:`CHARACTERS` +* :data:`PROCESSING_INSTRUCTION` +* :data:`IGNORABLE_WHITESPACE` + +``node`` is a object of type :class:`xml.dom.minidom.Document`, +:class:`xml.dom.minidom.Element` or :class:`xml.dom.minidom.Text`. + +Since the document is treated as a "flat" stream of events, the document "tree" +is implicitly traversed and the desired elements are found regardless of their +depth in the tree. In other words, one does not need to consider hierarchical +issues such as recursive searching of the document nodes, although if the +context of elements were important, one would either need to maintain some +context-related state (i.e. remembering where one is in the document at any +given point) or to make use of the :func:`DOMEventStream.expandNode` method +and switch to DOM-related processing. + + +.. class:: PullDom(documentFactory=None) - ... + Subclass of :class:`xml.sax.handler.ContentHandler`. .. class:: SAX2DOM(documentFactory=None) - :class:`xml.sax.handler.ContentHandler` implementation that ... + Subclass of :class:`xml.sax.handler.ContentHandler`. .. function:: parse(stream_or_string, parser=None, bufsize=None) - ... + Return a :class:`DOMEventStream` from the given input. *stream_or_string* may be + either a file name, or a file-like object. *parser*, if given, must be a + :class:`XmlReader` object. This function will change the document handler of the + parser and activate namespace support; other parser configuration (like + setting an entity resolver) must have been done in advance. +If you have XML in a string, you can use the :func:`parseString` function instead: .. function:: parseString(string, parser=None) - ... - + Return a :class:`DOMEventStream` that represents the (Unicode) *string*. .. data:: default_bufsize @@ -45,24 +83,37 @@ Object Model representation of a document from SAX events. The value of this variable can be changed before calling :func:`parse` and the new value will take effect. - .. _domeventstream-objects: DOMEventStream Objects ---------------------- +.. class:: DOMEventStream(stream, parser, bufsize) -.. method:: DOMEventStream.getEvent() - - ... + .. method:: getEvent() -.. method:: DOMEventStream.expandNode(node) + Return a tuple containing *event* and the current *node* as + :class:`xml.dom.minidom.Document` if event equals :data:`START_DOCUMENT`, + :class:`xml.dom.minidom.Element` if event equals :data:`START_ELEMENT` or + :data:`END_ELEMENT` or :class:`xml.dom.minidom.Text` if event equals + :data:`CHARACTERS`. + The current node does not contain informations about its children, unless + :func:`expandNode` is called. - ... + .. method:: expandNode(node) + Expands all children of *node* into *node*. Example:: -.. method:: DOMEventStream.reset() + xml = '<html><title>Foo</title> <p>Some text <div>and more</div></p> </html>' + doc = pulldom.parseString(xml) + for event, node in doc: + if event == pulldom.START_ELEMENT and node.tagName == 'p': + # Following statement only prints '<p/>' + print(node.toxml()) + doc.exandNode(node) + # Following statement prints node with all its children '<p>Some text <div>and more</div></p>' + print(node.toxml()) - ... + .. method:: DOMEventStream.reset() diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index a46d99d969..c5c880290c 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -95,11 +95,14 @@ Functions .. function:: iterparse(source, events=None, parser=None) Parses an XML section into an element tree incrementally, and reports what's - going on to the user. *source* is a filename or :term:`file object` containing - XML data. *events* is a list of events to report back. If omitted, only "end" - events are reported. *parser* is an optional parser instance. If not - given, the standard :class:`XMLParser` parser is used. Returns an - :term:`iterator` providing ``(event, elem)`` pairs. + going on to the user. *source* is a filename or :term:`file object` + containing XML data. *events* is a list of events to report back. The + supported events are the strings ``"start"``, ``"end"``, ``"start-ns"`` + and ``"end-ns"`` (the "ns" events are used to get detailed namespace + information). If *events* is omitted, only ``"end"`` events are reported. + *parser* is an optional parser instance. If not given, the standard + :class:`XMLParser` parser is used. Returns an :term:`iterator` providing + ``(event, elem)`` pairs. .. note:: |