Closes #25411: Merged fix from 3.5.

2 files changed, 273 insertions, 101 deletions

diff --git a/Misc/ACKS b/Misc/ACKS
index 120956ee23..aa80077027 100644
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -151,6 +151,7 @@ Finn Bock
 Paul Boddie
 Matthew Boedicker
 Robin Boerdijk
+Nikolay Bogoychev
 David Bolen
 Wouter Bolsterlee
 Gawain Bolton
@@ -879,6 +880,7 @@ Kang-Hao (Kenny) Lu
 Lukas Lueg
 Loren Luke
 Fredrik Lundh
+Mike Lundy
 Zhongyue Luo
 Mark Lutz
 Taras Lyapun
@@ -1415,6 +1417,7 @@ William Tanksley
 Christian Tanzer
 Steven Taschuk
 Amy Taylor
+Julian Taylor
 Monty Taylor
 Anatoly Techtonik
 Gustavo Temple
diff --git a/Misc/NEWS b/Misc/NEWS
index 71009af0a3..0cb9c183b3 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1,16 +1,21 @@
-+++++++++++
++++++++++++
 Python News
 +++++++++++
 
+What's New in Python 3.6.0 alpha 1?
+===================================
 
-What's New in Python 3.5.1 release candidate 1?
-===============================================
-
-Release date: TBA
+Release date: XXXX-XX-XX
 
 Core and Builtins
 -----------------
 
+- Issue #25401: Optimize bytes.fromhex() and bytearray.fromhex(): they are now
+  between 2x and 3.5x faster.
+
+- Issue #25399: Optimize bytearray % args using the new private _PyBytesWriter
+  API. Formatting is now between 2.5 and 5 times faster.
+
 - Issue #25274: sys.setrecursionlimit() now raises a RecursionError if the new
   recursion limit is too low depending at the current recursion depth. Modify
   also the "lower-water mark" formula to make it monotonic. This mark is used
@@ -19,28 +24,41 @@ Core and Builtins
 - Issue #24402: Fix input() to prompt to the redirected stdout when
   sys.stdout.fileno() fails.
 
+- Issue #25349: Optimize bytes % args using the new private _PyBytesWriter API.
+  Formatting is now up to 2 times faster.
+
 - Issue #24806: Prevent builtin types that are not allowed to be subclassed from
   being subclassed through multiple inheritance.
 
+* Issue #25301: The UTF-8 decoder is now up to 15 times as fast for error
+  handlers: ``ignore``, ``replace`` and ``surrogateescape``.
+
 - Issue #24848: Fixed a number of bugs in UTF-7 decoding of misformed data.
 
+- Issue #25267: The UTF-8 encoder is now up to 75 times as fast for error
+  handlers: ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass``.
+  Patch co-written with Serhiy Storchaka.
+
 - Issue #25280: Import trace messages emitted in verbose (-v) mode are no
   longer formatted twice.
 
+- Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape``
+  error handler: the encoders are now up to 3 times as fast. Initial patch
+  written by Serhiy Storchaka.
+
 - Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the
   getrandom() function instead of the getentropy() function. The getentropy()
   function is blocking to generate very good quality entropy, os.urandom()
   doesn't need such high-quality entropy.
 
-- Issue #25182: The stdprinter (used as sys.stderr before the io module is
-  imported at startup) now uses the backslashreplace error handler.
-
-- Issue #25131: Make the line number and column offset of set/dict literals and
-  comprehensions correspond to the opening brace.
+- Issue #9232: Modify Python's grammar to allow trailing commas in the
+  argument list of a function declaration.  For example, "def f(\*, a =
+  3,): pass" is now legal. Patch from Mark Dickinson.
 
-- Issue #25150: Hide the private _Py_atomic_xxx symbols from the public
-  Python.h header to fix a compilation error with OpenMP. PyThreadState_GET()
-  becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies.
+- Issue #24965: Implement PEP 498 "Literal String Interpolation". This
+  allows you to embed expressions inside f-strings, which are
+  converted to normal strings at run time. Given x=3, then
+  f'value={x}' == 'value=3'. Patch by Eric V. Smith.
 
 Library
 -------
@@ -48,6 +66,8 @@ Library
 - Issue #25411: Improved Unicode support in SMTPHandler through better use of
   the email package. Thanks to user simon04 for the patch.
 
+- Move the imp module  from a PendingDeprecationWarning to DeprecationWarning.
+
 - Issue #25407: Remove mentions of the formatter module being removed in
   Python 3.6.
 
@@ -55,11 +75,23 @@ Library
   that caused segmentation fault or hang in iterating after moving several
   items to the start of ordered dict.
 
+- Issue #25382: pickletools.dis() now outputs implicit memo index for the
+  MEMOIZE opcode.
+
+- Issue #25357: Add an optional newline paramer to binascii.b2a_base64().
+  base64.b64encode() uses it to avoid a memory copy.
+
+- Issue #24164: Objects that need calling ``__new__`` with keyword arguments,
+  can now be pickled using pickle protocols older than protocol version 4.
+
 - Issue #25364: zipfile now works in threads disabled builds.
 
 - Issue #25328: smtpd's SMTPChannel now correctly raises a ValueError if both
   decode_data and enable_SMTPUTF8 are set to true.
 
+- Issue #16099: RobotFileParser now supports Crawl-delay and Request-rate
+  extensions.  Patch by Nikolay Bogoychev.
+
 - Issue #25316: distutils raises OSError instead of DistutilsPlatformError
   when MSVC is not installed.
 
@@ -82,6 +114,221 @@ Library
 - Issue #24657: Prevent CGIRequestHandler from collapsing slashes in the
   query part of the URL as if it were a path. Patch from Xiang Zhang.
 
+- Issue #25287: Don't add crypt.METHOD_CRYPT to crypt.methods if it's not
+  supported. Check if it is supported, it may not be supported on OpenBSD for
+  example.
+
+- Issue #23600: Default implementation of tzinfo.fromutc() was returning
+  wrong results in some cases.
+
+- Issue #25203: Failed readline.set_completer_delims() no longer left the
+  module in inconsistent state.
+
+- Issue #25011: rlcomplete now omits private and special attribute names unless
+  the prefix starts with underscores.
+
+- Issue #25209: rlcomplete now can add a space or a colon after completed keyword.
+
+- Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'.
+
+- Issue #23517: fromtimestamp() and utcfromtimestamp() methods of
+  datetime.datetime now round microseconds to nearest with ties going to
+  nearest even integer (ROUND_HALF_EVEN), as round(float), instead of rounding
+  towards -Infinity (ROUND_FLOOR).
+
+- Issue #23552: Timeit now warns when there is substantial (4x) variance
+  between best and worst times. Patch from Serhiy Storchaka.
+
+- Issue #24633: site-packages/README -> README.txt.
+
+- Issue #24879:  help() and pydoc can now list named tuple fields in the
+  order they were defined rather than alphabetically.  The ordering is
+  determined by the _fields attribute if present.
+
+- Issue #24874:  Improve speed of itertools.cycle() and make its
+  pickle more compact.
+
+- Fix crash in itertools.cycle.__setstate__() when the first argument wasn't
+  a list.
+
+- Issue #20059: urllib.parse raises ValueError on all invalid ports.
+  Patch by Martin Panter.
+
+- Issue #24360: Improve __repr__ of argparse.Namespace() for invalid
+  identifiers.  Patch by Matthias Bussonnier.
+
+- Issue #23426: run_setup was broken in distutils.
+  Patch from Alexander Belopolsky.
+
+- Issue #13938: 2to3 converts StringTypes to a tuple. Patch from Mark Hammond.
+
+- Issue #2091: open() accepted a 'U' mode string containing '+', but 'U' can
+  only be used with 'r'. Patch from Jeff Balogh and John O'Connor.
+
+- Issue #8585: improved tests for zipimporter2. Patch from Mark Lawrence.
+
+- Issue #18622: unittest.mock.mock_open().reset_mock would recurse infinitely.
+  Patch from Nicola Palumbo and Laurent De Buyst.
+
+- Issue #24426: Fast searching optimization in regular expressions now works
+  for patterns that starts with capturing groups.  Fast searching optimization
+  now can't be disabled at compile time.
+
+- Issue #23661: unittest.mock side_effects can now be exceptions again. This
+  was a regression vs Python 3.4. Patch from Ignacio Rossi
+
+- Issue #13248: Remove deprecated inspect.getargspec and inspect.getmoduleinfo
+  functions.
+
+- Issue #25319: When threading.Event is reinitialized, the underlying condition
+  should use a regular lock rather than a recursive lock.
+
+IDLE
+----
+
+- Issue #24820: IDLE now has an 'IDLE Dark' built-in text color theme.
+  It is more or less IDLE Classic inverted, with a cobalt blue background.
+  Strings, comments, keywords, ... are still green, red, orange, ... .
+  Selecting it displays the following warning and solution.
+  "IDLE Dark is new in October, 2015.  Trying to run earlier versions
+  of IDLE with it selected will disable colorizing, or worse.
+  If you might ever run an earlier release of IDLE, then before
+  exiting this version, either switch to another theme or hit the
+  'Save as New Custom Theme' button.  The latter requires a new name,
+  such as 'Custom Dark', but the custom theme will work with any IDLE
+  release, and can be modified."
+
+- Issue #25224: README.txt is now an idlelib index for IDLE developers and
+  curious users.  The previous user content is now in the IDLE doc and is
+  redundant.  IDLE now means 'Integrated Development and Learning Environment'.
+
+- Issue #24820: Users can now set breakpoint colors in
+  Settings -> Custom Highlighting.  Original patch by Mark Roseman.
+
+- Issue #24972: Inactive selection background now matches active selection
+  background, as configured by user, on all systems.  Found items are now
+  always highlighted on Windows.  Initial patch by Mark Roseman.
+
+- Issue #24570: Idle: make calltip and completion boxes appear on Macs
+  affected by a tk regression.  Initial patch by Mark Roseman.
+
+- Issue #24988: Idle ScrolledList context menus (used in debugger)
+  now work on Mac Aqua.  Patch by Mark Roseman.
+
+- Issue #24801: Make right-click for context menu work on Mac Aqua.
+  Patch by Mark Roseman.
+
+- Issue #25173: Associate tkinter messageboxes with a specific widget.
+  For Mac OSX, make them a 'sheet'.  Patch by Mark Roseman.
+
+- Issue #25198: Enhance the initial html viewer now used for Idle Help.
+  * Properly indent fixed-pitch text (patch by Mark Roseman).
+  * Give code snippet a very Sphinx-like light blueish-gray background.
+  * Re-use initial width and height set by users for shell and editor.
+  * When the Table of Contents (TOC) menu is used, put the section header
+  at the top of the screen.
+
+- Issue #25225: Condense and rewrite Idle doc section on text colors.
+
+- Issue #21995: Explain some differences between IDLE and console Python.
+
+- Issue #22820: Explain need for *print* when running file from Idle editor.
+
+- Issue #25224: Doc: augment Idle feature list and no-subprocess section.
+
+- Issue #25219: Update doc for Idle command line options.
+  Some were missing and notes were not correct.
+
+- Issue #24861: Most of idlelib is private and subject to change.
+  Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__.
+
+- Issue #25199: Idle: add synchronization comments for future maintainers.
+
+- Issue #16893: Replace help.txt with help.html for Idle doc display.
+  The new idlelib/help.html is rstripped Doc/build/html/library/idle.html.
+  It looks better than help.txt and will better document Idle as released.
+  The tkinter html viewer that works for this file was written by Rose Roseman.
+  The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated.
+
+- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6.
+
+- Issue #24782: In Idle extension config dialog, replace tabs with sorted list.
+  Patch by Mark Roseman.
+
+- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts).
+
+Documentation
+-------------
+
+- Issue #24952: Clarify the default size argument of stack_size() in
+  the "threading" and "_thread" modules. Patch from Mattip.
+
+Tests
+-----
+
+- Issue #25188: Add -P/--pgo to test.regrtest to suppress error output when
+  running the test suite for the purposes of a PGO build. Initial patch by
+  Alecsandru Patrascu.
+
+- Issue #22806: Add ``python -m test --list-tests`` command to list tests.
+
+- Issue #18174: ``python -m test --huntrleaks ...`` now also checks for leak of
+  file descriptors. Patch written by Richard Oudkerk.
+
+- Issue #25260: Fix ``python -m test --coverage`` on Windows. Remove the
+  list of ignored directories.
+
+- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along
+  to regrtest.py.  Previously there was a limit of 9.
+
+Build
+-----
+
+- Issue #24986: It is now possible to build Python on Windows without errors
+  when external libraries are not available.
+
+Windows
+-------
+
+- Issue #25022: Removed very outdated PC/example_nt/ directory.
+
+Tools/Demos
+-----------
+
+- Issue #25154: The pyvenv script has been deprecated in favour of
+  `python3 -m venv`.
+
+
+What's New in Python 3.5.1 release candidate 1?
+===============================================
+
+Release date: TBA
+
+Core and Builtins
+-----------------
+
+- Issue #24402: Fix input() to prompt to the redirected stdout when
+  sys.stdout.fileno() fails.
+
+- Issue #25182: The stdprinter (used as sys.stderr before the io module is
+  imported at startup) now uses the backslashreplace error handler.
+
+- Issue #25131: Make the line number and column offset of set/dict literals and
+  comprehensions correspond to the opening brace.
+
+- Issue #25150: Hide the private _Py_atomic_xxx symbols from the public
+  Python.h header to fix a compilation error with OpenMP. PyThreadState_GET()
+  becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies.
+
+Library
+-------
+
+- Issue #25232: Fix CGIRequestHandler to split the query from the URL at the
+  first question mark (?) rather than the last. Patch from Xiang Zhang.
+
+- Issue #24657: Prevent CGIRequestHandler from collapsing slashes in the
+  query part of the URL as if it were a path. Patch from Xiang Zhang.
+
 - Issue #24483: C implementation of functools.lru_cache() now calculates key's
   hash only once.
 
@@ -103,9 +350,6 @@ Library
 - Issue #25233: Rewrite the guts of asyncio.Queue and
   asyncio.Semaphore to be more understandable and correct.
 
-- Issue #25203: Failed readline.set_completer_delims() no longer left the
-  module in inconsistent state.
-
 - Issue #23600: Default implementation of tzinfo.fromutc() was returning
   wrong results in some cases.
 
@@ -179,9 +423,6 @@ Library
 - Issue #24881: Fixed setting binary mode in Python implementation of FileIO
   on Windows and Cygwin.  Patch from Akira Li.
 
-- Issue #25319: When threading.Event is reinitialized, the underlying condition
-  should use a regular lock rather than a recursive lock.
-
 - Issue #21112: Fix regression in unittest.expectedFailure on subclasses.
   Patch from Berker Peksag.
 
@@ -203,81 +444,8 @@ Library
 - Issue #23572: Fixed functools.singledispatch on classes with falsy
   metaclasses.  Patch by Ethan Furman.
 
-- asyncio: ensure_future() now accepts awaitable objects.
-
-IDLE
-----
-
-- Issue #24820: IDLE now has an 'IDLE Dark' built-in text color theme.
-  It is more or less IDLE Classic inverted, with a cobalt blue background.
-  Strings, comments, keywords, ... are still green, red, orange, ... .
-  Selecting it displays the following warning and solution.
-  "IDLE Dark is new in October, 2015.  Trying to run earlier versions
-  of IDLE with it selected will disable colorizing, or worse.
-  If you might ever run an earlier release of IDLE, then before
-  exiting this version, either switch to another theme or hit the
-  'Save as New Custom Theme' button.  The latter requires a new name,
-  such as 'Custom Dark', but the custom theme will work with any IDLE
-  release, and can be modified."
-
-- Issue #25224: README.txt is now an idlelib index for IDLE developers and
-  curious users.  The previous user content is now in the IDLE doc and is
-  redundant.  IDLE now means 'Integrated Development and Learning Environment'.
-
-- Issue #24820: Users can now set breakpoint colors in
-  Settings -> Custom Highlighting.  Original patch by Mark Roseman.
-
-- Issue #24972: Inactive selection background now matches active selection
-  background, as configured by user, on all systems.  Found items are now
-  always highlighted on Windows.  Initial patch by Mark Roseman.
-
-- Issue #24570: Idle: make calltip and completion boxes appear on Macs
-  affected by a tk regression.  Initial patch by Mark Roseman.
-
-- Issue #24988: Idle ScrolledList context menus (used in debugger)
-  now work on Mac Aqua.  Patch by Mark Roseman.
-
-- Issue #24801: Make right-click for context menu work on Mac Aqua.
-  Patch by Mark Roseman.
-
-- Issue #25173: Associate tkinter messageboxes with a specific widget.
-  For Mac OSX, make them a 'sheet'.  Patch by Mark Roseman.
-
-- Issue #25198: Enhance the initial html viewer now used for Idle Help.
-  * Properly indent fixed-pitch text (patch by Mark Roseman).
-  * Give code snippet a very Sphinx-like light blueish-gray background.
-  * Re-use initial width and height set by users for shell and editor.
-  * When the Table of Contents (TOC) menu is used, put the section header
-  at the top of the screen.
-
-- Issue #25225: Condense and rewrite Idle doc section on text colors.
-
-- Issue #21995: Explain some differences between IDLE and console Python.
-
-- Issue #22820: Explain need for *print* when running file from Idle editor.
-
-- Issue #25224: Doc: augment Idle feature list and no-subprocess section.
-
-- Issue #25219: Update doc for Idle command line options.
-  Some were missing and notes were not correct.
-
-- Issue #24861: Most of idlelib is private and subject to change.
-  Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__.
-
-- Issue #25199: Idle: add synchronization comments for future maintainers.
-
-- Issue #16893: Replace help.txt with help.html for Idle doc display.
-  The new idlelib/help.html is rstripped Doc/build/html/library/idle.html.
-  It looks better than help.txt and will better document Idle as released.
-  The tkinter html viewer that works for this file was written by Mark Roseman.
-  The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated.
-
-- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6.
-
-- Issue #24782: In Idle extension config dialog, replace tabs with sorted list.
-  Patch by Mark Roseman.
-
-- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts).
+- Issue #12006: Add ISO 8601 year, week, and day directives (%G, %V, %u) to
+  strptime.
 
 Documentation
 -------------
@@ -323,6 +491,7 @@ Build
 - Issue #24986: It is now possible to build Python on Windows without errors
   when external libraries are not available.
 
+
 Windows
 -------
 
@@ -359,6 +528,7 @@ Windows
 
 - Issue #25022: Removed very outdated PC/example_nt/ directory.
 
+
 What's New in Python 3.5.0 final?
 =================================
 
@@ -601,12 +771,6 @@ Library
 
 - Issue #24631: Fixed regression in the timeit module with multiline setup.
 
-- Issue #18622: unittest.mock.mock_open().reset_mock would recurse infinitely.
-  Patch from Nicola Palumbo and Laurent De Buyst.
-
-- Issue #23661: unittest.mock side_effects can now be exceptions again. This
-  was a regression vs Python 3.4. Patch from Ignacio Rossi
-
 - Issue #24608: chunk.Chunk.read() now always returns bytes, not str.
 
 - Issue #18684: Fixed reading out of the buffer in the re module.
@@ -617,6 +781,9 @@ Library
 - Issue #15014: SMTP.auth() and SMTP.login() now support RFC 4954's optional
   initial-response argument to the SMTP AUTH command.
 
+- Issue #6549: Remove hyphen from ttk.Style().element options.  Only return result
+  from ttk.Style().configure if a result was generated or a query submitted.
+
 - Issue #24669: Fix inspect.getsource() for 'async def' functions.
   Patch by Kai Groner.
 
@@ -628,7 +795,6 @@ Build
 - Issue #24603: Update Windows builds and OS X 10.5 installer to use OpenSSL
   1.0.2d.
 
-
 What's New in Python 3.5.0 beta 3?
 ==================================
 
@@ -789,6 +955,9 @@ Core and Builtins
 - Issue #24268: PEP 489: Multi-phase extension module initialization.
   Patch by Petr Viktorin.
 
+- Issue #23359: Optimize set object internals by specializing the
+  hash table search into a lookup function and an insert function.
+
 - Issue #23955: Add pyvenv.cfg option to suppress registry/environment
   lookup for generating sys.path on Windows.