summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDavid Lord <davidism@gmail.com>2020-04-20 00:31:06 -0700
committerDavid Lord <davidism@gmail.com>2020-04-20 07:12:53 -0700
commita1d726b5a244a831533e4d27a5be3975288de698 (patch)
tree556f74bc68f4c97bfe77b4c928c85114f0518c22
parent9897d81d0830f2048360cbd45993de1b37ce7371 (diff)
downloadclick-a1d726b5a244a831533e4d27a5be3975288de698.tar.gz
remove Python 2/3 from docs
-rw-r--r--docs/index.rst2
-rw-r--r--docs/python3.rst191
-rw-r--r--docs/quickstart.rst6
-rw-r--r--docs/unicode-support.rst112
-rw-r--r--docs/utils.rst34
-rw-r--r--docs/why.rst1
-rw-r--r--docs/wincmd.rst18
-rw-r--r--src/click/_unicodefun.py10
-rw-r--r--src/click/core.py4
-rw-r--r--src/click/utils.py10
10 files changed, 138 insertions, 250 deletions
diff --git a/docs/index.rst b/docs/index.rst
index 683f1ef..24afcc8 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -81,7 +81,7 @@ usage patterns.
utils
bashcomplete
exceptions
- python3
+ unicode-support
wincmd
API Reference
diff --git a/docs/python3.rst b/docs/python3.rst
deleted file mode 100644
index c1736a8..0000000
--- a/docs/python3.rst
+++ /dev/null
@@ -1,191 +0,0 @@
-Python 3 Support
-================
-
-.. currentmodule:: click
-
-Click supports Python 3, but like all other command line utility libraries,
-it suffers from the Unicode text model in Python 3. All examples in the
-documentation were written so that they could run on both Python 2.x and
-Python 3.4 or higher.
-
-.. _python3-limitations:
-
-Python 3 Limitations
---------------------
-
-At the moment, Click suffers from a few problems with Python 3:
-
-* The command line in Unix traditionally is in bytes, not Unicode. While
- there are encoding hints for all of this, there are generally some
- situations where this can break. The most common one is SSH
- connections to machines with different locales.
-
- Misconfigured environments can currently cause a wide range of Unicode
- problems in Python 3 due to the lack of support for roundtripping
- surrogate escapes. This will not be fixed in Click itself!
-
- For more information see :ref:`python3-surrogates`.
-
-* Standard input and output in Python 3 is opened in Unicode mode by
- default. Click has to reopen the stream in binary mode in certain
- situations. Because there is no standardized way to do this, this
- might not always work. Primarily this can become a problem when
- testing command-line applications.
-
- This is not supported::
-
- sys.stdin = io.StringIO('Input here')
- sys.stdout = io.StringIO()
-
- Instead you need to do this::
-
- input = 'Input here'
- in_stream = io.BytesIO(input.encode('utf-8'))
- sys.stdin = io.TextIOWrapper(in_stream, encoding='utf-8')
- out_stream = io.BytesIO()
- sys.stdout = io.TextIOWrapper(out_stream, encoding='utf-8')
-
- Remember that in that case, you need to use ``out_stream.getvalue()``
- and not ``sys.stdout.getvalue()`` if you want to access the buffer
- contents as the wrapper will not forward that method.
-
-Python 2 and 3 Differences
---------------------------
-
-Click attempts to minimize the differences between Python 2 and Python 3
-by following best practices for both languages.
-
-In Python 2, the following is true:
-
-* ``sys.stdin``, ``sys.stdout``, and ``sys.stderr`` are opened in binary
- mode, but under some circumstances they support Unicode output. Click
- attempts to not subvert this but provides support for forcing streams
- to be Unicode-based.
-* ``sys.argv`` is always byte-based. Click will pass bytes to all
- input types and convert as necessary. The :class:`STRING` type
- automatically will decode properly the input value into a string by
- trying the most appropriate encodings.
-* When dealing with files, Click will never go through the Unicode APIs
- and will instead use the operating system's byte APIs to open the
- files.
-
-In Python 3, the following is true:
-
-* ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are by default
- text-based. When Click needs a binary stream, it attempts to discover
- the underlying binary stream. See :ref:`python3-limitations` for how
- this works.
-* ``sys.argv`` is always Unicode-based. This also means that the native
- type for input values to the types in Click is Unicode, and not bytes.
-
- This causes problems if the terminal is incorrectly set and Python
- does not figure out the encoding. In that case, the Unicode string
- will contain error bytes encoded as surrogate escapes.
-* When dealing with files, Click will always use the Unicode file system
- API calls by using the operating system's reported or guessed
- filesystem encoding. Surrogates are supported for filenames, so it
- should be possible to open files through the :class:`File` type even
- if the environment is misconfigured.
-
-.. _python3-surrogates:
-
-Python 3 Surrogate Handling
----------------------------
-
-Click in Python 3 does all the Unicode handling in the standard library
-and is subject to its behavior. In Python 2, Click does all the Unicode
-handling itself, which means there are differences in error behavior.
-
-The most glaring difference is that in Python 2, Unicode will "just work",
-while in Python 3, it requires extra care. The reason for this is that in
-Python 3, the encoding detection is done in the interpreter, and on Linux
-and certain other operating systems, its encoding handling is problematic.
-
-The biggest source of frustration is that Click scripts invoked by
-init systems (sysvinit, upstart, systemd, etc.), deployment tools (salt,
-puppet), or cron jobs (cron) will refuse to work unless a Unicode locale is
-exported.
-
-If Click encounters such an environment it will prevent further execution
-to force you to set a locale. This is done because Click cannot know
-about the state of the system once it's invoked and restore the values
-before Python's Unicode handling kicked in.
-
-If you see something like this error in Python 3::
-
- Traceback (most recent call last):
- ...
- RuntimeError: Click will abort further execution because Python 3 was
- configured to use ASCII as encoding for the environment. Either switch
- to Python 2 or consult the Python 3 section of the docs for
- mitigation steps.
-
-.. note::
-
- In Python 3.7 and later you will no longer get a ``RuntimeError`` in
- many cases thanks to :pep:`538` and :pep:`540`, which changed the
- default assumption in unconfigured environments.
-
-You are dealing with an environment where Python 3 thinks you are
-restricted to ASCII data. The solution to these problems is different
-depending on which locale your computer is running in.
-
-For instance, if you have a German Linux machine, you can fix the problem
-by exporting the locale to ``de_DE.utf-8``::
-
- export LC_ALL=de_DE.utf-8
- export LANG=de_DE.utf-8
-
-If you are on a US machine, ``en_US.utf-8`` is the encoding of choice. On
-some newer Linux systems, you could also try ``C.UTF-8`` as the locale::
-
- export LC_ALL=C.UTF-8
- export LANG=C.UTF-8
-
-On some systems it was reported that `UTF-8` has to be written as `UTF8`
-and vice versa. To see which locales are supported you can invoke
-``locale -a``::
-
- locale -a
-
-You need to do this before you invoke your Python script. If you are
-curious about the reasons for this, you can join the discussions in the
-Python 3 bug tracker:
-
-* `ASCII is a bad filesystem default encoding
- <https://bugs.python.org/issue13643#msg149941>`_
-* `Use surrogateescape as default error handler
- <https://bugs.python.org/issue19977>`_
-* `Python 3 raises Unicode errors in the C locale
- <https://bugs.python.org/issue19846>`_
-* `LC_CTYPE=C: pydoc leaves terminal in an unusable state
- <https://bugs.python.org/issue21398>`_ (this is relevant to Click
- because the pager support is provided by the stdlib pydoc module)
-
-Note (Python 3.7 onwards): Even though your locale may not be properly
-configured, Python 3.7 Click will not raise the above exception because Python
-3.7 programs are better at choosing default locales. This doesn't change the
-general issue that your locale may be misconfigured.
-
-Unicode Literals
-----------------
-
-Starting with Click 5.0 there will be a warning for the use of the
-``unicode_literals`` future import in Python 2. This has been done due to
-the negative consequences of this import with regards to unintentionally
-causing bugs due to introducing Unicode data to APIs that are incapable of
-handling them. For some examples of this issue, see the discussion on
-this github issue: `python-future#22
-<https://github.com/PythonCharmers/python-future/issues/22>`_.
-
-If you use ``unicode_literals`` in any file that defines a Click command
-or that invokes a click command you will be given a warning. You are
-strongly encouraged to not use ``unicode_literals`` and instead use
-explicit ``u`` prefixes for your Unicode strings.
-
-If you do want to ignore the warning and continue using
-``unicode_literals`` on your own peril, you can disable the warning as
-follows::
-
- import click
- click.disable_unicode_literals_warning = True
diff --git a/docs/quickstart.rst b/docs/quickstart.rst
index 51f7f22..fd6bce4 100644
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@@ -157,7 +157,7 @@ Echoing
Why does this example use :func:`echo` instead of the regular
:func:`print` function? The answer to this question is that Click
-attempts to support both Python 2 and Python 3 the same way and to be very
+attempts to support different environments consistently and to be very
robust even when the environment is misconfigured. Click wants to be
functional at least on a basic level even if everything is completely
broken.
@@ -169,9 +169,7 @@ correction in case the terminal is misconfigured instead of dying with an
As an added benefit, starting with Click 2.0, the echo function also
has good support for ANSI colors. It will automatically strip ANSI codes
if the output stream is a file and if colorama is supported, ANSI colors
-will also work on Windows. Note that in Python 2, the :func:`echo` function
-does not parse color code information from bytearrays. See :ref:`ansi-colors`
-for more information.
+will also work on Windows. See :ref:`ansi-colors`.
If you don't need this, you can also use the `print()` construct /
function.
diff --git a/docs/unicode-support.rst b/docs/unicode-support.rst
new file mode 100644
index 0000000..680e739
--- /dev/null
+++ b/docs/unicode-support.rst
@@ -0,0 +1,112 @@
+Unicode Support
+===============
+
+.. currentmodule:: click
+
+Click has to take extra care to support Unicode text in different
+environments.
+
+* The command line in Unix is traditionally bytes, not Unicode. While
+ there are encoding hints, there are some situations where this can
+ break. The most common one is SSH connections to machines with
+ different locales.
+
+ Misconfigured environments can cause a wide range of Unicode
+ problems due to the lack of support for roundtripping surrogate
+ escapes. This will not be fixed in Click itself!
+
+* Standard input and output is opened in text mode by default. Click
+ has to reopen the stream in binary mode in certain situations.
+ Because there is no standard way to do this, it might not always
+ work. Primarily this can become a problem when testing command-line
+ applications.
+
+ This is not supported::
+
+ sys.stdin = io.StringIO('Input here')
+ sys.stdout = io.StringIO()
+
+ Instead you need to do this::
+
+ input = 'Input here'
+ in_stream = io.BytesIO(input.encode('utf-8'))
+ sys.stdin = io.TextIOWrapper(in_stream, encoding='utf-8')
+ out_stream = io.BytesIO()
+ sys.stdout = io.TextIOWrapper(out_stream, encoding='utf-8')
+
+ Remember in that case, you need to use ``out_stream.getvalue()``
+ and not ``sys.stdout.getvalue()`` if you want to access the buffer
+ contents as the wrapper will not forward that method.
+
+* ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are by default
+ text-based. When Click needs a binary stream, it attempts to
+ discover the underlying binary stream.
+
+* ``sys.argv`` is always text. This means that the native type for
+ input values to the types in Click is Unicode, not bytes.
+
+ This causes problems if the terminal is incorrectly set and Python
+ does not figure out the encoding. In that case, the Unicode string
+ will contain error bytes encoded as surrogate escapes.
+
+* When dealing with files, Click will always use the Unicode file
+ system API by using the operating system's reported or guessed
+ filesystem encoding. Surrogates are supported for filenames, so it
+ should be possible to open files through the :class:`File` type even
+ if the environment is misconfigured.
+
+
+Surrogate Handling
+------------------
+
+Click does all the Unicode handling in the standard library and is
+subject to its behavior. Unicode requires extra care. The reason for
+this is that the encoding detection is done in the interpreter, and on
+Linux and certain other operating systems, its encoding handling is
+problematic.
+
+The biggest source of frustration is that Click scripts invoked by init
+systems, deployment tools, or cron jobs will refuse to work unless a
+Unicode locale is exported.
+
+If Click encounters such an environment it will prevent further
+execution to force you to set a locale. This is done because Click
+cannot know about the state of the system once it's invoked and restore
+the values before Python's Unicode handling kicked in.
+
+If you see something like this error::
+
+ Traceback (most recent call last):
+ ...
+ RuntimeError: Click will abort further execution because Python was
+ configured to use ASCII as encoding for the environment. Consult
+ https://click.palletsprojects.com/unicode-support/ for mitigation
+ steps.
+
+You are dealing with an environment where Python thinks you are
+restricted to ASCII data. The solution to these problems is different
+depending on which locale your computer is running in.
+
+For instance, if you have a German Linux machine, you can fix the
+problem by exporting the locale to ``de_DE.utf-8``::
+
+ export LC_ALL=de_DE.utf-8
+ export LANG=de_DE.utf-8
+
+If you are on a US machine, ``en_US.utf-8`` is the encoding of choice.
+On some newer Linux systems, you could also try ``C.UTF-8`` as the
+locale::
+
+ export LC_ALL=C.UTF-8
+ export LANG=C.UTF-8
+
+On some systems it was reported that ``UTF-8`` has to be written as
+``UTF8`` and vice versa. To see which locales are supported you can
+invoke ``locale -a``.
+
+You need to export the values before you invoke your Python script.
+
+In Python 3.7 and later you will no longer get a ``RuntimeError`` in
+many cases thanks to :pep:`538` and :pep:`540`, which changed the
+default assumption in unconfigured environments. This doesn't change the
+general issue that your locale may be misconfigured.
diff --git a/docs/utils.rst b/docs/utils.rst
index 7dd8dbb..6338df9 100644
--- a/docs/utils.rst
+++ b/docs/utils.rst
@@ -13,9 +13,7 @@ Printing to Stdout
The most obvious helper is the :func:`echo` function, which in many ways
works like the Python ``print`` statement or function. The main difference is
-that it works the same in Python 2 and 3, it intelligently detects
-misconfigured output streams, and it will never fail (except in Python 3; for
-more information see :ref:`python3-limitations`).
+that it works the same in many different terminal environments.
Example::
@@ -23,10 +21,8 @@ Example::
click.echo('Hello World!')
-Most importantly, it can print both Unicode and binary data, unlike the
-built-in ``print`` function in Python 3, which cannot output any bytes. It
-will, however, emit a trailing newline by default, which needs to be
-suppressed by passing ``nl=False``::
+It can output both text and binary data. It will emit a trailing newline
+by default, which needs to be suppressed by passing ``nl=False``::
click.echo(b'\xe2\x98\x83', nl=False)
@@ -34,19 +30,17 @@ Last but not least :func:`echo` uses click's intelligent internal output
streams to stdout and stderr which support unicode output on the Windows
console. This means for as long as you are using `click.echo` you can
output unicode characters (there are some limitations on the default font
-with regards to which characters can be displayed). This functionality is
-new in Click 6.0.
+with regards to which characters can be displayed).
.. versionadded:: 6.0
-Click now emulates output streams on Windows to support unicode to the
+Click emulates output streams on Windows to support unicode to the
Windows console through separate APIs. For more information see
:doc:`wincmd`.
.. versionadded:: 3.0
-Starting with Click 3.0 you can also easily print to standard error by
-passing ``err=True``::
+You can also easily print to standard error by passing ``err=True``::
click.echo('Hello World!', err=True)
@@ -58,11 +52,10 @@ ANSI Colors
.. versionadded:: 2.0
-Starting with Click 2.0, the :func:`echo` function gained extra
-functionality to deal with ANSI colors and styles. Note that on Windows,
-this functionality is only available if `colorama`_ is installed. If it
-is installed, then ANSI codes are intelligently handled. Note that in Python
-2, the echo function doesn't parse color code information from bytearrays.
+The :func:`echo` function gained extra functionality to deal with ANSI
+colors and styles. Note that on Windows, this functionality is only
+available if `colorama`_ is installed. If it is installed, then ANSI
+codes are intelligently handled.
Primarily this means that:
@@ -252,9 +245,7 @@ Printing Filenames
------------------
Because filenames might not be Unicode, formatting them can be a bit
-tricky. Generally, this is easier in Python 2 than on 3, as you can just
-write the bytes to stdout with the ``print`` function, but in Python 3, you
-will always need to operate in Unicode.
+tricky.
The way this works with click is through the :func:`format_filename`
function. It does a best-effort conversion of the filename to Unicode and
@@ -280,8 +271,7 @@ Because of this, click provides the :func:`get_binary_stream` and
different Python versions and for a wide variety of terminal configurations.
The end result is that these functions will always return a functional
-stream object (except in very odd cases in Python 3; see
-:ref:`python3-limitations`).
+stream object (except in very odd cases; see :doc:`/unicode-support`).
Example::
diff --git a/docs/why.rst b/docs/why.rst
index 9418bfe..d091213 100644
--- a/docs/why.rst
+++ b/docs/why.rst
@@ -12,7 +12,6 @@ line utility for Python out there which ticks the following boxes:
* supports loading values from environment variables out of the box
* support for prompting of custom values
* is fully nestable and composable
-* works the same in Python 2 and 3
* supports file handling out of the box
* comes with useful common helpers (getting terminal dimensions,
ANSI colors, fetching direct keyboard input, screen clearing,
diff --git a/docs/wincmd.rst b/docs/wincmd.rst
index 901ee95..5727f2f 100644
--- a/docs/wincmd.rst
+++ b/docs/wincmd.rst
@@ -3,11 +3,7 @@ Windows Console Notes
.. versionadded:: 6.0
-Until Click 6.0 there are various bugs and limitations with using Click on
-a Windows console. Most notably the decoding of command line arguments
-was performed with the wrong encoding on Python 2 and on all versions of
-Python output of unicode characters was impossible. Starting with Click
-6.0 we now emulate output streams on Windows to support unicode to the
+Click emulates output streams on Windows to support unicode to the
Windows console through separate APIs and we perform different decoding of
parameters.
@@ -22,18 +18,10 @@ performed to the type expected value as late as possible. This has some
advantages as it allows us to accept the data in the most appropriate form
for the operating system and Python version.
-For instance paths are left as bytes on Python 2 unless you explicitly
-tell it otherwise.
-
This caused some problems on Windows where initially the wrong encoding
was used and garbage ended up in your input data. We not only fixed the
encoding part, but we also now extract unicode parameters from `sys.argv`.
-This means that on Python 2 under Windows, the arguments processed will
-*most likely* be of unicode nature and not bytes. This was something that
-previously did not really happen unless you explicitly passed in unicode
-parameters so your custom types need to be aware of this.
-
There is also another limitation with this: if `sys.argv` was modified
prior to invoking a click handler, we have to fall back to the regular
byte input in which case not all unicode values are available but only a
@@ -55,10 +43,6 @@ stream will also use ``utf-16-le`` as internal encoding. However there is
some hackery going on that the underlying raw IO buffer is still bypassing
the unicode APIs and byte output through an indirection is still possible.
-This hackery is used on both Python 2 and Python 3 as neither version of
-Python has native support for cmd.exe with unicode characters. There are
-some limitations you need to be aware of:
-
* This unicode support is limited to ``click.echo``, ``click.prompt`` as
well as ``click.get_text_stream``.
* Depending on if unicode values or byte strings are passed the control
diff --git a/src/click/_unicodefun.py b/src/click/_unicodefun.py
index 7f3f234..53ec9d2 100644
--- a/src/click/_unicodefun.py
+++ b/src/click/_unicodefun.py
@@ -2,8 +2,8 @@ import codecs
import os
-def _verify_python3_env():
- """Ensures that the environment is good for unicode on Python 3."""
+def _verify_python_env():
+ """Ensures that the environment is good for Unicode."""
try:
import locale
@@ -75,8 +75,8 @@ def _verify_python3_env():
)
raise RuntimeError(
- "Click will abort further execution because Python 3 was"
+ "Click will abort further execution because Python was"
" configured to use ASCII as encoding for the environment."
- " Consult https://click.palletsprojects.com/python3/ for"
- f" mitigation steps.{extra}"
+ " Consult https://click.palletsprojects.com/unicode-support/"
+ f" for mitigation steps.{extra}"
)
diff --git a/src/click/core.py b/src/click/core.py
index e4061aa..b7124df 100644
--- a/src/click/core.py
+++ b/src/click/core.py
@@ -6,7 +6,7 @@ from contextlib import contextmanager
from functools import update_wrapper
from itertools import repeat
-from ._unicodefun import _verify_python3_env
+from ._unicodefun import _verify_python_env
from .exceptions import Abort
from .exceptions import BadParameter
from .exceptions import ClickException
@@ -787,7 +787,7 @@ class BaseCommand:
"""
# Verify that the environment is configured correctly, or reject
# further execution to avoid a broken script.
- _verify_python3_env()
+ _verify_python_env()
if args is None:
args = sys.argv[1:]
diff --git a/src/click/utils.py b/src/click/utils.py
index ffd26b3..bd9dd8e 100644
--- a/src/click/utils.py
+++ b/src/click/utils.py
@@ -265,11 +265,7 @@ def echo(message=None, file=None, nl=True, err=False, color=None):
def get_binary_stream(name):
- """Returns a system stream for byte processing. This essentially
- returns the stream from the sys module with the given name but it
- solves some compatibility issues between different Python versions.
- Primarily this function is necessary for getting binary streams on
- Python 3.
+ """Returns a system stream for byte processing.
:param name: the name of the stream to open. Valid names are ``'stdin'``,
``'stdout'`` and ``'stderr'``
@@ -283,8 +279,8 @@ def get_binary_stream(name):
def get_text_stream(name, encoding=None, errors="strict"):
"""Returns a system stream for text processing. This usually returns
a wrapped stream around a binary stream returned from
- :func:`get_binary_stream` but it also can take shortcuts on Python 3
- for already correctly configured streams.
+ :func:`get_binary_stream` but it also can take shortcuts for already
+ correctly configured streams.
:param name: the name of the stream to open. Valid names are ``'stdin'``,
``'stdout'`` and ``'stderr'``