diff options
author | Iwan Aucamp <aucampia@gmail.com> | 2022-01-10 22:10:51 +0100 |
---|---|---|
committer | Iwan Aucamp <aucampia@gmail.com> | 2022-01-10 22:10:51 +0100 |
commit | 8028c230a542d6f1bf244f354febde4d44fb3c49 (patch) | |
tree | ce854b22ccd84f39a5d2b89aa44e7188d040275b | |
parent | d9575336024d135318ac2e6f7a11592675cb8c2f (diff) | |
download | rdflib-8028c230a542d6f1bf244f354febde4d44fb3c49.tar.gz |
Remove narrow build detection
Before Python 3.3 and [PEP 393](https://www.python.org/dev/peps/pep-0393/)
(which came in with Python 3.3) there was a distinction between narrow
and wide python builds.
However this is no longer the situation, and `chr(0x10FFFF)` should not
raise `ValueError` on any python build.
From [PEP 393](https://www.python.org/dev/peps/pep-0393/):
> The Unicode string type is changed to support multiple internal
> representations, depending on the character with the largest Unicode
> ordinal (1, 2, or 4 bytes). This will allow a space-efficient
> representation in common cases, but give access to full UCS-4 on all
> systems. For compatibility with existing APIs, several representations
> may exist in parallel; over time, this compatibility should be phased
> out. The distinction between narrow and wide Unicode builds is dropped.
From [Python 3.7 docs for Unicode Objects and Codecs](https://docs.python.org/3.7/c-api/unicode.html)
docs regarding `Py_UNICODE`:
> Changed in version 3.3: In previous versions, this was a 16-bit type
> or a 32-bit type depending on whether you selected a “narrow” or “wide”
> Unicode version of Python at build time.
-rw-r--r-- | rdflib/compat.py | 23 |
1 files changed, 0 insertions, 23 deletions
diff --git a/rdflib/compat.py b/rdflib/compat.py index 6020a630..139f2428 100644 --- a/rdflib/compat.py +++ b/rdflib/compat.py @@ -5,7 +5,6 @@ and different versions of support libraries. import re import codecs -import warnings import typing as t if t.TYPE_CHECKING: @@ -59,28 +58,6 @@ def _unicodeExpand(s): return r_unicodeEscape.sub(lambda m: chr(int(m.group(0)[2:], 16)), s) -narrow_build = False -try: - chr(0x10FFFF) -except ValueError: - narrow_build = True - -if narrow_build: - - def _unicodeExpand(s): - try: - return r_unicodeEscape.sub(lambda m: chr(int(m.group(0)[2:], 16)), s) - except ValueError: - warnings.warn( - "Encountered a unicode char > 0xFFFF in a narrow python build. " - "Trying to degrade gracefully, but this can cause problems " - "later when working with the string:\n%s" % s - ) - return r_unicodeEscape.sub( - lambda m: codecs.decode(m.group(0), "unicode_escape"), s - ) - - def decodeStringEscape(s): r""" s is byte-string - replace \ escapes in string |