#11584: make Header and make_header handle binary unknown-8bit input

Analogous to the decode_header fix, this fix makes Header.append and
make_header correctly handle the unknown-8bit charset introduced by email5.1,
when the input to them is binary strings.  Previous to this fix the
make_header(decode_header(x)) == x invariant was broken in the face of the
unknown-8bit charset.

1 files changed, 4 insertions, 1 deletions

diff --git a/Lib/email/header.py b/Lib/email/header.py
index 06708853c2..2e687b7a6f 100644
--- a/Lib/email/header.py
+++ b/Lib/email/header.py
@@ -275,7 +275,10 @@ class Header:
             charset = Charset(charset)
         if not isinstance(s, str):
             input_charset = charset.input_codec or 'us-ascii'
-            s = s.decode(input_charset, errors)
+            if input_charset == _charset.UNKNOWN8BIT:
+                s = s.decode('us-ascii', 'surrogateescape')
+            else:
+                s = s.decode(input_charset, errors)
         # Ensure that the bytes we're storing can be decoded to the output
         # character set, otherwise an early error is thrown.
         output_charset = charset.output_codec or 'us-ascii'