diff options
author | Matthew Pickering <matthewtpickering@gmail.com> | 2021-06-11 10:48:25 +0100 |
---|---|---|
committer | Marge Bot <ben+marge-bot@smart-cactus.org> | 2021-06-23 02:58:35 -0400 |
commit | 7f6454fb8cd92b2b2ad4e88fa6d81e34d43edb9a (patch) | |
tree | 22dbe8c64e1761856913450fa297e2797c905fa2 /compiler/GHC | |
parent | 87f57ecf2523e83d8dd9cad919a6f2010f630ad0 (diff) | |
download | haskell-7f6454fb8cd92b2b2ad4e88fa6d81e34d43edb9a.tar.gz |
Optimiser: Correctly deal with strings starting with unicode characters in exprConApp_maybe
For example:
"\0" is encoded to "C0 80", then the rule would correct use a decoding
function to work out the first character was "C0 80" but then just used
BS.tail so the rest of the string was "80". This resulted in
"\0" being transformed into '\C0\80' : unpackCStringUTF8# "80"
Which is obviously bogus.
I rewrote the function to call utf8UnconsByteString directly and avoid
the roundtrip through Faststring so now the head/tail is computed by the
same call.
Fixes #19976
Diffstat (limited to 'compiler/GHC')
-rw-r--r-- | compiler/GHC/Core/SimpleOpt.hs | 38 |
1 files changed, 17 insertions, 21 deletions
diff --git a/compiler/GHC/Core/SimpleOpt.hs b/compiler/GHC/Core/SimpleOpt.hs index ffec34ff17..67dc4609c3 100644 --- a/compiler/GHC/Core/SimpleOpt.hs +++ b/compiler/GHC/Core/SimpleOpt.hs @@ -48,12 +48,12 @@ import GHC.Builtin.Types import GHC.Builtin.Names import GHC.Types.Basic import GHC.Unit.Module ( Module ) +import GHC.Utils.Encoding import GHC.Utils.Outputable import GHC.Utils.Panic import GHC.Utils.Panic.Plain import GHC.Utils.Misc import GHC.Data.Maybe ( orElse ) -import GHC.Data.FastString import Data.List (mapAccumL) import qualified Data.ByteString as BS @@ -874,9 +874,8 @@ calls to unpackCString# and returns: Just (':', [Char], ['a', unpackCString# "bc"]). -We need to be careful about UTF8 strings here. ""# contains a ByteString, so -we must parse it back into a FastString to split off the first character. -That way we can treat unpackCString# and unpackCStringUtf8# in the same way. +We need to be careful about UTF8 strings here. ""# contains an encoded ByteString, so +we call utf8UnconsByteString to correctly deal with the encoding and splitting. We must also be careful about lvl = "foo"# @@ -885,6 +884,8 @@ to ensure that we see through the let-binding for 'lvl'. Hence the (exprIsLiteral_maybe .. arg) in the guard before the call to dealWithStringLiteral. +The tests for this function are in T9400. + Note [Push coercions in exprIsConApp_maybe] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In #13025 I found a case where we had @@ -1237,23 +1238,18 @@ dealWithStringLiteral :: Var -> BS.ByteString -> Coercion -- This is not possible with user-supplied empty literals, GHC.Core.Make.mkStringExprFS -- turns those into [] automatically, but just in case something else in GHC -- generates a string literal directly. -dealWithStringLiteral _ str co - | BS.null str - = pushCoDataCon nilDataCon [Type charTy] co - -dealWithStringLiteral fun str co - = let strFS = mkFastStringByteString str - - char = mkConApp charDataCon [mkCharLit (headFS strFS)] - charTail = BS.tail (bytesFS strFS) - - -- In singleton strings, just add [] instead of unpackCstring# ""#. - rest = if BS.null charTail - then mkConApp nilDataCon [Type charTy] - else App (Var fun) - (Lit (LitString charTail)) - - in pushCoDataCon consDataCon [Type charTy, char, rest] co +dealWithStringLiteral fun str co = + case utf8UnconsByteString str of + Nothing -> pushCoDataCon nilDataCon [Type charTy] co + Just (char, charTail) -> + let char_expr = mkConApp charDataCon [mkCharLit char] + -- In singleton strings, just add [] instead of unpackCstring# ""#. + rest = if BS.null charTail + then mkConApp nilDataCon [Type charTy] + else App (Var fun) + (Lit (LitString charTail)) + + in pushCoDataCon consDataCon [Type charTy, char_expr, rest] co {- Note [Unfolding DFuns] |