Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | RawToken{Formatter,Lexer}: support Python 3 and handle exceptions (#1602) | Anders Kaseorg | 2021-02-14 | 1 | -0/+68 |
In Python 3, RawTokenFormatter would output non-ASCII for non-ASCII input, and RawTokenLexer would throw Unicode-related exceptions for ASCII or non-ASCII input; fix them. Also, handle all exceptions, so that callers who find RawTokenLexer via get_lexer_by_name on user input don’t unexpectedly get a lexer that throws exceptions. Signed-off-by: Anders Kaseorg <andersk@mit.edu> |