summaryrefslogtreecommitdiff
path: root/Lib/tokenize.py
Commit message (Expand)AuthorAgeFilesLines
* Merge with 3.3Terry Jan Reedy2014-02-231-1/+1
|\
| * Merge with 3.3Terry Jan Reedy2014-02-231-0/+6
| |\
| * \ Merge with 3.3Terry Jan Reedy2014-02-171-1/+1
| |\ \
| | * \ Untokenize, bad assert: Merge with 3.3Terry Jan Reedy2014-02-171-1/+3
| | |\ \
| | * \ \ Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-0/+3
| | |\ \ \
| | * \ \ \ #19620: merge with 3.3.Ezio Melotti2013-11-251-1/+1
| | |\ \ \ \
| | * \ \ \ \ Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py scriptSerhiy Storchaka2013-09-161-4/+4
| | |\ \ \ \ \
| | * | | | | | Replace IOError with OSError (#16715)Andrew Svetlov2012-12-251-1/+1
* | | | | | | | whitespaceTerry Jan Reedy2014-02-231-1/+1
| |_|_|_|_|_|/ |/| | | | | |
* | | | | | | Issue #9974: When untokenizing, use row info to insert backslash+newline.Terry Jan Reedy2014-02-231-0/+6
|/ / / / / /
* | | | | | Issue #8478: Untokenizer.compat now processes first token from iterator input.Terry Jan Reedy2014-02-171-13/+11
| |_|_|_|/ |/| | | |
* | | | | Untokenize: An logically incorrect assert tested user input validity.Terry Jan Reedy2014-02-171-1/+3
| |_|_|/ |/| | |
* | | | Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-0/+3
| |_|/ |/| |
* | | #19620: Fix typo in docstring (noticed by Christopher Welborn).Ezio Melotti2013-11-251-1/+1
| |/ |/|
* | Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py scriptSerhiy Storchaka2013-09-161-4/+4
|/
* #16152: merge with 3.2.Ezio Melotti2012-11-031-1/+3
|\
| * #16152: fix tokenize to ignore whitespace at the end of the code when no newl...Ezio Melotti2012-11-031-1/+3
* | Merge branchFlorent Xicluna2012-07-071-52/+173
|\ \ | |/ |/|
| * Issue #15096: Drop support for the ur string prefixChristian Heimes2012-06-201-9/+3
| * Issue #15054: Fix incorrect tokenization of 'b' string literals.Meador Inge2012-06-161-1/+1
| * Issue #14629: Mention the filename in SyntaxError exceptions fromBrett Cannon2012-04-201-3/+19
| * merge 3.2: issue 14629Martin v. L?wis2012-04-201-49/+160
| |\
| | * Updated tokenize to support the inverse byte literals new in 3.3Armin Ronacher2012-03-041-6/+16
| | * Basic support for PEP 414 without docs or tests.Armin Ronacher2012-03-041-8/+22
| | * Issue #2134: Add support for tokenize.TokenInfo.exact_type.Meador Inge2012-01-191-1/+58
| | * Issue #13150: The tokenize module doesn't compile large regular expressions a...Antoine Pitrou2011-10-111-19/+16
| | * Issue #12943: python -m tokenize support has been added to tokenize.Meador Inge2011-10-071-23/+56
* | | Issue #14990: tokenize: correctly fail with SyntaxError on invalid encoding d...Florent Xicluna2012-07-071-1/+1
|/ /
* | Issue #14629: Raise SyntaxError in tokenizer.detect_encodingMartin v. L?wis2012-04-201-2/+5
|/
* Merged revisions 88498 via svnmerge fromBrett Cannon2011-02-221-3/+2
* Issue #10386: Added __all__ to token module; this simplifies importingAlexander Belopolsky2010-11-111-3/+2
* Issue #10335: Add tokenize.open(), detect the file encoding usingVictor Stinner2010-11-091-0/+15
* A little bit more readable repr method.Raymond Hettinger2010-09-091-3/+3
* Experiment: Let collections.namedtuple() do the work. This should work now t...Raymond Hettinger2010-09-091-39/+3
* Improve the repr for the TokenInfo named tuple.Raymond Hettinger2010-09-091-1/+28
* Remove unused import, fix typo and rewrap docstrings.Florent Xicluna2010-09-031-17/+18
* handle names starting with non-ascii characters correctly #9712Benjamin Peterson2010-08-301-5/+10
* fix for files with coding cookies and BOMsBenjamin Peterson2010-03-181-3/+5
* in tokenize.detect_encoding(), return utf-8-sig when a BOM is foundBenjamin Peterson2010-03-181-6/+12
* use some more itertools magic to make '' be yielded after readline is doneBenjamin Peterson2009-11-141-3/+4
* simply by using itertools.chain()Benjamin Peterson2009-11-141-10/+5
* Merged revisions 75149,75260-75263,75265-75267,75292,75300,75376,75405,75429-...Benjamin Peterson2009-11-131-13/+20
* normalize latin-1 and utf-8 variant encodings like the builtin tokenizer doesBenjamin Peterson2009-10-091-1/+12
* Remove dependency on the collections module.Raymond Hettinger2009-04-291-3/+41
* Issue #5857: tokenize.tokenize() now returns named tuples.Raymond Hettinger2009-04-291-19/+22
* reuse tokenize.detect_encoding in linecache instead of a custom solutionBenjamin Peterson2009-03-241-3/+4
* raise a SyntaxError in detect_encoding() when a codec lookup fails like the b...Benjamin Peterson2008-12-121-13/+20
* #2834: Change re module semantics, so that str and bytes mixing is forbidden,Antoine Pitrou2008-08-191-6/+8
* use the more idomatic (and Py3k faster) while TrueBenjamin Peterson2008-06-051-1/+1
* Merged revisions 61964-61979 via svnmerge fromChristian Heimes2008-03-281-0/+9