summaryrefslogtreecommitdiff
path: root/Lib/tokenize.py
Commit message (Expand)AuthorAgeFilesLines
* fix for files with coding cookies and BOMsBenjamin Peterson2010-03-181-3/+5
* in tokenize.detect_encoding(), return utf-8-sig when a BOM is foundBenjamin Peterson2010-03-181-6/+12
* use some more itertools magic to make '' be yielded after readline is doneBenjamin Peterson2009-11-141-3/+4
* simply by using itertools.chain()Benjamin Peterson2009-11-141-10/+5
* Merged revisions 75149,75260-75263,75265-75267,75292,75300,75376,75405,75429-...Benjamin Peterson2009-11-131-13/+20
* normalize latin-1 and utf-8 variant encodings like the builtin tokenizer doesBenjamin Peterson2009-10-091-1/+12
* Remove dependency on the collections module.Raymond Hettinger2009-04-291-3/+41
* Issue #5857: tokenize.tokenize() now returns named tuples.Raymond Hettinger2009-04-291-19/+22
* reuse tokenize.detect_encoding in linecache instead of a custom solutionBenjamin Peterson2009-03-241-3/+4
* raise a SyntaxError in detect_encoding() when a codec lookup fails like the b...Benjamin Peterson2008-12-121-13/+20
* #2834: Change re module semantics, so that str and bytes mixing is forbidden,Antoine Pitrou2008-08-191-6/+8
* use the more idomatic (and Py3k faster) while TrueBenjamin Peterson2008-06-051-1/+1
* Merged revisions 61964-61979 via svnmerge fromChristian Heimes2008-03-281-0/+9
* - Issue #719888: Updated tokenize to use a bytes API. generate_tokens has beenTrent Nelson2008-03-181-57/+130
* Merged revisions 60080-60089,60091-60093 via svnmerge fromGeorg Brandl2008-01-191-3/+3
* Patch 1420 by Ron Adam.Guido van Rossum2007-11-121-18/+18
* Raise statement normalization in Lib/.Collin Winter2007-08-301-2/+2
* Merged revisions 55817-55961 via svnmerge fromGuido van Rossum2007-06-131-4/+5
* Merged revisions 55328-55341 via svnmerge fromGuido van Rossum2007-05-151-1/+2
* PEP 3114: rename .next() to .__next__() and add next() builtin.Georg Brandl2007-04-211-2/+2
* Hide list comp variables and support set comprehensionsNick Coghlan2007-04-151-1/+0
* Make ELLIPSIS a separate token. This makes it a syntax error to write ". . ...Georg Brandl2007-03-181-3/+3
* Fix most trivially-findable print statements.Guido van Rossum2007-02-091-2/+2
* PEP 3107 - Function Annotations thanks to Tony LowndsNeal Norwitz2006-12-281-1/+1
* Four months of trunk changes (including a few releases...)Thomas Wouters2006-12-131-32/+77
* Remove support for backticks from the grammar and compiler.Brett Cannon2006-08-251-1/+1
* Killed the <> operator. You must now use !=.Guido van Rossum2006-08-241-1/+1
* Merge current trunk into p3yk. This includes the PyNumber_Index API change,Thomas Wouters2006-08-211-1/+2
* Merge p3yk branch with the trunk up to revision 45595. This breaks a fairThomas Wouters2006-04-211-1/+1
* SF bug #1224621: tokenize module does not detect inconsistent dedentsRaymond Hettinger2005-06-211-0/+3
* Add untokenize() function to allow full round-trip tokenization.Raymond Hettinger2005-06-101-3/+49
* PEP-0318, @decorator-style. In Guido's words:Anthony Baxter2004-08-021-1/+1
* Get rid of many apply() calls.Guido van Rossum2003-02-271-3/+3
* SF 633560: tokenize.__all__ needs "generate_tokens"Raymond Hettinger2002-11-051-1/+2
* Speed up the most egregious "if token in (long tuple)" cases by usingGuido van Rossum2002-08-241-10/+19
* Whitespace normalization.Tim Peters2002-05-231-5/+5
* Added docstrings excerpted from Python Library Reference.Raymond Hettinger2002-05-151-0/+25
* Remove some now-obsolete generator future statements.Tim Peters2002-04-011-2/+0
* Cleanup x so it is not left in moduleNeal Norwitz2002-03-261-0/+1
* SF patch #455966: Allow leading 0 in float/imag literals.Tim Peters2001-08-301-2/+2
* Add new tokens // and //=, in support of PEP 238.Guido van Rossum2001-08-081-0/+1
* Use string.ascii_letters instead of string.letters (SF bug #226706).Fred Drake2001-07-201-1/+1
* Preliminary support for "from __future__ import generators" to enableGuido van Rossum2001-07-151-0/+2
* Turns out Neil didn't intend for *all* of his gen-branch work to getTim Peters2001-06-291-8/+21
* Merging the gen-branch into the main line, at Guido's direction. Yay!Tim Peters2001-06-181-15/+20
* Provide a StopTokenizing exception for conveniently exiting the loop.Ka-Ping Yee2001-03-231-10/+11
* Better __credits__.Ka-Ping Yee2001-03-011-1/+2
* Add __author__ and __credits__ variables.Ka-Ping Yee2001-03-011-1/+2
* final round of __all__ lists (I hope) - skipped urllib2 because Moshe may beSkip Montanaro2001-03-011-1/+5
* String method conversion.Eric S. Raymond2001-02-091-1/+1