diff options
author | Paul McGuire <ptmcg@austin.rr.com> | 2019-07-15 23:22:27 -0500 |
---|---|---|
committer | Paul McGuire <ptmcg@austin.rr.com> | 2019-07-15 23:22:27 -0500 |
commit | 6ea260a4211da88f3d28fd6266e6765a6c209baf (patch) | |
tree | c05ddf9352b9ec76c1927e22f7827f493fdbecf6 | |
parent | 5a566b59170fb3fe705a7691806c4afd158df520 (diff) | |
download | pyparsing-git-6ea260a4211da88f3d28fd6266e6765a6c209baf.tar.gz |
Add CONTRIBUTING.md guidelines; code and whitespace cleanup
-rw-r--r-- | CHANGES | 13 | ||||
-rw-r--r-- | CONTRIBUTING.md | 112 | ||||
-rw-r--r-- | pyparsing.py | 1914 |
3 files changed, 1091 insertions, 948 deletions
@@ -194,6 +194,19 @@ Version 2.4.1 - July, 2019 fix some omitted formats and upgrade to latest pyparsing idioms, beginning with writing an actual BNF. +- With the help and encouragement from several contributors, including + Matěj Cepl and Cengiz Kaygusuz, I've started cleaning up the internal + coding styles in core pyparsing, bringing it up to modern coding + practices from pyparsing's early development days dating back to + 2003. Whitespace has been largely standardized along PEP8 guidelines, + removing extra spaces around parentheses, and adding them around + arithmetic operators and after colons and commas. I was going to hold + off on doing this work until after 2.4.1, but after cleaning up a + few trial classes, the difference was so significant that I continued + on to the rest of the core code base. This should facilitate future + work and submitted PRs, allowing them to focus on substantive code + changes, and not get sidetracked by whitespace issues. + Version 2.4.0 - April, 2019 --------------------------- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..70715d6 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,112 @@ +# CONTRIBUTING + +Thank you for your interest in working on pyparsing! Pyparsing has become a popular module for creating simple +text parsing and data scraping applications. It has been incorporated in several widely-used packages, and is +often used by beginners as part of their first Python project. + +## Raising questions / asking for help + +If you have a question on using pyparsing, there are a number of resources available online. + +- [StackOverflow](https://stackoverflow.com/questions/tagged/pyparsing) - about 10 years of SO questions and answers + can be searched on StackOverflow, tagged with the `pyparsing` tag. Note that some of the older posts will refer + to features in Python 2, or to versions and coding practices for pyparsing that have been replaced by newer classes + and coding idioms. + +- [pyparsing sub-reddit](https://www.reddit.com/r/pyparsing/) - still very lightly attended, but open to anyone + wishing to post questions or links related to pyparsing. An alternative channel to StackOverflow for asking + questions. + +- [online docs](https://pyparsing-docs.readthedocs.io/en/latest/index.html) and a separately maintained set of class + library docs [here](https://pyparsing-doc.neocities.org/) - These docs are auto-generated from the docstrings + embedded in the pyparsing classes, so they can also be viewed in the interactive Python console's and Jupyter + Notebook's `help` commands. + +- [the pyparsing Wikispaces archive](https://github.com/pyparsing/wikispaces_archive) - Before hosting on GitHub, + pyparsing had a separate wiki on the wikispaces.com website. In 2018 this page was discontinued. The discussion + content archive has been reformatted into Markdown and can be viewed by year at the GitHub repository. Just as + with some of the older questions on StackOverflow, some of these older posts may reflect out-of-date pyparsing + and Python features. + +- [submit an issue](https://github.com/pyparsing/pyparsing/issues) - If you have a problem with pyparsing that looks + like an actual bug, please submit an issue on GitHub. Some pyparsing behavior may be counter-intuitive, so try to + review some of the other resources first, or some of the other open and closed issues. Or post your question on SO + or reddit. But don't wait until you are desperate and frustrated - just ask! :) + + +## Submitting changes + +If you are considering proposing updates to pyparsing, please bear in mind the following guidelines. + +Please review [_The Zen of Pyparsing_ and _The Zen of Pyparsing +Development_](https://github.com/pyparsing/pyparsing/wiki/Zen) +article on the pyparsing wiki, to get a general feel for the historical and future approaches to pyparsing's +design, and intended developer experience as an embedded DSL. + +## Some design points + +- Minimize additions to the module namespace. Over time, pyparsing's namespace has acquired a *lot* of names. + New features have been encapsulated into namespace classes to try to hold back the name flooding when importing + pyparsing. + +- New operator overloads will need to show broad applicability. + +- Performance tuning should focus on parse time performance. Optimizing parser definition performance is secondary. + +- New external dependencies will require substantial justification, and if included, will need to be guarded for + `ImportError`s raised if the external module is not installed. + +## Some coding points + +These coding styles are encouraged whether submitting code for core pyparsing or for submitting an example. + +- PEP8 - at this time, pyparsing is very non-compliant with many PEP8 guidelines, especially those regarding + name casing. I had just finished several years of Java and Smalltalk development, and camel case seemed to be the + future trend in coding styles. There are plans to convert these names to PEP8-conformant snake case, but this will + be done over several releases to provide a migration path for current pyparsing-dependent applications. See more + information at the [PEP8 wiki page](https://github.com/pyparsing/pyparsing/wiki/PEP-8-planning). + + If you wish to submit a new example, please follow PEP8 name and coding guidelines. Example code must be available + for distribution with the rest of pyparsing under the MIT open source license. + +- No backslashes for line continuations. + Continuation lines for expressions in ()'s should start with the continuing operator: + + really_long_line = (something + + some_other_long_thing + + even_another_long_thing) + +- Changes to core pyparsing must be compatible back to Py3.5 without conditionalizing. Later Py3 features may be + used in examples by way of illustration. + +- str.format() statements should use named format arguments (unless this proves to be a slowdown at parse time). + +- List, tuple, and dict literals should include a trailing comma after the last element, which reduces changeset + clutter when another element gets added to the end. + +- Examples should import pyparsing and the common namespace classes as: + + import pyparsing as pp + # if necessary + ppc = pp.pyparsing_common + ppu = pp.pyparsing_unicode + +- Where possible use operators to create composite parse expressions: + + expr = expr_a + expr_b | expr_c + + instead of: + + expr = pp.MatchFirst([pp.And([expr_a, expr_b]), expr_c]) + + Exception: if using a generator to create an expression: + + import keyword + python_keywords = keyword.kwlist + any_keyword = pp.MatchFirst(pp.Keyword(kw) + for kw in python_keywords)) + +- Learn [The Classic Blunders]([http://](https://github.com/pyparsing/pyparsing/wiki/TheClassicBlunders)) and + how to avoid them when developing new examples. + +- New features should be accompanied with updates to unitTests.py and a bullet in the CHANGES file. diff --git a/pyparsing.py b/pyparsing.py index 5ca86b0..0bdd8db 100644 --- a/pyparsing.py +++ b/pyparsing.py @@ -1,4 +1,4 @@ -#-*- coding: utf-8 -*- +# -*- coding: utf-8 -*- # module pyparsing.py # # Copyright (c) 2003-2019 Paul T. McGuire @@ -96,7 +96,7 @@ classes inherit from. Use the docstrings for examples of how to: """ __version__ = "2.4.1" -__versionTime__ = "13 Jul 2019 17:05 UTC" +__versionTime__ = "16 Jul 2019 04:10 UTC" __author__ = "Paul McGuire <ptmcg@users.sourceforge.net>" import string @@ -151,19 +151,18 @@ except ImportError: # version compatibility configuration __compat__ = SimpleNamespace() __compat__.__doc__ = """ - A cross-version compatibility configuration for pyparsing features that will be - released in a future version. By setting values in this configuration to True, - those features can be enabled in prior versions for compatibility development + A cross-version compatibility configuration for pyparsing features that will be + released in a future version. By setting values in this configuration to True, + those features can be enabled in prior versions for compatibility development and testing. - + - collect_all_And_tokens - flag to enable fix for Issue #63 that fixes erroneous grouping - of results names when an And expression is nested within an Or or MatchFirst; set to + of results names when an And expression is nested within an Or or MatchFirst; set to True to enable bugfix released in pyparsing 2.3.0, or False to preserve pre-2.3.0 handling of named results """ __compat__.collect_all_And_tokens = True - __diag__ = SimpleNamespace() __diag__.__doc__ = """ Diagnostic configuration @@ -171,13 +170,13 @@ Diagnostic configuration name is defined on a MatchFirst or Or expression with one or more And subexpressions (default=True) (only warns if __compat__.collect_all_And_tokens is False) - warn_ungrouped_named_tokens_in_collection - flag to enable warnings when a results - name is defined on a containing expression with ungrouped subexpressions that also + name is defined on a containing expression with ungrouped subexpressions that also have results names (default=True) - warn_name_set_on_empty_Forward - flag to enable warnings whan a Forward is defined with a results name, but has no contents defined (default=False) - warn_on_multiple_string_args_to_oneof - flag to enable warnings whan oneOf is incorrectly called with multiple str arguments (default=True) - - enable_debug_on_named_expressions - flag to auto-enable debug on all subsequent + - enable_debug_on_named_expressions - flag to auto-enable debug on all subsequent calls to ParserElement.setName() (default=False) """ __diag__.warn_multiple_tokens_in_named_alternation = True @@ -186,29 +185,29 @@ __diag__.warn_name_set_on_empty_Forward = False __diag__.warn_on_multiple_string_args_to_oneof = True __diag__.enable_debug_on_named_expressions = False -#~ sys.stderr.write( "testing pyparsing module, version %s, %s\n" % (__version__,__versionTime__ ) ) - -__all__ = [ '__version__', '__versionTime__', '__author__', '__compat__', '__diag__', -'And', 'CaselessKeyword', 'CaselessLiteral', 'CharsNotIn', 'Combine', 'Dict', 'Each', 'Empty', -'FollowedBy', 'Forward', 'GoToColumn', 'Group', 'Keyword', 'LineEnd', 'LineStart', 'Literal', -'PrecededBy', 'MatchFirst', 'NoMatch', 'NotAny', 'OneOrMore', 'OnlyOnce', 'Optional', 'Or', -'ParseBaseException', 'ParseElementEnhance', 'ParseException', 'ParseExpression', 'ParseFatalException', -'ParseResults', 'ParseSyntaxException', 'ParserElement', 'QuotedString', 'RecursiveGrammarException', -'Regex', 'SkipTo', 'StringEnd', 'StringStart', 'Suppress', 'Token', 'TokenConverter', -'White', 'Word', 'WordEnd', 'WordStart', 'ZeroOrMore', 'Char', -'alphanums', 'alphas', 'alphas8bit', 'anyCloseTag', 'anyOpenTag', 'cStyleComment', 'col', -'commaSeparatedList', 'commonHTMLEntity', 'countedArray', 'cppStyleComment', 'dblQuotedString', -'dblSlashComment', 'delimitedList', 'dictOf', 'downcaseTokens', 'empty', 'hexnums', -'htmlComment', 'javaStyleComment', 'line', 'lineEnd', 'lineStart', 'lineno', -'makeHTMLTags', 'makeXMLTags', 'matchOnlyAtCol', 'matchPreviousExpr', 'matchPreviousLiteral', -'nestedExpr', 'nullDebugAction', 'nums', 'oneOf', 'opAssoc', 'operatorPrecedence', 'printables', -'punc8bit', 'pythonStyleComment', 'quotedString', 'removeQuotes', 'replaceHTMLEntity', -'replaceWith', 'restOfLine', 'sglQuotedString', 'srange', 'stringEnd', -'stringStart', 'traceParseAction', 'unicodeString', 'upcaseTokens', 'withAttribute', -'indentedBlock', 'originalTextFor', 'ungroup', 'infixNotation','locatedExpr', 'withClass', -'CloseMatch', 'tokenMap', 'pyparsing_common', 'pyparsing_unicode', 'unicode_set', -'conditionAsParseAction', -] +# ~ sys.stderr.write("testing pyparsing module, version %s, %s\n" % (__version__, __versionTime__)) + +__all__ = ['__version__', '__versionTime__', '__author__', '__compat__', '__diag__', + 'And', 'CaselessKeyword', 'CaselessLiteral', 'CharsNotIn', 'Combine', 'Dict', 'Each', 'Empty', + 'FollowedBy', 'Forward', 'GoToColumn', 'Group', 'Keyword', 'LineEnd', 'LineStart', 'Literal', + 'PrecededBy', 'MatchFirst', 'NoMatch', 'NotAny', 'OneOrMore', 'OnlyOnce', 'Optional', 'Or', + 'ParseBaseException', 'ParseElementEnhance', 'ParseException', 'ParseExpression', 'ParseFatalException', + 'ParseResults', 'ParseSyntaxException', 'ParserElement', 'QuotedString', 'RecursiveGrammarException', + 'Regex', 'SkipTo', 'StringEnd', 'StringStart', 'Suppress', 'Token', 'TokenConverter', + 'White', 'Word', 'WordEnd', 'WordStart', 'ZeroOrMore', 'Char', + 'alphanums', 'alphas', 'alphas8bit', 'anyCloseTag', 'anyOpenTag', 'cStyleComment', 'col', + 'commaSeparatedList', 'commonHTMLEntity', 'countedArray', 'cppStyleComment', 'dblQuotedString', + 'dblSlashComment', 'delimitedList', 'dictOf', 'downcaseTokens', 'empty', 'hexnums', + 'htmlComment', 'javaStyleComment', 'line', 'lineEnd', 'lineStart', 'lineno', + 'makeHTMLTags', 'makeXMLTags', 'matchOnlyAtCol', 'matchPreviousExpr', 'matchPreviousLiteral', + 'nestedExpr', 'nullDebugAction', 'nums', 'oneOf', 'opAssoc', 'operatorPrecedence', 'printables', + 'punc8bit', 'pythonStyleComment', 'quotedString', 'removeQuotes', 'replaceHTMLEntity', + 'replaceWith', 'restOfLine', 'sglQuotedString', 'srange', 'stringEnd', + 'stringStart', 'traceParseAction', 'unicodeString', 'upcaseTokens', 'withAttribute', + 'indentedBlock', 'originalTextFor', 'ungroup', 'infixNotation', 'locatedExpr', 'withClass', + 'CloseMatch', 'tokenMap', 'pyparsing_common', 'pyparsing_unicode', 'unicode_set', + 'conditionAsParseAction', + ] system_version = tuple(sys.version_info)[:3] PY_3 = system_version[0] == 3 @@ -233,7 +232,7 @@ else: < returns the unicode object | encodes it with the default encoding | ... >. """ - if isinstance(obj,unicode): + if isinstance(obj, unicode): return obj try: @@ -251,9 +250,10 @@ else: # build list of single arg builtins, tolerant of Python version, that can be used as parse actions singleArgBuiltins = [] import __builtin__ + for fname in "sum len sorted reversed list tuple set any all min max".split(): try: - singleArgBuiltins.append(getattr(__builtin__,fname)) + singleArgBuiltins.append(getattr(__builtin__, fname)) except AttributeError: continue @@ -264,16 +264,16 @@ def _xml_escape(data): # ampersand must be replaced first from_symbols = '&><"\'' - to_symbols = ('&'+s+';' for s in "amp gt lt quot apos".split()) - for from_,to_ in zip(from_symbols, to_symbols): + to_symbols = ('&' + s + ';' for s in "amp gt lt quot apos".split()) + for from_, to_ in zip(from_symbols, to_symbols): data = data.replace(from_, to_) return data -alphas = string.ascii_uppercase + string.ascii_lowercase -nums = "0123456789" -hexnums = nums + "ABCDEFabcdef" -alphanums = alphas + nums -_bslash = chr(92) +alphas = string.ascii_uppercase + string.ascii_lowercase +nums = "0123456789" +hexnums = nums + "ABCDEFabcdef" +alphanums = alphas + nums +_bslash = chr(92) printables = "".join(c for c in string.printable if c not in string.whitespace) @@ -293,7 +293,7 @@ class ParseBaseException(Exception): """base exception class for all parsing runtime exceptions""" # Performance tuning: we construct a *lot* of these, so keep this # constructor as small and fast as possible - def __init__( self, pstr, loc=0, msg=None, elem=None ): + def __init__(self, pstr, loc=0, msg=None, elem=None): self.loc = loc if msg is None: self.msg = pstr @@ -312,34 +312,34 @@ class ParseBaseException(Exception): """ return cls(pe.pstr, pe.loc, pe.msg, pe.parserElement) - def __getattr__( self, aname ): + def __getattr__(self, aname): """supported attributes by name are: - lineno - returns the line number of the exception text - col - returns the column number of the exception text - line - returns the line containing the exception text """ - if( aname == "lineno" ): - return lineno( self.loc, self.pstr ) - elif( aname in ("col", "column") ): - return col( self.loc, self.pstr ) - elif( aname == "line" ): - return line( self.loc, self.pstr ) + if aname == "lineno": + return lineno(self.loc, self.pstr) + elif aname in ("col", "column"): + return col(self.loc, self.pstr) + elif aname == "line": + return line(self.loc, self.pstr) else: raise AttributeError(aname) - def __str__( self ): + def __str__(self): if self.pstr: if self.loc >= len(self.pstr): foundstr = ', found end of text' else: - foundstr = (', found %r' % self.pstr[self.loc:self.loc+1]).replace(r'\\', '\\') + foundstr = (', found %r' % self.pstr[self.loc:self.loc + 1]).replace(r'\\', '\\') else: foundstr = '' - return "%s%s (at char %d), (line:%d, col:%d)" % \ - ( self.msg, foundstr, self.loc, self.lineno, self.column ) - def __repr__( self ): + return ("%s%s (at char %d), (line:%d, col:%d)" % + (self.msg, foundstr, self.loc, self.lineno, self.column)) + def __repr__(self): return _ustr(self) - def markInputline( self, markerString = ">!<" ): + def markInputline(self, markerString=">!<"): """Extracts the exception line from the input string, and marks the location of the exception with a special symbol. """ @@ -475,21 +475,21 @@ class RecursiveGrammarException(Exception): """exception thrown by :class:`ParserElement.validate` if the grammar could be improperly recursive """ - def __init__( self, parseElementList ): + def __init__(self, parseElementList): self.parseElementTrace = parseElementList - def __str__( self ): + def __str__(self): return "RecursiveGrammarException: %s" % self.parseElementTrace class _ParseResultsWithOffset(object): - def __init__(self,p1,p2): - self.tup = (p1,p2) - def __getitem__(self,i): + def __init__(self, p1, p2): + self.tup = (p1, p2) + def __getitem__(self, i): return self.tup[i] def __repr__(self): return repr(self.tup[0]) - def setOffset(self,i): - self.tup = (self.tup[0],i) + def setOffset(self, i): + self.tup = (self.tup[0], i) class ParseResults(object): """Structured parse results, to provide multiple means of access to @@ -534,7 +534,7 @@ class ParseResults(object): - month: 12 - year: 1999 """ - def __new__(cls, toklist=None, name=None, asList=True, modal=True ): + def __new__(cls, toklist=None, name=None, asList=True, modal=True): if isinstance(toklist, cls): return toklist retobj = object.__new__(cls) @@ -543,7 +543,7 @@ class ParseResults(object): # Performance tuning: we construct a *lot* of these, so keep this # constructor as small and fast as possible - def __init__( self, toklist=None, name=None, asList=True, modal=True, isinstance=isinstance ): + def __init__(self, toklist=None, name=None, asList=True, modal=True, isinstance=isinstance): if self.__doinit: self.__doinit = False self.__name = None @@ -564,85 +564,93 @@ class ParseResults(object): if name is not None and name: if not modal: self.__accumNames[name] = 0 - if isinstance(name,int): - name = _ustr(name) # will always return a str, but use _ustr for consistency + if isinstance(name, int): + name = _ustr(name) # will always return a str, but use _ustr for consistency self.__name = name - if not (isinstance(toklist, (type(None), basestring, list)) and toklist in (None,'',[])): - if isinstance(toklist,basestring): - toklist = [ toklist ] + if not (isinstance(toklist, (type(None), basestring, list)) and toklist in (None, '', [])): + if isinstance(toklist, basestring): + toklist = [toklist] if asList: - if isinstance(toklist,ParseResults): + if isinstance(toklist, ParseResults): self[name] = _ParseResultsWithOffset(ParseResults(toklist.__toklist), 0) else: - self[name] = _ParseResultsWithOffset(ParseResults(toklist[0]),0) + self[name] = _ParseResultsWithOffset(ParseResults(toklist[0]), 0) self[name].__name = name else: try: self[name] = toklist[0] - except (KeyError,TypeError,IndexError): + except (KeyError, TypeError, IndexError): self[name] = toklist - def __getitem__( self, i ): - if isinstance( i, (int,slice) ): + def __getitem__(self, i): + if isinstance(i, (int, slice)): return self.__toklist[i] else: if i not in self.__accumNames: return self.__tokdict[i][-1][0] else: - return ParseResults([ v[0] for v in self.__tokdict[i] ]) + return ParseResults([v[0] for v in self.__tokdict[i]]) - def __setitem__( self, k, v, isinstance=isinstance ): - if isinstance(v,_ParseResultsWithOffset): - self.__tokdict[k] = self.__tokdict.get(k,list()) + [v] + def __setitem__(self, k, v, isinstance=isinstance): + if isinstance(v, _ParseResultsWithOffset): + self.__tokdict[k] = self.__tokdict.get(k, list()) + [v] sub = v[0] - elif isinstance(k,(int,slice)): + elif isinstance(k, (int, slice)): self.__toklist[k] = v sub = v else: - self.__tokdict[k] = self.__tokdict.get(k,list()) + [_ParseResultsWithOffset(v,0)] + self.__tokdict[k] = self.__tokdict.get(k, list()) + [_ParseResultsWithOffset(v, 0)] sub = v - if isinstance(sub,ParseResults): + if isinstance(sub, ParseResults): sub.__parent = wkref(self) - def __delitem__( self, i ): - if isinstance(i,(int,slice)): - mylen = len( self.__toklist ) + def __delitem__(self, i): + if isinstance(i, (int, slice)): + mylen = len(self.__toklist) del self.__toklist[i] # convert int to slice if isinstance(i, int): if i < 0: i += mylen - i = slice(i, i+1) + i = slice(i, i + 1) # get removed indices removed = list(range(*i.indices(mylen))) removed.reverse() # fixup indices in token dictionary - for name,occurrences in self.__tokdict.items(): + for name, occurrences in self.__tokdict.items(): for j in removed: for k, (value, position) in enumerate(occurrences): occurrences[k] = _ParseResultsWithOffset(value, position - (position > j)) else: del self.__tokdict[i] - def __contains__( self, k ): + def __contains__(self, k): return k in self.__tokdict - def __len__( self ): return len( self.__toklist ) - def __bool__(self): return ( not not self.__toklist ) + def __len__(self): + return len(self.__toklist) + + def __bool__(self): + return (not not self.__toklist) __nonzero__ = __bool__ - def __iter__( self ): return iter( self.__toklist ) - def __reversed__( self ): return iter( self.__toklist[::-1] ) - def _iterkeys( self ): + + def __iter__(self): + return iter(self.__toklist) + + def __reversed__(self): + return iter(self.__toklist[::-1]) + + def _iterkeys(self): if hasattr(self.__tokdict, "iterkeys"): return self.__tokdict.iterkeys() else: return iter(self.__tokdict) - def _itervalues( self ): + def _itervalues(self): return (self[k] for k in self._iterkeys()) - def _iteritems( self ): + def _iteritems(self): return ((k, self[k]) for k in self._iterkeys()) if PY_3: @@ -665,24 +673,24 @@ class ParseResults(object): iteritems = _iteritems """Returns an iterator of all named result key-value tuples (Python 2.x only).""" - def keys( self ): + def keys(self): """Returns all named result keys (as a list in Python 2.x, as an iterator in Python 3.x).""" return list(self.iterkeys()) - def values( self ): + def values(self): """Returns all named result values (as a list in Python 2.x, as an iterator in Python 3.x).""" return list(self.itervalues()) - def items( self ): + def items(self): """Returns all named result key-values (as a list of tuples in Python 2.x, as an iterator in Python 3.x).""" return list(self.iteritems()) - def haskeys( self ): + def haskeys(self): """Since keys() returns an iterator, this method is helpful in bypassing code that looks for the existence of any defined results names.""" return bool(self.__tokdict) - def pop( self, *args, **kwargs): + def pop(self, *args, **kwargs): """ Removes and returns item at specified index (default= ``last``). Supports both ``list`` and ``dict`` semantics for ``pop()``. If @@ -721,14 +729,14 @@ class ParseResults(object): """ if not args: args = [-1] - for k,v in kwargs.items(): + for k, v in kwargs.items(): if k == 'default': args = (args[0], v) else: raise TypeError("pop() got an unexpected keyword argument '%s'" % k) - if (isinstance(args[0], int) or - len(args) == 1 or - args[0] in self): + if (isinstance(args[0], int) + or len(args) == 1 + or args[0] in self): index = args[0] ret = self[index] del self[index] @@ -760,7 +768,7 @@ class ParseResults(object): else: return defaultValue - def insert( self, index, insStr ): + def insert(self, index, insStr): """ Inserts new element at location index in the list of parsed tokens. @@ -777,11 +785,11 @@ class ParseResults(object): """ self.__toklist.insert(index, insStr) # fixup indices in token dictionary - for name,occurrences in self.__tokdict.items(): + for name, occurrences in self.__tokdict.items(): for k, (value, position) in enumerate(occurrences): occurrences[k] = _ParseResultsWithOffset(value, position + (position > index)) - def append( self, item ): + def append(self, item): """ Add single element to end of ParseResults list of elements. @@ -796,7 +804,7 @@ class ParseResults(object): """ self.__toklist.append(item) - def extend( self, itemseq ): + def extend(self, itemseq): """ Add sequence of elements to end of ParseResults list of elements. @@ -815,74 +823,66 @@ class ParseResults(object): else: self.__toklist.extend(itemseq) - def clear( self ): + def clear(self): """ Clear all elements and results names. """ del self.__toklist[:] self.__tokdict.clear() - def __getattr__( self, name ): + def __getattr__(self, name): try: return self[name] except KeyError: return "" - if name in self.__tokdict: - if name not in self.__accumNames: - return self.__tokdict[name][-1][0] - else: - return ParseResults([ v[0] for v in self.__tokdict[name] ]) - else: - return "" - - def __add__( self, other ): + def __add__(self, other): ret = self.copy() ret += other return ret - def __iadd__( self, other ): + def __iadd__(self, other): if other.__tokdict: offset = len(self.__toklist) - addoffset = lambda a: offset if a<0 else a+offset + addoffset = lambda a: offset if a < 0 else a + offset otheritems = other.__tokdict.items() - otherdictitems = [(k, _ParseResultsWithOffset(v[0],addoffset(v[1])) ) - for (k,vlist) in otheritems for v in vlist] - for k,v in otherdictitems: + otherdictitems = [(k, _ParseResultsWithOffset(v[0], addoffset(v[1]))) + for k, vlist in otheritems for v in vlist] + for k, v in otherdictitems: self[k] = v - if isinstance(v[0],ParseResults): + if isinstance(v[0], ParseResults): v[0].__parent = wkref(self) self.__toklist += other.__toklist - self.__accumNames.update( other.__accumNames ) + self.__accumNames.update(other.__accumNames) return self def __radd__(self, other): - if isinstance(other,int) and other == 0: + if isinstance(other, int) and other == 0: # useful for merging many ParseResults using sum() builtin return self.copy() else: # this may raise a TypeError - so be it return other + self - def __repr__( self ): - return "(%s, %s)" % ( repr( self.__toklist ), repr( self.__tokdict ) ) + def __repr__(self): + return "(%s, %s)" % (repr(self.__toklist), repr(self.__tokdict)) - def __str__( self ): + def __str__(self): return '[' + ', '.join(_ustr(i) if isinstance(i, ParseResults) else repr(i) for i in self.__toklist) + ']' - def _asStringList( self, sep='' ): + def _asStringList(self, sep=''): out = [] for item in self.__toklist: if out and sep: out.append(sep) - if isinstance( item, ParseResults ): + if isinstance(item, ParseResults): out += item._asStringList() else: - out.append( _ustr(item) ) + out.append(_ustr(item)) return out - def asList( self ): + def asList(self): """ Returns the parse results as a nested list of matching tokens, all converted to strings. @@ -897,9 +897,9 @@ class ParseResults(object): result_list = result.asList() print(type(result_list), result_list) # -> <class 'list'> ['sldkj', 'lsdkj', 'sldkj'] """ - return [res.asList() if isinstance(res,ParseResults) else res for res in self.__toklist] + return [res.asList() if isinstance(res, ParseResults) else res for res in self.__toklist] - def asDict( self ): + def asDict(self): """ Returns the named parse results as a nested dictionary. @@ -933,27 +933,27 @@ class ParseResults(object): else: return obj - return dict((k,toItem(v)) for k,v in item_fn()) + return dict((k, toItem(v)) for k, v in item_fn()) - def copy( self ): + def copy(self): """ Returns a new copy of a :class:`ParseResults` object. """ - ret = ParseResults( self.__toklist ) + ret = ParseResults(self.__toklist) ret.__tokdict = dict(self.__tokdict.items()) ret.__parent = self.__parent - ret.__accumNames.update( self.__accumNames ) + ret.__accumNames.update(self.__accumNames) ret.__name = self.__name return ret - def asXML( self, doctag=None, namedItemsOnly=False, indent="", formatted=True ): + def asXML(self, doctag=None, namedItemsOnly=False, indent="", formatted=True): """ (Deprecated) Returns the parse results as XML. Tags are created for tokens and lists that have defined results names. """ nl = "\n" out = [] - namedItems = dict((v[1],k) for (k,vlist) in self.__tokdict.items() - for v in vlist) + namedItems = dict((v[1], k) for (k, vlist) in self.__tokdict.items() + for v in vlist) nextLevelIndent = indent + " " # collapse out indents if formatting is not desired @@ -975,20 +975,20 @@ class ParseResults(object): else: selfTag = "ITEM" - out += [ nl, indent, "<", selfTag, ">" ] + out += [nl, indent, "<", selfTag, ">"] - for i,res in enumerate(self.__toklist): - if isinstance(res,ParseResults): + for i, res in enumerate(self.__toklist): + if isinstance(res, ParseResults): if i in namedItems: - out += [ res.asXML(namedItems[i], - namedItemsOnly and doctag is None, - nextLevelIndent, - formatted)] + out += [res.asXML(namedItems[i], + namedItemsOnly and doctag is None, + nextLevelIndent, + formatted)] else: - out += [ res.asXML(None, - namedItemsOnly and doctag is None, - nextLevelIndent, - formatted)] + out += [res.asXML(None, + namedItemsOnly and doctag is None, + nextLevelIndent, + formatted)] else: # individual token, see if there is a name for it resTag = None @@ -1000,16 +1000,16 @@ class ParseResults(object): else: resTag = "ITEM" xmlBodyText = _xml_escape(_ustr(res)) - out += [ nl, nextLevelIndent, "<", resTag, ">", - xmlBodyText, - "</", resTag, ">" ] + out += [nl, nextLevelIndent, "<", resTag, ">", + xmlBodyText, + "</", resTag, ">"] - out += [ nl, indent, "</", selfTag, ">" ] + out += [nl, indent, "</", selfTag, ">"] return "".join(out) - def __lookup(self,sub): - for k,vlist in self.__tokdict.items(): - for v,loc in vlist: + def __lookup(self, sub): + for k, vlist in self.__tokdict.items(): + for v, loc in vlist: if sub is v: return k return None @@ -1047,9 +1047,9 @@ class ParseResults(object): return par.__lookup(self) else: return None - elif (len(self) == 1 and - len(self.__tokdict) == 1 and - next(iter(self.__tokdict.values()))[0][1] in (0,-1)): + elif (len(self) == 1 + and len(self.__tokdict) == 1 + and next(iter(self.__tokdict.values()))[0][1] in (0, -1)): return next(iter(self.__tokdict.keys())) else: return None @@ -1078,43 +1078,43 @@ class ParseResults(object): out = [] NL = '\n' if include_list: - out.append(indent+_ustr(self.asList())) + out.append(indent + _ustr(self.asList())) else: out.append('') if full: if self.haskeys(): - items = sorted((str(k), v) for k,v in self.items()) - for k,v in items: + items = sorted((str(k), v) for k, v in self.items()) + for k, v in items: if out: out.append(NL) out.append("%s%s- %s: " % (indent, (' ' * _depth), k)) - if isinstance(v,ParseResults): + if isinstance(v, ParseResults): if v: - out.append(v.dump(indent=indent, full=full, include_list=include_list, _depth=_depth+1)) + out.append(v.dump(indent=indent, full=full, include_list=include_list, _depth=_depth + 1)) else: out.append(_ustr(v)) else: out.append(repr(v)) - elif any(isinstance(vv,ParseResults) for vv in self): + elif any(isinstance(vv, ParseResults) for vv in self): v = self - for i,vv in enumerate(v): - if isinstance(vv,ParseResults): + for i, vv in enumerate(v): + if isinstance(vv, ParseResults): out.append("\n%s%s[%d]:\n%s%s%s" % (indent, (' ' * (_depth)), i, indent, - (' ' * (_depth+1)), + (' ' * (_depth + 1)), vv.dump(indent=indent, full=full, include_list=include_list, - _depth=_depth+1))) + _depth=_depth + 1))) else: out.append("\n%s%s[%d]:\n%s%s%s" % (indent, (' ' * (_depth)), i, indent, - (' ' * (_depth+1)), + (' ' * (_depth + 1)), _ustr(vv))) return "".join(out) @@ -1148,18 +1148,15 @@ class ParseResults(object): # add support for pickle protocol def __getstate__(self): - return ( self.__toklist, - ( self.__tokdict.copy(), - self.__parent is not None and self.__parent() or None, - self.__accumNames, - self.__name ) ) + return (self.__toklist, + (self.__tokdict.copy(), + self.__parent is not None and self.__parent() or None, + self.__accumNames, + self.__name)) - def __setstate__(self,state): + def __setstate__(self, state): self.__toklist = state[0] - (self.__tokdict, - par, - inAccumNames, - self.__name) = state[1] + self.__tokdict, par, inAccumNames, self.__name = state[1] self.__accumNames = {} self.__accumNames.update(inAccumNames) if par is not None: @@ -1171,7 +1168,7 @@ class ParseResults(object): return self.__toklist, self.__name, self.__asList, self.__modal def __dir__(self): - return (dir(type(self)) + list(self.keys())) + return dir(type(self)) + list(self.keys()) @classmethod def from_dict(cls, other, name=None): @@ -1203,7 +1200,7 @@ class ParseResults(object): MutableMapping.register(ParseResults) -def col (loc,strg): +def col (loc, strg): """Returns current column within a string, counting newlines as line separators. The first column is number 1. @@ -1215,9 +1212,9 @@ def col (loc,strg): location, and line and column positions within the parsed string. """ s = strg - return 1 if 0<loc<len(s) and s[loc-1] == '\n' else loc - s.rfind("\n", 0, loc) + return 1 if 0 < loc < len(s) and s[loc-1] == '\n' else loc - s.rfind("\n", 0, loc) -def lineno(loc,strg): +def lineno(loc, strg): """Returns current line number within a string, counting newlines as line separators. The first line is number 1. @@ -1227,26 +1224,26 @@ def lineno(loc,strg): suggested methods to maintain a consistent view of the parsed string, the parse location, and line and column positions within the parsed string. """ - return strg.count("\n",0,loc) + 1 + return strg.count("\n", 0, loc) + 1 -def line( loc, strg ): +def line(loc, strg): """Returns the line of text containing loc within a string, counting newlines as line separators. """ lastCR = strg.rfind("\n", 0, loc) nextCR = strg.find("\n", loc) if nextCR >= 0: - return strg[lastCR+1:nextCR] + return strg[lastCR + 1:nextCR] else: - return strg[lastCR+1:] + return strg[lastCR + 1:] -def _defaultStartDebugAction( instring, loc, expr ): - print (("Match " + _ustr(expr) + " at loc " + _ustr(loc) + "(%d,%d)" % ( lineno(loc,instring), col(loc,instring) ))) +def _defaultStartDebugAction(instring, loc, expr): + print(("Match " + _ustr(expr) + " at loc " + _ustr(loc) + "(%d,%d)" % (lineno(loc, instring), col(loc, instring)))) -def _defaultSuccessDebugAction( instring, startloc, endloc, expr, toks ): - print ("Matched " + _ustr(expr) + " -> " + str(toks.asList())) +def _defaultSuccessDebugAction(instring, startloc, endloc, expr, toks): + print("Matched " + _ustr(expr) + " -> " + str(toks.asList())) -def _defaultExceptionDebugAction( instring, loc, expr, exc ): - print ("Exception raised:" + _ustr(exc)) +def _defaultExceptionDebugAction(instring, loc, expr, exc): + print("Exception raised:" + _ustr(exc)) def nullDebugAction(*args): """'Do-nothing' debug action, to suppress debugging output during parsing.""" @@ -1277,16 +1274,16 @@ def nullDebugAction(*args): 'decorator to trim function calls to match the arity of the target' def _trim_arity(func, maxargs=2): if func in singleArgBuiltins: - return lambda s,l,t: func(t) + return lambda s, l, t: func(t) limit = [0] foundArity = [False] # traceback return data structure changed in Py3.5 - normalize back to plain tuples - if system_version[:2] >= (3,5): + if system_version[:2] >= (3, 5): def extract_stack(limit=0): # special handling for Python 3.5.0 - extra deep call stack by 1 - offset = -3 if system_version == (3,5,0) else -2 - frame_summary = traceback.extract_stack(limit=-offset+limit-1)[offset] + offset = -3 if system_version == (3, 5, 0) else -2 + frame_summary = traceback.extract_stack(limit=-offset + limit - 1)[offset] return [frame_summary[:2]] def extract_tb(tb, limit=0): frames = traceback.extract_tb(tb, limit=limit) @@ -1303,7 +1300,7 @@ def _trim_arity(func, maxargs=2): # IF ANY CODE CHANGES, EVEN JUST COMMENTS OR BLANK LINES, BETWEEN THE NEXT LINE AND # THE CALL TO FUNC INSIDE WRAPPER, LINE_DIFF MUST BE MODIFIED!!!! this_line = extract_stack(limit=2)[-1] - pa_call_line_synth = (this_line[0], this_line[1]+LINE_DIFF) + pa_call_line_synth = (this_line[0], this_line[1] + LINE_DIFF) def wrapper(*args): while 1: @@ -1349,7 +1346,7 @@ class ParserElement(object): verbose_stacktrace = False @staticmethod - def setDefaultWhitespaceChars( chars ): + def setDefaultWhitespaceChars(chars): r""" Overrides the default whitespace chars @@ -1386,10 +1383,10 @@ class ParserElement(object): """ ParserElement._literalStringClass = cls - def __init__( self, savelist=False ): + def __init__(self, savelist=False): self.parseAction = list() self.failAction = None - #~ self.name = "<unknown>" # don't define self.name, let subclasses try/except upcall + # ~ self.name = "<unknown>" # don't define self.name, let subclasses try/except upcall self.strRepr = None self.resultsName = None self.saveAsList = savelist @@ -1404,12 +1401,12 @@ class ParserElement(object): self.mayIndexError = True # used to optimize exception handling for subclasses that don't advance parse index self.errmsg = "" self.modalResults = True # used to mark results names as modal (report only last) or cumulative (list all) - self.debugActions = ( None, None, None ) #custom debug actions + self.debugActions = (None, None, None) # custom debug actions self.re = None self.callPreparse = True # used to avoid redundant calls to preParse self.callDuringTry = False - def copy( self ): + def copy(self): """ Make a copy of this :class:`ParserElement`. Useful for defining different parse actions for the same parsing pattern, using copies of @@ -1418,8 +1415,8 @@ class ParserElement(object): Example:: integer = Word(nums).setParseAction(lambda toks: int(toks[0])) - integerK = integer.copy().addParseAction(lambda toks: toks[0]*1024) + Suppress("K") - integerM = integer.copy().addParseAction(lambda toks: toks[0]*1024*1024) + Suppress("M") + integerK = integer.copy().addParseAction(lambda toks: toks[0] * 1024) + Suppress("K") + integerM = integer.copy().addParseAction(lambda toks: toks[0] * 1024 * 1024) + Suppress("M") print(OneOrMore(integerK | integerM | integer).parseString("5K 100 640K 256M")) @@ -1429,16 +1426,16 @@ class ParserElement(object): Equivalent form of ``expr.copy()`` is just ``expr()``:: - integerM = integer().addParseAction(lambda toks: toks[0]*1024*1024) + Suppress("M") + integerM = integer().addParseAction(lambda toks: toks[0] * 1024 * 1024) + Suppress("M") """ - cpy = copy.copy( self ) + cpy = copy.copy(self) cpy.parseAction = self.parseAction[:] cpy.ignoreExprs = self.ignoreExprs[:] if self.copyDefaultWhiteChars: cpy.whiteChars = ParserElement.DEFAULT_WHITE_CHARS return cpy - def setName( self, name ): + def setName(self, name): """ Define name for this expression, makes debugging and exception messages clearer. @@ -1453,7 +1450,7 @@ class ParserElement(object): self.setDebug() return self - def setResultsName( self, name, listAllMatches=False ): + def setResultsName(self, name, listAllMatches=False): """ Define name for referencing matching tokens as a nested attribute of the returned parse results. @@ -1480,12 +1477,12 @@ class ParserElement(object): newself = self.copy() if name.endswith("*"): name = name[:-1] - listAllMatches=True + listAllMatches = True newself.resultsName = name newself.modalResults = not listAllMatches return newself - def setBreak(self,breakFlag = True): + def setBreak(self, breakFlag=True): """Method to invoke the Python pdb debugger when this element is about to be parsed. Set ``breakFlag`` to True to enable, False to disable. @@ -1496,19 +1493,19 @@ class ParserElement(object): import pdb # this call to pdb.set_trace() is intentional, not a checkin error pdb.set_trace() - return _parseMethod( instring, loc, doActions, callPreParse ) + return _parseMethod(instring, loc, doActions, callPreParse) breaker._originalParseMethod = _parseMethod self._parse = breaker else: - if hasattr(self._parse,"_originalParseMethod"): + if hasattr(self._parse, "_originalParseMethod"): self._parse = self._parse._originalParseMethod return self - def setParseAction( self, *fns, **kwargs ): + def setParseAction(self, *fns, **kwargs): """ Define one or more actions to perform when successfully matching parse element definition. - Parse action fn is a callable method with 0-3 arguments, called as ``fn(s,loc,toks)`` , - ``fn(loc,toks)`` , ``fn(toks)`` , or just ``fn()`` , where: + Parse action fn is a callable method with 0-3 arguments, called as ``fn(s, loc, toks)`` , + ``fn(loc, toks)`` , ``fn(toks)`` , or just ``fn()`` , where: - s = the original string being parsed (see note below) - loc = the location of the matching substring @@ -1522,7 +1519,7 @@ class ParserElement(object): expression are cleared. Optional keyword arguments: - - callDuringTry = (default= ``False`` ) indicate if parse action should be run during lookaheads and alternate testing + - callDuringTry = (default= ``False``) indicate if parse action should be run during lookaheads and alternate testing Note: the default parsing behavior is to expand tabs in the input string before starting the parsing process. See :class:`parseString for more @@ -1544,7 +1541,7 @@ class ParserElement(object): # note that integer fields are now ints, not strings date_str.parseString("1999/12/31") # -> [1999, '/', 12, '/', 31] """ - if list(fns) == [None, ]: + if list(fns) == [None,]: self.parseAction = None else: if not all(callable(fn) for fn in fns): @@ -1553,7 +1550,7 @@ class ParserElement(object): self.callDuringTry = kwargs.get("callDuringTry", False) return self - def addParseAction( self, *fns, **kwargs ): + def addParseAction(self, *fns, **kwargs): """ Add one or more parse actions to expression's list of parse actions. See :class:`setParseAction`. @@ -1588,10 +1585,10 @@ class ParserElement(object): self.callDuringTry = self.callDuringTry or kwargs.get("callDuringTry", False) return self - def setFailAction( self, fn ): + def setFailAction(self, fn): """Define action to perform if parsing fails at this expression. Fail acton fn is a callable function that takes the arguments - ``fn(s,loc,expr,err)`` where: + ``fn(s, loc, expr, err)`` where: - s = string being parsed - loc = location where expression match was attempted and failed - expr = the parse expression that failed @@ -1601,22 +1598,22 @@ class ParserElement(object): self.failAction = fn return self - def _skipIgnorables( self, instring, loc ): + def _skipIgnorables(self, instring, loc): exprsFound = True while exprsFound: exprsFound = False for e in self.ignoreExprs: try: while 1: - loc,dummy = e._parse( instring, loc ) + loc, dummy = e._parse(instring, loc) exprsFound = True except ParseException: pass return loc - def preParse( self, instring, loc ): + def preParse(self, instring, loc): if self.ignoreExprs: - loc = self._skipIgnorables( instring, loc ) + loc = self._skipIgnorables(instring, loc) if self.skipWhitespace: wt = self.whiteChars @@ -1626,21 +1623,21 @@ class ParserElement(object): return loc - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): return loc, [] - def postParse( self, instring, loc, tokenlist ): + def postParse(self, instring, loc, tokenlist): return tokenlist - #~ @profile - def _parseNoCache( self, instring, loc, doActions=True, callPreParse=True ): + # ~ @profile + def _parseNoCache(self, instring, loc, doActions=True, callPreParse=True): TRY, MATCH, FAIL = 0, 1, 2 - debugging = ( self.debug ) #and doActions ) + debugging = (self.debug) # and doActions) if debugging or self.failAction: - #~ print ("Match",self,"at loc",loc,"(%d,%d)" % ( lineno(loc,instring), col(loc,instring) )) + # ~ print ("Match", self, "at loc", loc, "(%d, %d)" % (lineno(loc, instring), col(loc, instring))) if self.debugActions[TRY]: - self.debugActions[TRY]( instring, loc, self ) + self.debugActions[TRY](instring, loc, self) try: if callPreParse and self.callPreparse: preloc = self.preParse(instring, loc) @@ -1655,76 +1652,76 @@ class ParserElement(object): else: loc, tokens = self.parseImpl(instring, preloc, doActions) except Exception as err: - #~ print ("Exception raised:", err) + # ~ print ("Exception raised:", err) if self.debugActions[FAIL]: - self.debugActions[FAIL]( instring, tokensStart, self, err ) + self.debugActions[FAIL](instring, tokensStart, self, err) if self.failAction: - self.failAction( instring, tokensStart, self, err ) + self.failAction(instring, tokensStart, self, err) raise else: if callPreParse and self.callPreparse: - preloc = self.preParse( instring, loc ) + preloc = self.preParse(instring, loc) else: preloc = loc tokensStart = preloc if self.mayIndexError or preloc >= len(instring): try: - loc,tokens = self.parseImpl( instring, preloc, doActions ) + loc, tokens = self.parseImpl(instring, preloc, doActions) except IndexError: - raise ParseException( instring, len(instring), self.errmsg, self ) + raise ParseException(instring, len(instring), self.errmsg, self) else: - loc,tokens = self.parseImpl( instring, preloc, doActions ) + loc, tokens = self.parseImpl(instring, preloc, doActions) - tokens = self.postParse( instring, loc, tokens ) + tokens = self.postParse(instring, loc, tokens) - retTokens = ParseResults( tokens, self.resultsName, asList=self.saveAsList, modal=self.modalResults ) + retTokens = ParseResults(tokens, self.resultsName, asList=self.saveAsList, modal=self.modalResults) if self.parseAction and (doActions or self.callDuringTry): if debugging: try: for fn in self.parseAction: try: - tokens = fn( instring, tokensStart, retTokens ) + tokens = fn(instring, tokensStart, retTokens) except IndexError as parse_action_exc: exc = ParseException("exception raised in parse action") exc.__cause__ = parse_action_exc raise exc if tokens is not None and tokens is not retTokens: - retTokens = ParseResults( tokens, + retTokens = ParseResults(tokens, self.resultsName, - asList=self.saveAsList and isinstance(tokens,(ParseResults,list)), - modal=self.modalResults ) + asList=self.saveAsList and isinstance(tokens, (ParseResults, list)), + modal=self.modalResults) except Exception as err: - #~ print "Exception raised in user parse action:", err + # ~ print "Exception raised in user parse action:", err if self.debugActions[FAIL]: - self.debugActions[FAIL]( instring, tokensStart, self, err ) + self.debugActions[FAIL](instring, tokensStart, self, err) raise else: for fn in self.parseAction: try: - tokens = fn( instring, tokensStart, retTokens ) + tokens = fn(instring, tokensStart, retTokens) except IndexError as parse_action_exc: exc = ParseException("exception raised in parse action") exc.__cause__ = parse_action_exc raise exc if tokens is not None and tokens is not retTokens: - retTokens = ParseResults( tokens, + retTokens = ParseResults(tokens, self.resultsName, - asList=self.saveAsList and isinstance(tokens,(ParseResults,list)), - modal=self.modalResults ) + asList=self.saveAsList and isinstance(tokens, (ParseResults, list)), + modal=self.modalResults) if debugging: - #~ print ("Matched",self,"->",retTokens.asList()) + # ~ print ("Matched", self, "->", retTokens.asList()) if self.debugActions[MATCH]: - self.debugActions[MATCH]( instring, tokensStart, loc, self, retTokens ) + self.debugActions[MATCH](instring, tokensStart, loc, self, retTokens) return loc, retTokens - def tryParse( self, instring, loc ): + def tryParse(self, instring, loc): try: - return self._parse( instring, loc, doActions=False )[0] + return self._parse(instring, loc, doActions=False)[0] except ParseFatalException: - raise ParseException( instring, loc, self.errmsg, self) + raise ParseException(instring, loc, self.errmsg, self) def canParseNext(self, instring, loc): try: @@ -1821,7 +1818,7 @@ class ParserElement(object): # this method gets repeatedly called during backtracking with the same arguments - # we can cache these arguments and save ourselves the trouble of re-parsing the contained expression - def _parseCache( self, instring, loc, doActions=True, callPreParse=True ): + def _parseCache(self, instring, loc, doActions=True, callPreParse=True): HIT, MISS = 0, 1 lookup = (self, instring, loc, callPreParse, doActions) with ParserElement.packrat_cache_lock: @@ -1842,7 +1839,7 @@ class ParserElement(object): ParserElement.packrat_cache_stats[HIT] += 1 if isinstance(value, Exception): raise value - return (value[0], value[1].copy()) + return value[0], value[1].copy() _parse = _parseNoCache @@ -1887,7 +1884,7 @@ class ParserElement(object): ParserElement.packrat_cache = ParserElement._FifoCache(cache_size_limit) ParserElement._parse = ParserElement._parseCache - def parseString( self, instring, parseAll=False ): + def parseString(self, instring, parseAll=False): """ Execute the parse expression with the given string. This is the main interface to the client code, once the complete @@ -1910,7 +1907,7 @@ class ParserElement(object): - calling ``parseWithTabs`` on your grammar before calling ``parseString`` (see :class:`parseWithTabs`) - - define your parse action using the full ``(s,loc,toks)`` signature, and + - define your parse action using the full ``(s, loc, toks)`` signature, and reference the input string using the parse action's ``s`` argument - explictly expand the tabs in your input string before calling ``parseString`` @@ -1923,17 +1920,17 @@ class ParserElement(object): ParserElement.resetCache() if not self.streamlined: self.streamline() - #~ self.saveAsList = True + # ~ self.saveAsList = True for e in self.ignoreExprs: e.streamline() if not self.keepTabs: instring = instring.expandtabs() try: - loc, tokens = self._parse( instring, 0 ) + loc, tokens = self._parse(instring, 0) if parseAll: - loc = self.preParse( instring, loc ) + loc = self.preParse(instring, loc) se = Empty() + StringEnd() - se._parse( instring, loc ) + se._parse(instring, loc) except ParseBaseException as exc: if ParserElement.verbose_stacktrace: raise @@ -1943,7 +1940,7 @@ class ParserElement(object): else: return tokens - def scanString( self, instring, maxMatches=_MAX_INT, overlap=False ): + def scanString(self, instring, maxMatches=_MAX_INT, overlap=False): """ Scan the input string for expression matches. Each match will return the matching tokens, start location, and end location. May be called with optional @@ -1958,7 +1955,7 @@ class ParserElement(object): source = "sldjf123lsdjjkf345sldkjf879lkjsfd987" print(source) - for tokens,start,end in Word(alphas).scanString(source): + for tokens, start, end in Word(alphas).scanString(source): print(' '*start + '^'*(end-start)) print(' '*start + tokens[0]) @@ -1990,16 +1987,16 @@ class ParserElement(object): try: while loc <= instrlen and matches < maxMatches: try: - preloc = preparseFn( instring, loc ) - nextLoc,tokens = parseFn( instring, preloc, callPreParse=False ) + preloc = preparseFn(instring, loc) + nextLoc, tokens = parseFn(instring, preloc, callPreParse=False) except ParseException: - loc = preloc+1 + loc = preloc + 1 else: if nextLoc > loc: matches += 1 yield tokens, preloc, nextLoc if overlap: - nextloc = preparseFn( instring, loc ) + nextloc = preparseFn(instring, loc) if nextloc > loc: loc = nextLoc else: @@ -2007,7 +2004,7 @@ class ParserElement(object): else: loc = nextLoc else: - loc = preloc+1 + loc = preloc + 1 except ParseBaseException as exc: if ParserElement.verbose_stacktrace: raise @@ -2015,7 +2012,7 @@ class ParserElement(object): # catch and re-raise exception from here, clears out pyparsing internal stack trace raise exc - def transformString( self, instring ): + def transformString(self, instring): """ Extension to :class:`scanString`, to modify matching text with modified tokens that may be returned from a parse action. To use ``transformString``, define a grammar and @@ -2041,19 +2038,19 @@ class ParserElement(object): # keep string locs straight between transformString and scanString self.keepTabs = True try: - for t,s,e in self.scanString( instring ): - out.append( instring[lastE:s] ) + for t, s, e in self.scanString(instring): + out.append(instring[lastE:s]) if t: - if isinstance(t,ParseResults): + if isinstance(t, ParseResults): out += t.asList() - elif isinstance(t,list): + elif isinstance(t, list): out += t else: out.append(t) lastE = e out.append(instring[lastE:]) out = [o for o in out if o] - return "".join(map(_ustr,_flatten(out))) + return "".join(map(_ustr, _flatten(out))) except ParseBaseException as exc: if ParserElement.verbose_stacktrace: raise @@ -2061,7 +2058,7 @@ class ParserElement(object): # catch and re-raise exception from here, clears out pyparsing internal stack trace raise exc - def searchString( self, instring, maxMatches=_MAX_INT ): + def searchString(self, instring, maxMatches=_MAX_INT): """ Another extension to :class:`scanString`, simplifying the access to the tokens found to match the given parse expression. May be called with optional @@ -2083,7 +2080,7 @@ class ParserElement(object): ['More', 'Iron', 'Lead', 'Gold', 'I', 'Electricity'] """ try: - return ParseResults([ t for t,s,e in self.scanString( instring, maxMatches ) ]) + return ParseResults([t for t, s, e in self.scanString(instring, maxMatches)]) except ParseBaseException as exc: if ParserElement.verbose_stacktrace: raise @@ -2109,14 +2106,14 @@ class ParserElement(object): """ splits = 0 last = 0 - for t,s,e in self.scanString(instring, maxMatches=maxsplit): + for t, s, e in self.scanString(instring, maxMatches=maxsplit): yield instring[last:s] if includeSeparators: yield t[0] last = e yield instring[last:] - def __add__(self, other ): + def __add__(self, other): """ Implementation of + operator - returns :class:`And`. Adding strings to a ParserElement converts them to :class:`Literal`s by default. @@ -2146,15 +2143,15 @@ class ParserElement(object): if other is Ellipsis: return _PendingSkip(self) - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None - return And( [ self, other ] ) + return And([self, other]) - def __radd__(self, other ): + def __radd__(self, other): """ Implementation of + operator when left operand is not a :class:`ParserElement` """ @@ -2162,10 +2159,10 @@ class ParserElement(object): return SkipTo(self)("_skipped*") + self if isinstance(other, basestring): - other = ParserElement._literalStringClass(other) + other = self._literalStringClass(other) if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return other + self @@ -2173,67 +2170,67 @@ class ParserElement(object): """ Implementation of - operator, returns :class:`And` with error stop """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return self + And._ErrorStop() + other - def __rsub__(self, other ): + def __rsub__(self, other): """ Implementation of - operator when left operand is not a :class:`ParserElement` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return other - self - def __mul__(self,other): + def __mul__(self, other): """ Implementation of * operator, allows use of ``expr * 3`` in place of ``expr + expr + expr``. Expressions may also me multiplied by a 2-integer - tuple, similar to ``{min,max}`` multipliers in regular expressions. Tuples + tuple, similar to ``{min, max}`` multipliers in regular expressions. Tuples may also include ``None`` as in: - - ``expr*(n,None)`` or ``expr*(n,)`` is equivalent + - ``expr*(n, None)`` or ``expr*(n, )`` is equivalent to ``expr*n + ZeroOrMore(expr)`` (read as "at least n instances of ``expr``") - - ``expr*(None,n)`` is equivalent to ``expr*(0,n)`` + - ``expr*(None, n)`` is equivalent to ``expr*(0, n)`` (read as "0 to n instances of ``expr``") - - ``expr*(None,None)`` is equivalent to ``ZeroOrMore(expr)`` - - ``expr*(1,None)`` is equivalent to ``OneOrMore(expr)`` + - ``expr*(None, None)`` is equivalent to ``ZeroOrMore(expr)`` + - ``expr*(1, None)`` is equivalent to ``OneOrMore(expr)`` - Note that ``expr*(None,n)`` does not raise an exception if + Note that ``expr*(None, n)`` does not raise an exception if more than n exprs exist in the input stream; that is, - ``expr*(None,n)`` does not enforce a maximum number of expr + ``expr*(None, n)`` does not enforce a maximum number of expr occurrences. If this behavior is desired, then write - ``expr*(None,n) + ~expr`` + ``expr*(None, n) + ~expr`` """ - if other is Ellipsis or other == (Ellipsis, ): + if other is Ellipsis or other == (Ellipsis,): other = (1, None) - if isinstance(other,int): - minElements, optElements = other,0 + if isinstance(other, int): + minElements, optElements = other, 0 elif isinstance(other, tuple): other = tuple(o if o is not Ellipsis else None for o in other) other = (other + (None, None))[:2] if other[0] is None: other = (0, other[1]) - if isinstance(other[0],int) and other[1] is None: + if isinstance(other[0], int) and other[1] is None: if other[0] == 0: return ZeroOrMore(self) if other[0] == 1: return OneOrMore(self) else: - return self*other[0] + ZeroOrMore(self) - elif isinstance(other[0],int) and isinstance(other[1],int): + return self * other[0] + ZeroOrMore(self) + elif isinstance(other[0], int) and isinstance(other[1], int): minElements, optElements = other optElements -= minElements else: - raise TypeError("cannot multiply 'ParserElement' and ('%s','%s') objects", type(other[0]),type(other[1])) + raise TypeError("cannot multiply 'ParserElement' and ('%s', '%s') objects", type(other[0]), type(other[1])) else: raise TypeError("cannot multiply 'ParserElement' and '%s' objects", type(other)) @@ -2242,111 +2239,111 @@ class ParserElement(object): if optElements < 0: raise ValueError("second tuple value must be greater or equal to first tuple value") if minElements == optElements == 0: - raise ValueError("cannot multiply ParserElement by 0 or (0,0)") + raise ValueError("cannot multiply ParserElement by 0 or (0, 0)") - if (optElements): + if optElements: def makeOptionalList(n): - if n>1: - return Optional(self + makeOptionalList(n-1)) + if n > 1: + return Optional(self + makeOptionalList(n - 1)) else: return Optional(self) if minElements: if minElements == 1: ret = self + makeOptionalList(optElements) else: - ret = And([self]*minElements) + makeOptionalList(optElements) + ret = And([self] * minElements) + makeOptionalList(optElements) else: ret = makeOptionalList(optElements) else: if minElements == 1: ret = self else: - ret = And([self]*minElements) + ret = And([self] * minElements) return ret def __rmul__(self, other): return self.__mul__(other) - def __or__(self, other ): + def __or__(self, other): """ Implementation of | operator - returns :class:`MatchFirst` """ if other is Ellipsis: return _PendingSkip(self, must_skip=True) - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None - return MatchFirst( [ self, other ] ) + return MatchFirst([self, other]) - def __ror__(self, other ): + def __ror__(self, other): """ Implementation of | operator when left operand is not a :class:`ParserElement` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return other | self - def __xor__(self, other ): + def __xor__(self, other): """ Implementation of ^ operator - returns :class:`Or` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None - return Or( [ self, other ] ) + return Or([self, other]) - def __rxor__(self, other ): + def __rxor__(self, other): """ Implementation of ^ operator when left operand is not a :class:`ParserElement` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return other ^ self - def __and__(self, other ): + def __and__(self, other): """ Implementation of & operator - returns :class:`Each` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None - return Each( [ self, other ] ) + return Each([self, other]) - def __rand__(self, other ): + def __rand__(self, other): """ Implementation of & operator when left operand is not a :class:`ParserElement` """ - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - if not isinstance( other, ParserElement ): + if isinstance(other, basestring): + other = self._literalStringClass(other) + if not isinstance(other, ParserElement): warnings.warn("Cannot combine element of type %s with ParserElement" % type(other), - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) return None return other & self - def __invert__( self ): + def __invert__(self): """ Implementation of ~ operator - returns :class:`NotAny` """ - return NotAny( self ) + return NotAny(self) def __getitem__(self, key): """ @@ -2356,7 +2353,7 @@ class ParserElement(object): - ``expr[n, ...]`` or ``expr[n,]`` is equivalent to ``expr*n + ZeroOrMore(expr)`` (read as "at least n instances of ``expr``") - - ``expr[..., n]`` is equivalent to ``expr*(0,n)`` + - ``expr[..., n]`` is equivalent to ``expr*(0, n)`` (read as "0 to n instances of ``expr``") - ``expr[0, ...]`` is equivalent to ``ZeroOrMore(expr)`` - ``expr[1, ...]`` is equivalent to ``OneOrMore(expr)`` @@ -2397,22 +2394,22 @@ class ParserElement(object): Example:: # these are equivalent - userdata = Word(alphas).setResultsName("name") + Word(nums+"-").setResultsName("socsecno") - userdata = Word(alphas)("name") + Word(nums+"-")("socsecno") + userdata = Word(alphas).setResultsName("name") + Word(nums + "-").setResultsName("socsecno") + userdata = Word(alphas)("name") + Word(nums + "-")("socsecno") """ if name is not None: return self._setResultsName(name) else: return self.copy() - def suppress( self ): + def suppress(self): """ Suppresses the output of this :class:`ParserElement`; useful to keep punctuation from cluttering up returned output. """ - return Suppress( self ) + return Suppress(self) - def leaveWhitespace( self ): + def leaveWhitespace(self): """ Disables the skipping of whitespace before matching the characters in the :class:`ParserElement`'s defined pattern. This is normally only used internally by @@ -2421,7 +2418,7 @@ class ParserElement(object): self.skipWhitespace = False return self - def setWhitespaceChars( self, chars ): + def setWhitespaceChars(self, chars): """ Overrides the default whitespace chars """ @@ -2430,7 +2427,7 @@ class ParserElement(object): self.copyDefaultWhiteChars = False return self - def parseWithTabs( self ): + def parseWithTabs(self): """ Overrides default behavior to expand ``<TAB>``s to spaces before parsing the input string. Must be called before ``parseString`` when the input grammar contains elements that @@ -2439,7 +2436,7 @@ class ParserElement(object): self.keepTabs = True return self - def ignore( self, other ): + def ignore(self, other): """ Define expression to be ignored (e.g., comments) while doing pattern matching; may be called repeatedly, to define multiple comment or other @@ -2456,14 +2453,14 @@ class ParserElement(object): if isinstance(other, basestring): other = Suppress(other) - if isinstance( other, Suppress ): + if isinstance(other, Suppress): if other not in self.ignoreExprs: self.ignoreExprs.append(other) else: - self.ignoreExprs.append( Suppress( other.copy() ) ) + self.ignoreExprs.append(Suppress(other.copy())) return self - def setDebugActions( self, startAction, successAction, exceptionAction ): + def setDebugActions(self, startAction, successAction, exceptionAction): """ Enable display of debugging messages while doing pattern matching. """ @@ -2473,7 +2470,7 @@ class ParserElement(object): self.debug = True return self - def setDebug( self, flag=True ): + def setDebug(self, flag=True): """ Enable display of debugging messages while doing pattern matching. Set ``flag`` to True to enable, False to disable. @@ -2511,32 +2508,32 @@ class ParserElement(object): name created for the :class:`Word` expression without calling ``setName`` is ``"W:(ABCD...)"``. """ if flag: - self.setDebugActions( _defaultStartDebugAction, _defaultSuccessDebugAction, _defaultExceptionDebugAction ) + self.setDebugActions(_defaultStartDebugAction, _defaultSuccessDebugAction, _defaultExceptionDebugAction) else: self.debug = False return self - def __str__( self ): + def __str__(self): return self.name - def __repr__( self ): + def __repr__(self): return _ustr(self) - def streamline( self ): + def streamline(self): self.streamlined = True self.strRepr = None return self - def checkRecursion( self, parseElementList ): + def checkRecursion(self, parseElementList): pass - def validate( self, validateTrace=[] ): + def validate(self, validateTrace=None): """ Check defined expressions for valid structure, check for infinite recursive definitions. """ - self.checkRecursion( [] ) + self.checkRecursion([]) - def parseFile( self, file_or_filename, parseAll=False ): + def parseFile(self, file_or_filename, parseAll=False): """ Execute the parse expression on the given file or filename. If a filename is specified (instead of a file object), @@ -2565,18 +2562,18 @@ class ParserElement(object): elif isinstance(other, basestring): return self.matches(other) else: - return super(ParserElement,self) == other + return super(ParserElement, self) == other - def __ne__(self,other): + def __ne__(self, other): return not (self == other) def __hash__(self): return id(self) - def __req__(self,other): + def __req__(self, other): return self == other - def __rne__(self,other): + def __rne__(self, other): return not (self == other) def matches(self, testString, parseAll=True): @@ -2724,9 +2721,9 @@ class ParserElement(object): fatal = "(FATAL)" if isinstance(pe, ParseFatalException) else "" if '\n' in t: out.append(line(pe.loc, t)) - out.append(' '*(col(pe.loc,t)-1) + '^' + fatal) + out.append(' ' * (col(pe.loc, t) - 1) + '^' + fatal) else: - out.append(' '*pe.loc + '^' + fatal) + out.append(' ' * pe.loc + '^' + fatal) out.append("FAIL: " + str(pe)) success = success and failureTests result = pe @@ -2799,15 +2796,15 @@ class Token(ParserElement): """Abstract :class:`ParserElement` subclass, for defining atomic matching patterns. """ - def __init__( self ): - super(Token,self).__init__( savelist=False ) + def __init__(self): + super(Token, self).__init__(savelist=False) class Empty(Token): """An empty token, will always match. """ - def __init__( self ): - super(Empty,self).__init__() + def __init__(self): + super(Empty, self).__init__() self.name = "Empty" self.mayReturnEmpty = True self.mayIndexError = False @@ -2816,14 +2813,14 @@ class Empty(Token): class NoMatch(Token): """A token that will never match. """ - def __init__( self ): - super(NoMatch,self).__init__() + def __init__(self): + super(NoMatch, self).__init__() self.name = "NoMatch" self.mayReturnEmpty = True self.mayIndexError = False self.errmsg = "Unmatchable token" - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): raise ParseException(instring, loc, self.errmsg, self) @@ -2841,8 +2838,8 @@ class Literal(Token): For keyword matching (force word break before and after the matched string), use :class:`Keyword` or :class:`CaselessKeyword`. """ - def __init__( self, matchString ): - super(Literal,self).__init__() + def __init__(self, matchString): + super(Literal, self).__init__() self.match = matchString self.matchLen = len(matchString) try: @@ -2862,7 +2859,7 @@ class Literal(Token): self.__class__ = _SingleCharLiteral def parseImpl(self, instring, loc, doActions=True): - if instring[loc] == self.firstMatchChar and instring.startswith(self.match,loc): + if instring[loc] == self.firstMatchChar and instring.startswith(self.match, loc): return loc + self.matchLen, self.match raise ParseException(instring, loc, self.errmsg, self) @@ -2900,10 +2897,10 @@ class Keyword(Token): For case-insensitive matching, use :class:`CaselessKeyword`. """ - DEFAULT_KEYWORD_CHARS = alphanums+"_$" + DEFAULT_KEYWORD_CHARS = alphanums + "_$" - def __init__( self, matchString, identChars=None, caseless=False ): - super(Keyword,self).__init__() + def __init__(self, matchString, identChars=None, caseless=False): + super(Keyword, self).__init__() if identChars is None: identChars = Keyword.DEFAULT_KEYWORD_CHARS self.match = matchString @@ -2912,7 +2909,7 @@ class Keyword(Token): self.firstMatchChar = matchString[0] except IndexError: warnings.warn("null string passed to Keyword; use Empty() instead", - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) self.name = '"%s"' % self.match self.errmsg = "Expected " + self.name self.mayReturnEmpty = False @@ -2923,27 +2920,32 @@ class Keyword(Token): identChars = identChars.upper() self.identChars = set(identChars) - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if self.caseless: - if ( (instring[ loc:loc+self.matchLen ].upper() == self.caselessmatch) and - (loc >= len(instring)-self.matchLen or instring[loc+self.matchLen].upper() not in self.identChars) and - (loc == 0 or instring[loc-1].upper() not in self.identChars) ): - return loc+self.matchLen, self.match + if ((instring[loc:loc + self.matchLen].upper() == self.caselessmatch) + and (loc >= len(instring) - self.matchLen + or instring[loc + self.matchLen].upper() not in self.identChars) + and (loc == 0 + or instring[loc - 1].upper() not in self.identChars)): + return loc + self.matchLen, self.match + else: - if (instring[loc] == self.firstMatchChar and - (self.matchLen==1 or instring.startswith(self.match,loc)) and - (loc >= len(instring)-self.matchLen or instring[loc+self.matchLen] not in self.identChars) and - (loc == 0 or instring[loc-1] not in self.identChars) ): - return loc+self.matchLen, self.match + if instring[loc] == self.firstMatchChar: + if ((self.matchLen == 1 or instring.startswith(self.match, loc)) + and (loc >= len(instring) - self.matchLen + or instring[loc + self.matchLen] not in self.identChars) + and (loc == 0 or instring[loc - 1] not in self.identChars)): + return loc + self.matchLen, self.match + raise ParseException(instring, loc, self.errmsg, self) def copy(self): - c = super(Keyword,self).copy() + c = super(Keyword, self).copy() c.identChars = Keyword.DEFAULT_KEYWORD_CHARS return c @staticmethod - def setDefaultKeywordChars( chars ): + def setDefaultKeywordChars(chars): """Overrides the default Keyword chars """ Keyword.DEFAULT_KEYWORD_CHARS = chars @@ -2959,16 +2961,16 @@ class CaselessLiteral(Literal): (Contrast with example for :class:`CaselessKeyword`.) """ - def __init__( self, matchString ): - super(CaselessLiteral,self).__init__( matchString.upper() ) + def __init__(self, matchString): + super(CaselessLiteral, self).__init__(matchString.upper()) # Preserve the defining literal. self.returnString = matchString self.name = "'%s'" % self.returnString self.errmsg = "Expected " + self.name - def parseImpl( self, instring, loc, doActions=True ): - if instring[ loc:loc+self.matchLen ].upper() == self.match: - return loc+self.matchLen, self.returnString + def parseImpl(self, instring, loc, doActions=True): + if instring[loc:loc + self.matchLen].upper() == self.match: + return loc + self.matchLen, self.returnString raise ParseException(instring, loc, self.errmsg, self) class CaselessKeyword(Keyword): @@ -2981,8 +2983,8 @@ class CaselessKeyword(Keyword): (Contrast with example for :class:`CaselessLiteral`.) """ - def __init__( self, matchString, identChars=None ): - super(CaselessKeyword,self).__init__( matchString, identChars, caseless=True ) + def __init__(self, matchString, identChars=None): + super(CaselessKeyword, self).__init__(matchString, identChars, caseless=True) class CloseMatch(Token): """A variation on :class:`Literal` which matches "close" matches, @@ -3018,7 +3020,7 @@ class CloseMatch(Token): patt.parseString("ATCAXCGAAXGGA") # -> (['ATCAXCGAAXGGA'], {'mismatches': [[4, 9]], 'original': ['ATCATCGAATGGA']}) """ def __init__(self, match_string, maxMismatches=1): - super(CloseMatch,self).__init__() + super(CloseMatch, self).__init__() self.name = match_string self.match_string = match_string self.maxMismatches = maxMismatches @@ -3026,7 +3028,7 @@ class CloseMatch(Token): self.mayIndexError = False self.mayReturnEmpty = False - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): start = loc instrlen = len(instring) maxloc = start + len(self.match_string) @@ -3037,8 +3039,8 @@ class CloseMatch(Token): mismatches = [] maxMismatches = self.maxMismatches - for match_stringloc,s_m in enumerate(zip(instring[loc:maxloc], self.match_string)): - src,mat = s_m + for match_stringloc, s_m in enumerate(zip(instring[loc:maxloc], match_string)): + src, mat = s_m if src != mat: mismatches.append(match_stringloc) if len(mismatches) > maxMismatches: @@ -3046,7 +3048,7 @@ class CloseMatch(Token): else: loc = match_stringloc + 1 results = ParseResults([instring[start:loc]]) - results['original'] = self.match_string + results['original'] = match_string results['mismatches'] = mismatches return loc, results @@ -3098,7 +3100,7 @@ class Word(Token): capital_word = Word(alphas.upper(), alphas.lower()) # hostnames are alphanumeric, with leading alpha, and '-' - hostname = Word(alphas, alphanums+'-') + hostname = Word(alphas, alphanums + '-') # roman numeral (not a strict parser, accepts invalid mix of characters) roman = Word("IVXLCDM") @@ -3106,8 +3108,8 @@ class Word(Token): # any string of non-whitespace characters, except for ',' csv_value = Word(printables, excludeChars=",") """ - def __init__( self, initChars, bodyChars=None, min=1, max=0, exact=0, asKeyword=False, excludeChars=None ): - super(Word,self).__init__() + def __init__(self, initChars, bodyChars=None, min=1, max=0, exact=0, asKeyword=False, excludeChars=None): + super(Word, self).__init__() if excludeChars: excludeChars = set(excludeChars) initChars = ''.join(c for c in initChars if c not in excludeChars) @@ -3115,7 +3117,7 @@ class Word(Token): bodyChars = ''.join(c for c in bodyChars if c not in excludeChars) self.initCharsOrig = initChars self.initChars = set(initChars) - if bodyChars : + if bodyChars: self.bodyCharsOrig = bodyChars self.bodyChars = set(bodyChars) else: @@ -3143,29 +3145,27 @@ class Word(Token): self.mayIndexError = False self.asKeyword = asKeyword - if ' ' not in self.initCharsOrig+self.bodyCharsOrig and (min==1 and max==0 and exact==0): + if ' ' not in self.initCharsOrig + self.bodyCharsOrig and (min == 1 and max == 0 and exact == 0): if self.bodyCharsOrig == self.initCharsOrig: self.reString = "[%s]+" % _escapeRegexRangeChars(self.initCharsOrig) elif len(self.initCharsOrig) == 1: - self.reString = "%s[%s]*" % \ - (re.escape(self.initCharsOrig), - _escapeRegexRangeChars(self.bodyCharsOrig),) + self.reString = "%s[%s]*" % (re.escape(self.initCharsOrig), + _escapeRegexRangeChars(self.bodyCharsOrig),) else: - self.reString = "[%s][%s]*" % \ - (_escapeRegexRangeChars(self.initCharsOrig), - _escapeRegexRangeChars(self.bodyCharsOrig),) + self.reString = "[%s][%s]*" % (_escapeRegexRangeChars(self.initCharsOrig), + _escapeRegexRangeChars(self.bodyCharsOrig),) if self.asKeyword: - self.reString = r"\b"+self.reString+r"\b" + self.reString = r"\b" + self.reString + r"\b" try: - self.re = re.compile( self.reString ) + self.re = re.compile(self.reString) except Exception: self.re = None else: self.re_match = self.re.match self.__class__ = _WordRegex - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if instring[loc] not in self.initChars: raise ParseException(instring, loc, self.errmsg, self) @@ -3174,7 +3174,7 @@ class Word(Token): instrlen = len(instring) bodychars = self.bodyChars maxloc = start + self.maxLen - maxloc = min( maxloc, instrlen ) + maxloc = min(maxloc, instrlen) while loc < maxloc and instring[loc] in bodychars: loc += 1 @@ -3184,7 +3184,8 @@ class Word(Token): elif self.maxSpecified and loc < instrlen and instring[loc] in bodychars: throwException = True elif self.asKeyword: - if (start>0 and instring[start-1] in bodychars) or (loc<instrlen and instring[loc] in bodychars): + if (start > 0 and instring[start - 1] in bodychars + or loc < instrlen and instring[loc] in bodychars): throwException = True if throwException: @@ -3192,30 +3193,30 @@ class Word(Token): return loc, instring[start:loc] - def __str__( self ): + def __str__(self): try: - return super(Word,self).__str__() + return super(Word, self).__str__() except Exception: pass if self.strRepr is None: def charsAsStr(s): - if len(s)>4: - return s[:4]+"..." + if len(s) > 4: + return s[:4] + "..." else: return s - if ( self.initCharsOrig != self.bodyCharsOrig ): - self.strRepr = "W:(%s,%s)" % ( charsAsStr(self.initCharsOrig), charsAsStr(self.bodyCharsOrig) ) + if self.initCharsOrig != self.bodyCharsOrig: + self.strRepr = "W:(%s, %s)" % (charsAsStr(self.initCharsOrig), charsAsStr(self.bodyCharsOrig)) else: self.strRepr = "W:(%s)" % charsAsStr(self.initCharsOrig) return self.strRepr class _WordRegex(Word): - def parseImpl( self, instring, loc, doActions=True ): - result = self.re_match(instring,loc) + def parseImpl(self, instring, loc, doActions=True): + result = self.re_match(instring, loc) if not result: raise ParseException(instring, loc, self.errmsg, self) @@ -3252,18 +3253,18 @@ class Regex(Token): roman = Regex(r"M{0,4}(CM|CD|D?{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})") """ compiledREtype = type(re.compile("[A-Z]")) - def __init__( self, pattern, flags=0, asGroupList=False, asMatch=False): + def __init__(self, pattern, flags=0, asGroupList=False, asMatch=False): """The parameters ``pattern`` and ``flags`` are passed to the ``re.compile()`` function as-is. See the Python `re module <https://docs.python.org/3/library/re.html>`_ module for an explanation of the acceptable patterns and flags. """ - super(Regex,self).__init__() + super(Regex, self).__init__() if isinstance(pattern, basestring): if not pattern: warnings.warn("null string passed to Regex; use Empty() instead", - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) self.pattern = pattern self.flags = flags @@ -3273,13 +3274,12 @@ class Regex(Token): self.reString = self.pattern except sre_constants.error: warnings.warn("invalid pattern (%s) passed to Regex" % pattern, - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) raise elif isinstance(pattern, Regex.compiledREtype): self.re = pattern - self.pattern = \ - self.reString = str(pattern) + self.pattern = self.reString = str(pattern) self.flags = flags else: @@ -3299,7 +3299,7 @@ class Regex(Token): self.parseImpl = self.parseImplAsMatch def parseImpl(self, instring, loc, doActions=True): - result = self.re_match(instring,loc) + result = self.re_match(instring, loc) if not result: raise ParseException(instring, loc, self.errmsg, self) @@ -3312,7 +3312,7 @@ class Regex(Token): return loc, ret def parseImplAsGroupList(self, instring, loc, doActions=True): - result = self.re_match(instring,loc) + result = self.re_match(instring, loc) if not result: raise ParseException(instring, loc, self.errmsg, self) @@ -3321,7 +3321,7 @@ class Regex(Token): return loc, ret def parseImplAsMatch(self, instring, loc, doActions=True): - result = self.re_match(instring,loc) + result = self.re_match(instring, loc) if not result: raise ParseException(instring, loc, self.errmsg, self) @@ -3329,9 +3329,9 @@ class Regex(Token): ret = result return loc, ret - def __str__( self ): + def __str__(self): try: - return super(Regex,self).__str__() + return super(Regex, self).__str__() except Exception: pass @@ -3353,12 +3353,12 @@ class Regex(Token): """ if self.asGroupList: warnings.warn("cannot use sub() with Regex(asGroupList=True)", - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) raise SyntaxError() if self.asMatch and callable(repl): warnings.warn("cannot use sub() with a callable with Regex(asMatch=True)", - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) raise SyntaxError() if self.asMatch: @@ -3378,20 +3378,20 @@ class QuotedString(Token): - quoteChar - string of one or more characters defining the quote delimiting string - escChar - character to escape quotes, typically backslash - (default= ``None`` ) + (default= ``None``) - escQuote - special quote sequence to escape an embedded quote string (such as SQL's ``""`` to escape an embedded ``"``) - (default= ``None`` ) + (default= ``None``) - multiline - boolean indicating whether quotes can span - multiple lines (default= ``False`` ) + multiple lines (default= ``False``) - unquoteResults - boolean indicating whether the matched text - should be unquoted (default= ``True`` ) + should be unquoted (default= ``True``) - endQuoteChar - string of one or more characters defining the end of the quote delimited string (default= ``None`` => same as quoteChar) - convertWhitespaceEscapes - convert escaped whitespace (``'\t'``, ``'\n'``, etc.) to actual whitespace - (default= ``True`` ) + (default= ``True``) Example:: @@ -3408,13 +3408,14 @@ class QuotedString(Token): [['This is the "quote"']] [['This is the quote with "embedded" quotes']] """ - def __init__( self, quoteChar, escChar=None, escQuote=None, multiline=False, unquoteResults=True, endQuoteChar=None, convertWhitespaceEscapes=True): - super(QuotedString,self).__init__() + def __init__(self, quoteChar, escChar=None, escQuote=None, multiline=False, + unquoteResults=True, endQuoteChar=None, convertWhitespaceEscapes=True): + super(QuotedString, self).__init__() # remove white space from quote chars - wont work anyway quoteChar = quoteChar.strip() if not quoteChar: - warnings.warn("quoteChar cannot be the empty string",SyntaxWarning,stacklevel=2) + warnings.warn("quoteChar cannot be the empty string", SyntaxWarning, stacklevel=2) raise SyntaxError() if endQuoteChar is None: @@ -3422,7 +3423,7 @@ class QuotedString(Token): else: endQuoteChar = endQuoteChar.strip() if not endQuoteChar: - warnings.warn("endQuoteChar cannot be the empty string",SyntaxWarning,stacklevel=2) + warnings.warn("endQuoteChar cannot be the empty string", SyntaxWarning, stacklevel=2) raise SyntaxError() self.quoteChar = quoteChar @@ -3437,36 +3438,34 @@ class QuotedString(Token): if multiline: self.flags = re.MULTILINE | re.DOTALL - self.pattern = r'%s(?:[^%s%s]' % \ - ( re.escape(self.quoteChar), - _escapeRegexRangeChars(self.endQuoteChar[0]), - (escChar is not None and _escapeRegexRangeChars(escChar) or '') ) + self.pattern = r'%s(?:[^%s%s]' % (re.escape(self.quoteChar), + _escapeRegexRangeChars(self.endQuoteChar[0]), + (escChar is not None and _escapeRegexRangeChars(escChar) or '')) else: self.flags = 0 - self.pattern = r'%s(?:[^%s\n\r%s]' % \ - ( re.escape(self.quoteChar), - _escapeRegexRangeChars(self.endQuoteChar[0]), - (escChar is not None and _escapeRegexRangeChars(escChar) or '') ) + self.pattern = r'%s(?:[^%s\n\r%s]' % (re.escape(self.quoteChar), + _escapeRegexRangeChars(self.endQuoteChar[0]), + (escChar is not None and _escapeRegexRangeChars(escChar) or '')) if len(self.endQuoteChar) > 1: self.pattern += ( '|(?:' + ')|(?:'.join("%s[^%s]" % (re.escape(self.endQuoteChar[:i]), - _escapeRegexRangeChars(self.endQuoteChar[i])) - for i in range(len(self.endQuoteChar)-1,0,-1)) + ')' - ) + _escapeRegexRangeChars(self.endQuoteChar[i])) + for i in range(len(self.endQuoteChar) - 1, 0, -1)) + ')') + if escQuote: self.pattern += (r'|(?:%s)' % re.escape(escQuote)) if escChar: self.pattern += (r'|(?:%s.)' % re.escape(escChar)) - self.escCharReplacePattern = re.escape(self.escChar)+"(.)" + self.escCharReplacePattern = re.escape(self.escChar) + "(.)" self.pattern += (r')*%s' % re.escape(self.endQuoteChar)) try: self.re = re.compile(self.pattern, self.flags) self.reString = self.pattern - self.re_match = self.re.match + self.re_match = self.re.match except sre_constants.error: warnings.warn("invalid pattern (%s) passed to Regex" % self.pattern, - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) raise self.name = _ustr(self) @@ -3474,8 +3473,8 @@ class QuotedString(Token): self.mayIndexError = False self.mayReturnEmpty = True - def parseImpl( self, instring, loc, doActions=True ): - result = instring[loc] == self.firstQuoteChar and self.re_match(instring,loc) or None + def parseImpl(self, instring, loc, doActions=True): + result = instring[loc] == self.firstQuoteChar and self.re_match(instring, loc) or None if not result: raise ParseException(instring, loc, self.errmsg, self) @@ -3485,18 +3484,18 @@ class QuotedString(Token): if self.unquoteResults: # strip off quotes - ret = ret[self.quoteCharLen:-self.endQuoteCharLen] + ret = ret[self.quoteCharLen: -self.endQuoteCharLen] - if isinstance(ret,basestring): + if isinstance(ret, basestring): # replace escaped whitespace if '\\' in ret and self.convertWhitespaceEscapes: ws_map = { - r'\t' : '\t', - r'\n' : '\n', - r'\f' : '\f', - r'\r' : '\r', + r'\t': '\t', + r'\n': '\n', + r'\f': '\f', + r'\r': '\r', } - for wslit,wschar in ws_map.items(): + for wslit, wschar in ws_map.items(): ret = ret.replace(wslit, wschar) # replace escaped characters @@ -3509,9 +3508,9 @@ class QuotedString(Token): return loc, ret - def __str__( self ): + def __str__(self): try: - return super(QuotedString,self).__str__() + return super(QuotedString, self).__str__() except Exception: pass @@ -3541,15 +3540,14 @@ class CharsNotIn(Token): ['dkls', 'lsdkjf', 's12 34', '@!#', '213'] """ - def __init__( self, notChars, min=1, max=0, exact=0 ): - super(CharsNotIn,self).__init__() + def __init__(self, notChars, min=1, max=0, exact=0): + super(CharsNotIn, self).__init__() self.skipWhitespace = False self.notChars = notChars if min < 1: - raise ValueError( - "cannot specify a minimum length < 1; use " + - "Optional(CharsNotIn()) if zero-length char group is permitted") + raise ValueError("cannot specify a minimum length < 1; use " + "Optional(CharsNotIn()) if zero-length char group is permitted") self.minLen = min @@ -3564,19 +3562,18 @@ class CharsNotIn(Token): self.name = _ustr(self) self.errmsg = "Expected " + self.name - self.mayReturnEmpty = ( self.minLen == 0 ) + self.mayReturnEmpty = (self.minLen == 0) self.mayIndexError = False - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if instring[loc] in self.notChars: raise ParseException(instring, loc, self.errmsg, self) start = loc loc += 1 notchars = self.notChars - maxlen = min( start+self.maxLen, len(instring) ) - while loc < maxlen and \ - (instring[loc] not in notchars): + maxlen = min(start + self.maxLen, len(instring)) + while loc < maxlen and instring[loc] not in notchars: loc += 1 if loc - start < self.minLen: @@ -3584,7 +3581,7 @@ class CharsNotIn(Token): return loc, instring[start:loc] - def __str__( self ): + def __str__(self): try: return super(CharsNotIn, self).__str__() except Exception: @@ -3633,10 +3630,10 @@ class White(Token): 'u\3000': '<IDEOGRAPHIC_SPACE>', } def __init__(self, ws=" \t\r\n", min=1, max=0, exact=0): - super(White,self).__init__() + super(White, self).__init__() self.matchWhite = ws - self.setWhitespaceChars( "".join(c for c in self.whiteChars if c not in self.matchWhite) ) - #~ self.leaveWhitespace() + self.setWhitespaceChars("".join(c for c in self.whiteChars if c not in self.matchWhite)) + # ~ self.leaveWhitespace() self.name = ("".join(White.whiteStrs[c] for c in self.matchWhite)) self.mayReturnEmpty = True self.errmsg = "Expected " + self.name @@ -3652,13 +3649,13 @@ class White(Token): self.maxLen = exact self.minLen = exact - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if instring[loc] not in self.matchWhite: raise ParseException(instring, loc, self.errmsg, self) start = loc loc += 1 maxloc = start + self.maxLen - maxloc = min( maxloc, len(instring) ) + maxloc = min(maxloc, len(instring)) while loc < maxloc and instring[loc] in self.matchWhite: loc += 1 @@ -3669,9 +3666,9 @@ class White(Token): class _PositionToken(Token): - def __init__( self ): - super(_PositionToken,self).__init__() - self.name=self.__class__.__name__ + def __init__(self): + super(_PositionToken, self).__init__() + self.name = self.__class__.__name__ self.mayReturnEmpty = True self.mayIndexError = False @@ -3679,25 +3676,25 @@ class GoToColumn(_PositionToken): """Token to advance to a specific column of input text; useful for tabular report scraping. """ - def __init__( self, colno ): - super(GoToColumn,self).__init__() + def __init__(self, colno): + super(GoToColumn, self).__init__() self.col = colno - def preParse( self, instring, loc ): - if col(loc,instring) != self.col: + def preParse(self, instring, loc): + if col(loc, instring) != self.col: instrlen = len(instring) if self.ignoreExprs: - loc = self._skipIgnorables( instring, loc ) - while loc < instrlen and instring[loc].isspace() and col( loc, instring ) != self.col : + loc = self._skipIgnorables(instring, loc) + while loc < instrlen and instring[loc].isspace() and col(loc, instring) != self.col: loc += 1 return loc - def parseImpl( self, instring, loc, doActions=True ): - thiscol = col( loc, instring ) + def parseImpl(self, instring, loc, doActions=True): + thiscol = col(loc, instring) if thiscol > self.col: - raise ParseException( instring, loc, "Text not in expected column", self ) + raise ParseException(instring, loc, "Text not in expected column", self) newloc = loc + self.col - thiscol - ret = instring[ loc: newloc ] + ret = instring[loc: newloc] return newloc, ret @@ -3723,11 +3720,11 @@ class LineStart(_PositionToken): ['AAA', ' and this line'] """ - def __init__( self ): - super(LineStart,self).__init__() + def __init__(self): + super(LineStart, self).__init__() self.errmsg = "Expected start of line" - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if col(loc, instring) == 1: return loc, [] raise ParseException(instring, loc, self.errmsg, self) @@ -3736,19 +3733,19 @@ class LineEnd(_PositionToken): """Matches if current position is at the end of a line within the parse string """ - def __init__( self ): - super(LineEnd,self).__init__() - self.setWhitespaceChars( ParserElement.DEFAULT_WHITE_CHARS.replace("\n","") ) + def __init__(self): + super(LineEnd, self).__init__() + self.setWhitespaceChars(ParserElement.DEFAULT_WHITE_CHARS.replace("\n", "")) self.errmsg = "Expected end of line" - def parseImpl( self, instring, loc, doActions=True ): - if loc<len(instring): + def parseImpl(self, instring, loc, doActions=True): + if loc < len(instring): if instring[loc] == "\n": - return loc+1, "\n" + return loc + 1, "\n" else: raise ParseException(instring, loc, self.errmsg, self) elif loc == len(instring): - return loc+1, [] + return loc + 1, [] else: raise ParseException(instring, loc, self.errmsg, self) @@ -3756,29 +3753,29 @@ class StringStart(_PositionToken): """Matches if current position is at the beginning of the parse string """ - def __init__( self ): - super(StringStart,self).__init__() + def __init__(self): + super(StringStart, self).__init__() self.errmsg = "Expected start of text" - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if loc != 0: # see if entire string up to here is just whitespace and ignoreables - if loc != self.preParse( instring, 0 ): + if loc != self.preParse(instring, 0): raise ParseException(instring, loc, self.errmsg, self) return loc, [] class StringEnd(_PositionToken): """Matches if current position is at the end of the parse string """ - def __init__( self ): - super(StringEnd,self).__init__() + def __init__(self): + super(StringEnd, self).__init__() self.errmsg = "Expected end of text" - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if loc < len(instring): raise ParseException(instring, loc, self.errmsg, self) elif loc == len(instring): - return loc+1, [] + return loc + 1, [] elif loc > len(instring): return loc, [] else: @@ -3793,15 +3790,15 @@ class WordStart(_PositionToken): the beginning of the string being parsed, or at the beginning of a line. """ - def __init__(self, wordChars = printables): - super(WordStart,self).__init__() + def __init__(self, wordChars=printables): + super(WordStart, self).__init__() self.wordChars = set(wordChars) self.errmsg = "Not at the start of a word" - def parseImpl(self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if loc != 0: - if (instring[loc-1] in self.wordChars or - instring[loc] not in self.wordChars): + if (instring[loc - 1] in self.wordChars + or instring[loc] not in self.wordChars): raise ParseException(instring, loc, self.errmsg, self) return loc, [] @@ -3813,17 +3810,17 @@ class WordEnd(_PositionToken): will also match at the end of the string being parsed, or at the end of a line. """ - def __init__(self, wordChars = printables): - super(WordEnd,self).__init__() + def __init__(self, wordChars=printables): + super(WordEnd, self).__init__() self.wordChars = set(wordChars) self.skipWhitespace = False self.errmsg = "Not at the end of a word" - def parseImpl(self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): instrlen = len(instring) - if instrlen>0 and loc<instrlen: + if instrlen > 0 and loc < instrlen: if (instring[loc] in self.wordChars or - instring[loc-1] not in self.wordChars): + instring[loc - 1] not in self.wordChars): raise ParseException(instring, loc, self.errmsg, self) return loc, [] @@ -3832,87 +3829,87 @@ class ParseExpression(ParserElement): """Abstract subclass of ParserElement, for combining and post-processing parsed tokens. """ - def __init__( self, exprs, savelist = False ): - super(ParseExpression,self).__init__(savelist) - if isinstance( exprs, _generatorType ): + def __init__(self, exprs, savelist=False): + super(ParseExpression, self).__init__(savelist) + if isinstance(exprs, _generatorType): exprs = list(exprs) - if isinstance( exprs, basestring ): - self.exprs = [ ParserElement._literalStringClass( exprs ) ] - elif isinstance( exprs, Iterable ): + if isinstance(exprs, basestring): + self.exprs = [self._literalStringClass(exprs)] + elif isinstance(exprs, Iterable): exprs = list(exprs) # if sequence of strings provided, wrap with Literal if any(isinstance(expr, basestring) for expr in exprs): - exprs = (ParserElement._literalStringClass(e) if isinstance(e, basestring) else e for e in exprs) + exprs = (self._literalStringClass(e) if isinstance(e, basestring) else e for e in exprs) self.exprs = list(exprs) else: try: - self.exprs = list( exprs ) + self.exprs = list(exprs) except TypeError: - self.exprs = [ exprs ] + self.exprs = [exprs] self.callPreparse = False - def append( self, other ): - self.exprs.append( other ) + def append(self, other): + self.exprs.append(other) self.strRepr = None return self - def leaveWhitespace( self ): + def leaveWhitespace(self): """Extends ``leaveWhitespace`` defined in base class, and also invokes ``leaveWhitespace`` on all contained expressions.""" self.skipWhitespace = False - self.exprs = [ e.copy() for e in self.exprs ] + self.exprs = [e.copy() for e in self.exprs] for e in self.exprs: e.leaveWhitespace() return self - def ignore( self, other ): - if isinstance( other, Suppress ): + def ignore(self, other): + if isinstance(other, Suppress): if other not in self.ignoreExprs: - super( ParseExpression, self).ignore( other ) + super(ParseExpression, self).ignore(other) for e in self.exprs: - e.ignore( self.ignoreExprs[-1] ) + e.ignore(self.ignoreExprs[-1]) else: - super( ParseExpression, self).ignore( other ) + super(ParseExpression, self).ignore(other) for e in self.exprs: - e.ignore( self.ignoreExprs[-1] ) + e.ignore(self.ignoreExprs[-1]) return self - def __str__( self ): + def __str__(self): try: - return super(ParseExpression,self).__str__() + return super(ParseExpression, self).__str__() except Exception: pass if self.strRepr is None: - self.strRepr = "%s:(%s)" % ( self.__class__.__name__, _ustr(self.exprs) ) + self.strRepr = "%s:(%s)" % (self.__class__.__name__, _ustr(self.exprs)) return self.strRepr - def streamline( self ): - super(ParseExpression,self).streamline() + def streamline(self): + super(ParseExpression, self).streamline() for e in self.exprs: e.streamline() - # collapse nested And's of the form And( And( And( a,b), c), d) to And( a,b,c,d ) + # collapse nested And's of the form And(And(And(a, b), c), d) to And(a, b, c, d) # but only if there are no parse actions or resultsNames on the nested And's # (likewise for Or's and MatchFirst's) - if ( len(self.exprs) == 2 ): + if len(self.exprs) == 2: other = self.exprs[0] - if ( isinstance( other, self.__class__ ) and - not(other.parseAction) and - other.resultsName is None and - not other.debug ): - self.exprs = other.exprs[:] + [ self.exprs[1] ] + if (isinstance(other, self.__class__) + and not other.parseAction + and other.resultsName is None + and not other.debug): + self.exprs = other.exprs[:] + [self.exprs[1]] self.strRepr = None self.mayReturnEmpty |= other.mayReturnEmpty self.mayIndexError |= other.mayIndexError other = self.exprs[-1] - if ( isinstance( other, self.__class__ ) and - not(other.parseAction) and - other.resultsName is None and - not other.debug ): + if (isinstance(other, self.__class__) + and not other.parseAction + and other.resultsName is None + and not other.debug): self.exprs = self.exprs[:-1] + other.exprs[:] self.strRepr = None self.mayReturnEmpty |= other.mayReturnEmpty @@ -3923,13 +3920,13 @@ class ParseExpression(ParserElement): return self def validate(self, validateTrace=None): - tmp = (validateTrace if validateTrace is not None else [])[:]+[self] + tmp = (validateTrace if validateTrace is not None else [])[:] + [self] for e in self.exprs: e.validate(tmp) self.checkRecursion([]) def copy(self): - ret = super(ParseExpression,self).copy() + ret = super(ParseExpression, self).copy() ret.exprs = [e.copy() for e in self.exprs] return ret @@ -3960,33 +3957,33 @@ class And(ParseExpression): integer = Word(nums) name_expr = OneOrMore(Word(alphas)) - expr = And([integer("id"),name_expr("name"),integer("age")]) + expr = And([integer("id"), name_expr("name"), integer("age")]) # more easily written as: expr = integer("id") + name_expr("name") + integer("age") """ class _ErrorStop(Empty): def __init__(self, *args, **kwargs): - super(And._ErrorStop,self).__init__(*args, **kwargs) + super(And._ErrorStop, self).__init__(*args, **kwargs) self.name = '-' self.leaveWhitespace() - def __init__( self, exprs, savelist = True ): + def __init__(self, exprs, savelist=True): if exprs and Ellipsis in exprs: tmp = [] for i, expr in enumerate(exprs): if expr is Ellipsis: - if i < len(exprs)-1: - skipto_arg = (Empty() + exprs[i+1]).exprs[-1] + if i < len(exprs) - 1: + skipto_arg = (Empty() + exprs[i + 1]).exprs[-1] tmp.append(SkipTo(skipto_arg)("_skipped*")) else: raise Exception("cannot construct And with sequence ending in ...") else: tmp.append(expr) exprs[:] = tmp - super(And,self).__init__(exprs, savelist) + super(And, self).__init__(exprs, savelist) self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs) - self.setWhitespaceChars( self.exprs[0].whiteChars ) + self.setWhitespaceChars(self.exprs[0].whiteChars) self.skipWhitespace = self.exprs[0].skipWhitespace self.callPreparse = True @@ -3996,19 +3993,20 @@ class And(ParseExpression): for i, e in enumerate(self.exprs[:-1]): if e is None: continue - if (isinstance(e, ParseExpression) and isinstance(e.exprs[-1], _PendingSkip)): - e.exprs[-1] = e.exprs[-1] + self.exprs[i+1] - self.exprs[i+1] = None + if (isinstance(e, ParseExpression) + and isinstance(e.exprs[-1], _PendingSkip)): + e.exprs[-1] = e.exprs[-1] + self.exprs[i + 1] + self.exprs[i + 1] = None self.exprs = [e for e in self.exprs if e is not None] super(And, self).streamline() self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs) return self - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): # pass False as last arg to _parse for first element, since we already # pre-parsed the string as part of our And pre-parsing - loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False ) + loc, resultlist = self.exprs[0]._parse(instring, loc, doActions, callPreParse=False) errorStop = False for e in self.exprs[1:]: if isinstance(e, And._ErrorStop): @@ -4016,7 +4014,7 @@ class And(ParseExpression): continue if errorStop: try: - loc, exprtokens = e._parse( instring, loc, doActions ) + loc, exprtokens = e._parse(instring, loc, doActions) except ParseSyntaxException: raise except ParseBaseException as pe: @@ -4025,25 +4023,25 @@ class And(ParseExpression): except IndexError: raise ParseSyntaxException(instring, len(instring), self.errmsg, self) else: - loc, exprtokens = e._parse( instring, loc, doActions ) + loc, exprtokens = e._parse(instring, loc, doActions) if exprtokens or exprtokens.haskeys(): resultlist += exprtokens return loc, resultlist - def __iadd__(self, other ): - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - return self.append( other ) #And( [ self, other ] ) + def __iadd__(self, other): + if isinstance(other, basestring): + other = self._literalStringClass(other) + return self.append(other) # And([self, other]) - def checkRecursion( self, parseElementList ): - subRecCheckList = parseElementList[:] + [ self ] + def checkRecursion(self, parseElementList): + subRecCheckList = parseElementList[:] + [self] for e in self.exprs: - e.checkRecursion( subRecCheckList ) + e.checkRecursion(subRecCheckList) if not e.mayReturnEmpty: break - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4069,8 +4067,8 @@ class Or(ParseExpression): [['123'], ['3.1416'], ['789']] """ - def __init__( self, exprs, savelist = False ): - super(Or,self).__init__(exprs, savelist) + def __init__(self, exprs, savelist=False): + super(Or, self).__init__(exprs, savelist) if self.exprs: self.mayReturnEmpty = any(e.mayReturnEmpty for e in self.exprs) else: @@ -4082,13 +4080,13 @@ class Or(ParseExpression): self.saveAsList = any(e.saveAsList for e in self.exprs) return self - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): maxExcLoc = -1 maxException = None matches = [] for e in self.exprs: try: - loc2 = e.tryParse( instring, loc ) + loc2 = e.tryParse(instring, loc) except ParseException as err: err.__traceback__ = None if err.loc > maxExcLoc: @@ -4096,7 +4094,7 @@ class Or(ParseExpression): maxExcLoc = err.loc except IndexError: if len(instring) > maxExcLoc: - maxException = ParseException(instring,len(instring),e.errmsg,self) + maxException = ParseException(instring, len(instring), e.errmsg, self) maxExcLoc = len(instring) else: # save match among all matches, to retry longest to shortest @@ -4137,13 +4135,13 @@ class Or(ParseExpression): raise ParseException(instring, loc, "no defined alternatives to match", self) - def __ixor__(self, other ): - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - return self.append( other ) #Or( [ self, other ] ) + def __ixor__(self, other): + if isinstance(other, basestring): + other = self._literalStringClass(other) + return self.append(other) # Or([self, other]) - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4151,10 +4149,10 @@ class Or(ParseExpression): return self.strRepr - def checkRecursion( self, parseElementList ): - subRecCheckList = parseElementList[:] + [ self ] + def checkRecursion(self, parseElementList): + subRecCheckList = parseElementList[:] + [self] for e in self.exprs: - e.checkRecursion( subRecCheckList ) + e.checkRecursion(subRecCheckList) def _setResultsName(self, name, listAllMatches=False): if (not __compat__.collect_all_And_tokens @@ -4186,8 +4184,8 @@ class MatchFirst(ParseExpression): number = Combine(Word(nums) + '.' + Word(nums)) | Word(nums) print(number.searchString("123 3.1416 789")) # Better -> [['123'], ['3.1416'], ['789']] """ - def __init__( self, exprs, savelist = False ): - super(MatchFirst,self).__init__(exprs, savelist) + def __init__(self, exprs, savelist=False): + super(MatchFirst, self).__init__(exprs, savelist) if self.exprs: self.mayReturnEmpty = any(e.mayReturnEmpty for e in self.exprs) else: @@ -4199,12 +4197,12 @@ class MatchFirst(ParseExpression): self.saveAsList = any(e.saveAsList for e in self.exprs) return self - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): maxExcLoc = -1 maxException = None for e in self.exprs: try: - ret = e._parse( instring, loc, doActions ) + ret = e._parse(instring, loc, doActions) return ret except ParseException as err: if err.loc > maxExcLoc: @@ -4212,7 +4210,7 @@ class MatchFirst(ParseExpression): maxExcLoc = err.loc except IndexError: if len(instring) > maxExcLoc: - maxException = ParseException(instring,len(instring),e.errmsg,self) + maxException = ParseException(instring, len(instring), e.errmsg, self) maxExcLoc = len(instring) # only got here if no expression matched, raise exception for match that made it the furthest @@ -4223,13 +4221,13 @@ class MatchFirst(ParseExpression): else: raise ParseException(instring, loc, "no defined alternatives to match", self) - def __ior__(self, other ): - if isinstance( other, basestring ): - other = ParserElement._literalStringClass( other ) - return self.append( other ) #MatchFirst( [ self, other ] ) + def __ior__(self, other): + if isinstance(other, basestring): + other = self._literalStringClass(other) + return self.append(other) # MatchFirst([self, other]) - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4237,10 +4235,10 @@ class MatchFirst(ParseExpression): return self.strRepr - def checkRecursion( self, parseElementList ): - subRecCheckList = parseElementList[:] + [ self ] + def checkRecursion(self, parseElementList): + subRecCheckList = parseElementList[:] + [self] for e in self.exprs: - e.checkRecursion( subRecCheckList ) + e.checkRecursion(subRecCheckList) def _setResultsName(self, name, listAllMatches=False): if (not __compat__.collect_all_And_tokens @@ -4312,8 +4310,8 @@ class Each(ParseExpression): - shape: TRIANGLE - size: 20 """ - def __init__( self, exprs, savelist = True ): - super(Each,self).__init__(exprs, savelist) + def __init__(self, exprs, savelist=True): + super(Each, self).__init__(exprs, savelist) self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs) self.skipWhitespace = True self.initExprGroups = True @@ -4324,15 +4322,15 @@ class Each(ParseExpression): self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs) return self - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if self.initExprGroups: - self.opt1map = dict((id(e.expr),e) for e in self.exprs if isinstance(e,Optional)) - opt1 = [ e.expr for e in self.exprs if isinstance(e,Optional) ] - opt2 = [ e for e in self.exprs if e.mayReturnEmpty and not isinstance(e,Optional)] + self.opt1map = dict((id(e.expr), e) for e in self.exprs if isinstance(e, Optional)) + opt1 = [e.expr for e in self.exprs if isinstance(e, Optional)] + opt2 = [e for e in self.exprs if e.mayReturnEmpty and not isinstance(e, Optional)] self.optionals = opt1 + opt2 - self.multioptionals = [ e.expr for e in self.exprs if isinstance(e,ZeroOrMore) ] - self.multirequired = [ e.expr for e in self.exprs if isinstance(e,OneOrMore) ] - self.required = [ e for e in self.exprs if not isinstance(e,(Optional,ZeroOrMore,OneOrMore)) ] + self.multioptionals = [e.expr for e in self.exprs if isinstance(e, ZeroOrMore)] + self.multirequired = [e.expr for e in self.exprs if isinstance(e, OneOrMore)] + self.required = [e for e in self.exprs if not isinstance(e, (Optional, ZeroOrMore, OneOrMore))] self.required += self.multirequired self.initExprGroups = False tmpLoc = loc @@ -4346,11 +4344,11 @@ class Each(ParseExpression): failed = [] for e in tmpExprs: try: - tmpLoc = e.tryParse( instring, tmpLoc ) + tmpLoc = e.tryParse(instring, tmpLoc) except ParseException: failed.append(e) else: - matchOrder.append(self.opt1map.get(id(e),e)) + matchOrder.append(self.opt1map.get(id(e), e)) if e in tmpReqd: tmpReqd.remove(e) elif e in tmpOpt: @@ -4360,21 +4358,21 @@ class Each(ParseExpression): if tmpReqd: missing = ", ".join(_ustr(e) for e in tmpReqd) - raise ParseException(instring,loc,"Missing one or more required elements (%s)" % missing ) + raise ParseException(instring, loc, "Missing one or more required elements (%s)" % missing) # add any unmatched Optionals, in case they have default values defined - matchOrder += [e for e in self.exprs if isinstance(e,Optional) and e.expr in tmpOpt] + matchOrder += [e for e in self.exprs if isinstance(e, Optional) and e.expr in tmpOpt] resultlist = [] for e in matchOrder: - loc,results = e._parse(instring,loc,doActions) + loc, results = e._parse(instring, loc, doActions) resultlist.append(results) finalResults = sum(resultlist, ParseResults([])) return loc, finalResults - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4382,86 +4380,88 @@ class Each(ParseExpression): return self.strRepr - def checkRecursion( self, parseElementList ): - subRecCheckList = parseElementList[:] + [ self ] + def checkRecursion(self, parseElementList): + subRecCheckList = parseElementList[:] + [self] for e in self.exprs: - e.checkRecursion( subRecCheckList ) + e.checkRecursion(subRecCheckList) class ParseElementEnhance(ParserElement): """Abstract subclass of :class:`ParserElement`, for combining and post-processing parsed tokens. """ - def __init__( self, expr, savelist=False ): - super(ParseElementEnhance,self).__init__(savelist) - if isinstance( expr, basestring ): - if issubclass(ParserElement._literalStringClass, Token): - expr = ParserElement._literalStringClass(expr) + def __init__(self, expr, savelist=False): + super(ParseElementEnhance, self).__init__(savelist) + if isinstance(expr, basestring): + if issubclass(self._literalStringClass, Token): + expr = self._literalStringClass(expr) else: - expr = ParserElement._literalStringClass(Literal(expr)) + expr = self._literalStringClass(Literal(expr)) self.expr = expr self.strRepr = None if expr is not None: self.mayIndexError = expr.mayIndexError self.mayReturnEmpty = expr.mayReturnEmpty - self.setWhitespaceChars( expr.whiteChars ) + self.setWhitespaceChars(expr.whiteChars) self.skipWhitespace = expr.skipWhitespace self.saveAsList = expr.saveAsList self.callPreparse = expr.callPreparse self.ignoreExprs.extend(expr.ignoreExprs) - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if self.expr is not None: - return self.expr._parse( instring, loc, doActions, callPreParse=False ) + return self.expr._parse(instring, loc, doActions, callPreParse=False) else: - raise ParseException("",loc,self.errmsg,self) + raise ParseException("", loc, self.errmsg, self) - def leaveWhitespace( self ): + def leaveWhitespace(self): self.skipWhitespace = False self.expr = self.expr.copy() if self.expr is not None: self.expr.leaveWhitespace() return self - def ignore( self, other ): - if isinstance( other, Suppress ): + def ignore(self, other): + if isinstance(other, Suppress): if other not in self.ignoreExprs: - super( ParseElementEnhance, self).ignore( other ) + super(ParseElementEnhance, self).ignore(other) if self.expr is not None: - self.expr.ignore( self.ignoreExprs[-1] ) + self.expr.ignore(self.ignoreExprs[-1]) else: - super( ParseElementEnhance, self).ignore( other ) + super(ParseElementEnhance, self).ignore(other) if self.expr is not None: - self.expr.ignore( self.ignoreExprs[-1] ) + self.expr.ignore(self.ignoreExprs[-1]) return self - def streamline( self ): - super(ParseElementEnhance,self).streamline() + def streamline(self): + super(ParseElementEnhance, self).streamline() if self.expr is not None: self.expr.streamline() return self - def checkRecursion( self, parseElementList ): + def checkRecursion(self, parseElementList): if self in parseElementList: - raise RecursiveGrammarException( parseElementList+[self] ) - subRecCheckList = parseElementList[:] + [ self ] + raise RecursiveGrammarException(parseElementList + [self]) + subRecCheckList = parseElementList[:] + [self] if self.expr is not None: - self.expr.checkRecursion( subRecCheckList ) + self.expr.checkRecursion(subRecCheckList) - def validate( self, validateTrace=[] ): - tmp = validateTrace[:]+[self] + def validate(self, validateTrace=None): + if validateTrace is None: + validateTrace = [] + tmp = validateTrace[:] + [self] if self.expr is not None: self.expr.validate(tmp) - self.checkRecursion( [] ) + self.checkRecursion([]) - def __str__( self ): + def __str__(self): try: - return super(ParseElementEnhance,self).__str__() + return super(ParseElementEnhance, self).__str__() except Exception: pass if self.strRepr is None and self.expr is not None: - self.strRepr = "%s:(%s)" % ( self.__class__.__name__, _ustr(self.expr) ) + self.strRepr = "%s:(%s)" % (self.__class__.__name__, _ustr(self.expr)) return self.strRepr @@ -4487,11 +4487,11 @@ class FollowedBy(ParseElementEnhance): [['shape', 'SQUARE'], ['color', 'BLACK'], ['posn', 'upper left']] """ - def __init__( self, expr ): - super(FollowedBy,self).__init__(expr) + def __init__(self, expr): + super(FollowedBy, self).__init__(expr) self.mayReturnEmpty = True - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): # by using self._expr.parse and deleting the contents of the returned ParseResults list # we keep any named results that were defined in the FollowedBy expression _, ret = self.expr._parse(instring, loc, doActions=doActions) @@ -4561,9 +4561,9 @@ class PrecededBy(ParseElementEnhance): test_expr = self.expr + StringEnd() instring_slice = instring[:loc] last_expr = ParseException(instring, loc, self.errmsg) - for offset in range(1, min(loc, self.retreat+1)): + for offset in range(1, min(loc, self.retreat + 1)): try: - _, ret = test_expr._parse(instring_slice, loc-offset) + _, ret = test_expr._parse(instring_slice, loc - offset) except ParseBaseException as pbe: last_expr = pbe else: @@ -4598,20 +4598,20 @@ class NotAny(ParseElementEnhance): # integers that are followed by "." are actually floats integer = Word(nums) + ~Char(".") """ - def __init__( self, expr ): - super(NotAny,self).__init__(expr) - #~ self.leaveWhitespace() + def __init__(self, expr): + super(NotAny, self).__init__(expr) + # ~ self.leaveWhitespace() self.skipWhitespace = False # do NOT use self.leaveWhitespace(), don't want to propagate to exprs self.mayReturnEmpty = True - self.errmsg = "Found unwanted token, "+_ustr(self.expr) + self.errmsg = "Found unwanted token, " + _ustr(self.expr) - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): if self.expr.canParseNext(instring, loc): raise ParseException(instring, loc, self.errmsg, self) return loc, [] - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4620,21 +4620,21 @@ class NotAny(ParseElementEnhance): return self.strRepr class _MultipleMatch(ParseElementEnhance): - def __init__( self, expr, stopOn=None): + def __init__(self, expr, stopOn=None): super(_MultipleMatch, self).__init__(expr) self.saveAsList = True ender = stopOn if isinstance(ender, basestring): - ender = ParserElement._literalStringClass(ender) + ender = self._literalStringClass(ender) self.stopOn(ender) def stopOn(self, ender): if isinstance(ender, basestring): - ender = ParserElement._literalStringClass(ender) + ender = self._literalStringClass(ender) self.not_ender = ~ender if ender is not None else None return self - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): self_expr_parse = self.expr._parse self_skip_ignorables = self._skipIgnorables check_ender = self.not_ender is not None @@ -4645,20 +4645,20 @@ class _MultipleMatch(ParseElementEnhance): # if so, fail) if check_ender: try_not_ender(instring, loc) - loc, tokens = self_expr_parse( instring, loc, doActions, callPreParse=False ) + loc, tokens = self_expr_parse(instring, loc, doActions, callPreParse=False) try: hasIgnoreExprs = (not not self.ignoreExprs) while 1: if check_ender: try_not_ender(instring, loc) if hasIgnoreExprs: - preloc = self_skip_ignorables( instring, loc ) + preloc = self_skip_ignorables(instring, loc) else: preloc = loc - loc, tmptokens = self_expr_parse( instring, preloc, doActions ) + loc, tmptokens = self_expr_parse(instring, preloc, doActions) if tmptokens or tmptokens.haskeys(): tokens += tmptokens - except (ParseException,IndexError): + except (ParseException, IndexError): pass return loc, tokens @@ -4703,8 +4703,8 @@ class OneOrMore(_MultipleMatch): (attr_expr * (1,)).parseString(text).pprint() """ - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4723,18 +4723,18 @@ class ZeroOrMore(_MultipleMatch): Example: similar to :class:`OneOrMore` """ - def __init__( self, expr, stopOn=None): - super(ZeroOrMore,self).__init__(expr, stopOn=stopOn) + def __init__(self, expr, stopOn=None): + super(ZeroOrMore, self).__init__(expr, stopOn=stopOn) self.mayReturnEmpty = True - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): try: return super(ZeroOrMore, self).parseImpl(instring, loc, doActions) - except (ParseException,IndexError): + except (ParseException, IndexError): return loc, [] - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4789,28 +4789,28 @@ class Optional(ParseElementEnhance): """ __optionalNotMatched = _NullToken() - def __init__( self, expr, default=__optionalNotMatched ): - super(Optional,self).__init__( expr, savelist=False ) + def __init__(self, expr, default=__optionalNotMatched): + super(Optional, self).__init__(expr, savelist=False) self.saveAsList = self.expr.saveAsList self.defaultValue = default self.mayReturnEmpty = True - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): try: - loc, tokens = self.expr._parse( instring, loc, doActions, callPreParse=False ) - except (ParseException,IndexError): + loc, tokens = self.expr._parse(instring, loc, doActions, callPreParse=False) + except (ParseException, IndexError): if self.defaultValue is not self.__optionalNotMatched: if self.expr.resultsName: - tokens = ParseResults([ self.defaultValue ]) + tokens = ParseResults([self.defaultValue]) tokens[self.expr.resultsName] = self.defaultValue else: - tokens = [ self.defaultValue ] + tokens = [self.defaultValue] else: tokens = [] return loc, tokens - def __str__( self ): - if hasattr(self,"name"): + def __str__(self): + if hasattr(self, "name"): return self.name if self.strRepr is None: @@ -4876,20 +4876,20 @@ class SkipTo(ParseElementEnhance): - issue_num: 79 - sev: Minor """ - def __init__( self, other, include=False, ignore=None, failOn=None ): - super( SkipTo, self ).__init__( other ) + def __init__(self, other, include=False, ignore=None, failOn=None): + super(SkipTo, self).__init__(other) self.ignoreExpr = ignore self.mayReturnEmpty = True self.mayIndexError = False self.includeMatch = include self.saveAsList = False if isinstance(failOn, basestring): - self.failOn = ParserElement._literalStringClass(failOn) + self.failOn = self._literalStringClass(failOn) else: self.failOn = failOn - self.errmsg = "No match found for "+_ustr(self.expr) + self.errmsg = "No match found for " + _ustr(self.expr) - def parseImpl( self, instring, loc, doActions=True ): + def parseImpl(self, instring, loc, doActions=True): startloc = loc instrlen = len(instring) expr = self.expr @@ -4931,7 +4931,7 @@ class SkipTo(ParseElementEnhance): skipresult = ParseResults(skiptext) if self.includeMatch: - loc, mat = expr_parse(instring,loc,doActions,callPreParse=False) + loc, mat = expr_parse(instring, loc, doActions, callPreParse=False) skipresult += mat return loc, skipresult @@ -4963,17 +4963,17 @@ class Forward(ParseElementEnhance): See :class:`ParseResults.pprint` for an example of a recursive parser created using ``Forward``. """ - def __init__( self, other=None ): - super(Forward,self).__init__( other, savelist=False ) + def __init__(self, other=None): + super(Forward, self).__init__(other, savelist=False) - def __lshift__( self, other ): - if isinstance( other, basestring ): - other = ParserElement._literalStringClass(other) + def __lshift__(self, other): + if isinstance(other, basestring): + other = self._literalStringClass(other) self.expr = other self.strRepr = None self.mayIndexError = self.expr.mayIndexError self.mayReturnEmpty = self.expr.mayReturnEmpty - self.setWhitespaceChars( self.expr.whiteChars ) + self.setWhitespaceChars(self.expr.whiteChars) self.skipWhitespace = self.expr.skipWhitespace self.saveAsList = self.expr.saveAsList self.ignoreExprs.extend(self.expr.ignoreExprs) @@ -4982,25 +4982,28 @@ class Forward(ParseElementEnhance): def __ilshift__(self, other): return self << other - def leaveWhitespace( self ): + def leaveWhitespace(self): self.skipWhitespace = False return self - def streamline( self ): + def streamline(self): if not self.streamlined: self.streamlined = True if self.expr is not None: self.expr.streamline() return self - def validate( self, validateTrace=[] ): + def validate(self, validateTrace=None): + if validateTrace is None: + validateTrace = [] + if self not in validateTrace: - tmp = validateTrace[:]+[self] + tmp = validateTrace[:] + [self] if self.expr is not None: self.expr.validate(tmp) self.checkRecursion([]) - def __str__( self ): + def __str__(self): if hasattr(self, "name"): return self.name if self.strRepr is not None: @@ -5022,7 +5025,7 @@ class Forward(ParseElementEnhance): def copy(self): if self.expr is not None: - return super(Forward,self).copy() + return super(Forward, self).copy() else: ret = Forward() ret <<= self @@ -5043,8 +5046,8 @@ class TokenConverter(ParseElementEnhance): """ Abstract subclass of :class:`ParseExpression`, for converting parsed results. """ - def __init__( self, expr, savelist=False ): - super(TokenConverter,self).__init__( expr )#, savelist ) + def __init__(self, expr, savelist=False): + super(TokenConverter, self).__init__(expr) # , savelist) self.saveAsList = False class Combine(TokenConverter): @@ -5065,8 +5068,8 @@ class Combine(TokenConverter): # no match when there are internal spaces print(real.parseString('3. 1416')) # -> Exception: Expected W:(0123...) """ - def __init__( self, expr, joinString="", adjacent=True ): - super(Combine,self).__init__( expr ) + def __init__(self, expr, joinString="", adjacent=True): + super(Combine, self).__init__(expr) # suppress whitespace-stripping in contained parse expressions, but re-enable it on the Combine itself if adjacent: self.leaveWhitespace() @@ -5075,20 +5078,20 @@ class Combine(TokenConverter): self.joinString = joinString self.callPreparse = True - def ignore( self, other ): + def ignore(self, other): if self.adjacent: ParserElement.ignore(self, other) else: - super( Combine, self).ignore( other ) + super(Combine, self).ignore(other) return self - def postParse( self, instring, loc, tokenlist ): + def postParse(self, instring, loc, tokenlist): retToks = tokenlist.copy() del retToks[:] - retToks += ParseResults([ "".join(tokenlist._asStringList(self.joinString)) ], modal=self.modalResults) + retToks += ParseResults(["".join(tokenlist._asStringList(self.joinString))], modal=self.modalResults) if self.resultsName and retToks.haskeys(): - return [ retToks ] + return [retToks] else: return retToks @@ -5102,17 +5105,17 @@ class Group(TokenConverter): num = Word(nums) term = ident | num func = ident + Optional(delimitedList(term)) - print(func.parseString("fn a,b,100")) # -> ['fn', 'a', 'b', '100'] + print(func.parseString("fn a, b, 100")) # -> ['fn', 'a', 'b', '100'] func = ident + Group(Optional(delimitedList(term))) - print(func.parseString("fn a,b,100")) # -> ['fn', ['a', 'b', '100']] + print(func.parseString("fn a, b, 100")) # -> ['fn', ['a', 'b', '100']] """ - def __init__( self, expr ): - super(Group,self).__init__( expr ) + def __init__(self, expr): + super(Group, self).__init__(expr) self.saveAsList = True - def postParse( self, instring, loc, tokenlist ): - return [ tokenlist ] + def postParse(self, instring, loc, tokenlist): + return [tokenlist] class Dict(TokenConverter): """Converter to return a repetitive expression as a list, but also @@ -5153,31 +5156,31 @@ class Dict(TokenConverter): See more examples at :class:`ParseResults` of accessing fields by results name. """ - def __init__( self, expr ): - super(Dict,self).__init__( expr ) + def __init__(self, expr): + super(Dict, self).__init__(expr) self.saveAsList = True - def postParse( self, instring, loc, tokenlist ): - for i,tok in enumerate(tokenlist): + def postParse(self, instring, loc, tokenlist): + for i, tok in enumerate(tokenlist): if len(tok) == 0: continue ikey = tok[0] - if isinstance(ikey,int): + if isinstance(ikey, int): ikey = _ustr(tok[0]).strip() - if len(tok)==1: - tokenlist[ikey] = _ParseResultsWithOffset("",i) - elif len(tok)==2 and not isinstance(tok[1],ParseResults): - tokenlist[ikey] = _ParseResultsWithOffset(tok[1],i) + if len(tok) == 1: + tokenlist[ikey] = _ParseResultsWithOffset("", i) + elif len(tok) == 2 and not isinstance(tok[1], ParseResults): + tokenlist[ikey] = _ParseResultsWithOffset(tok[1], i) else: - dictvalue = tok.copy() #ParseResults(i) + dictvalue = tok.copy() # ParseResults(i) del dictvalue[0] - if len(dictvalue)!= 1 or (isinstance(dictvalue,ParseResults) and dictvalue.haskeys()): - tokenlist[ikey] = _ParseResultsWithOffset(dictvalue,i) + if len(dictvalue) != 1 or (isinstance(dictvalue, ParseResults) and dictvalue.haskeys()): + tokenlist[ikey] = _ParseResultsWithOffset(dictvalue, i) else: - tokenlist[ikey] = _ParseResultsWithOffset(dictvalue[0],i) + tokenlist[ikey] = _ParseResultsWithOffset(dictvalue[0], i) if self.resultsName: - return [ tokenlist ] + return [tokenlist] else: return tokenlist @@ -5204,10 +5207,10 @@ class Suppress(TokenConverter): (See also :class:`delimitedList`.) """ - def postParse( self, instring, loc, tokenlist ): + def postParse(self, instring, loc, tokenlist): return [] - def suppress( self ): + def suppress(self): return self @@ -5217,12 +5220,12 @@ class OnlyOnce(object): def __init__(self, methodCall): self.callable = _trim_arity(methodCall) self.called = False - def __call__(self,s,l,t): + def __call__(self, s, l, t): if not self.called: - results = self.callable(s,l,t) + results = self.callable(s, l, t) self.called = True return results - raise ParseException(s,l,"") + raise ParseException(s, l, "") def reset(self): self.called = False @@ -5254,16 +5257,16 @@ def traceParseAction(f): f = _trim_arity(f) def z(*paArgs): thisFunc = f.__name__ - s,l,t = paArgs[-3:] - if len(paArgs)>3: + s, l, t = paArgs[-3:] + if len(paArgs) > 3: thisFunc = paArgs[0].__class__.__name__ + '.' + thisFunc - sys.stderr.write( ">>entering %s(line: '%s', %d, %r)\n" % (thisFunc,line(l,s),l,t) ) + sys.stderr.write(">>entering %s(line: '%s', %d, %r)\n" % (thisFunc, line(l, s), l, t)) try: ret = f(*paArgs) except Exception as exc: - sys.stderr.write( "<<leaving %s (exception: %s)\n" % (thisFunc,exc) ) + sys.stderr.write("<<leaving %s (exception: %s)\n" % (thisFunc, exc)) raise - sys.stderr.write( "<<leaving %s (ret: %r)\n" % (thisFunc,ret) ) + sys.stderr.write("<<leaving %s (ret: %r)\n" % (thisFunc, ret)) return ret try: z.__name__ = f.__name__ @@ -5274,7 +5277,7 @@ def traceParseAction(f): # # global helpers # -def delimitedList( expr, delim=",", combine=False ): +def delimitedList(expr, delim=",", combine=False): """Helper to define a delimited list of expressions - the delimiter defaults to ','. By default, the list elements and delimiters can have intervening whitespace, and comments, but this can be @@ -5289,13 +5292,13 @@ def delimitedList( expr, delim=",", combine=False ): delimitedList(Word(alphas)).parseString("aa,bb,cc") # -> ['aa', 'bb', 'cc'] delimitedList(Word(hexnums), delim=':', combine=True).parseString("AA:BB:CC:DD:EE") # -> ['AA:BB:CC:DD:EE'] """ - dlName = _ustr(expr)+" ["+_ustr(delim)+" "+_ustr(expr)+"]..." + dlName = _ustr(expr) + " [" + _ustr(delim) + " " + _ustr(expr) + "]..." if combine: - return Combine( expr + ZeroOrMore( delim + expr ) ).setName(dlName) + return Combine(expr + ZeroOrMore(delim + expr)).setName(dlName) else: - return ( expr + ZeroOrMore( Suppress( delim ) + expr ) ).setName(dlName) + return (expr + ZeroOrMore(Suppress(delim) + expr)).setName(dlName) -def countedArray( expr, intExpr=None ): +def countedArray(expr, intExpr=None): """Helper to define a counted list of expressions. This helper defines a pattern of the form:: @@ -5319,22 +5322,22 @@ def countedArray( expr, intExpr=None ): countedArray(Word(alphas), intExpr=binaryConstant).parseString('10 ab cd ef') # -> ['ab', 'cd'] """ arrayExpr = Forward() - def countFieldParseAction(s,l,t): + def countFieldParseAction(s, l, t): n = t[0] - arrayExpr << (n and Group(And([expr]*n)) or Group(empty)) + arrayExpr << (n and Group(And([expr] * n)) or Group(empty)) return [] if intExpr is None: - intExpr = Word(nums).setParseAction(lambda t:int(t[0])) + intExpr = Word(nums).setParseAction(lambda t: int(t[0])) else: intExpr = intExpr.copy() intExpr.setName("arrayLen") intExpr.addParseAction(countFieldParseAction, callDuringTry=True) - return ( intExpr + arrayExpr ).setName('(len) ' + _ustr(expr) + '...') + return (intExpr + arrayExpr).setName('(len) ' + _ustr(expr) + '...') def _flatten(L): ret = [] for i in L: - if isinstance(i,list): + if isinstance(i, list): ret.extend(_flatten(i)) else: ret.append(i) @@ -5356,7 +5359,7 @@ def matchPreviousLiteral(expr): enabled. """ rep = Forward() - def copyTokenToRepeater(s,l,t): + def copyTokenToRepeater(s, l, t): if t: if len(t) == 1: rep << t[0] @@ -5388,23 +5391,23 @@ def matchPreviousExpr(expr): rep = Forward() e2 = expr.copy() rep <<= e2 - def copyTokenToRepeater(s,l,t): + def copyTokenToRepeater(s, l, t): matchTokens = _flatten(t.asList()) - def mustMatchTheseTokens(s,l,t): + def mustMatchTheseTokens(s, l, t): theseTokens = _flatten(t.asList()) - if theseTokens != matchTokens: - raise ParseException("",0,"") - rep.setParseAction( mustMatchTheseTokens, callDuringTry=True ) + if theseTokens != matchTokens: + raise ParseException('', 0, '') + rep.setParseAction(mustMatchTheseTokens, callDuringTry=True) expr.addParseAction(copyTokenToRepeater, callDuringTry=True) rep.setName('(prev) ' + _ustr(expr)) return rep def _escapeRegexRangeChars(s): - #~ escape these chars: ^-] + # ~ escape these chars: ^-] for c in r"\^-]": - s = s.replace(c,_bslash+c) - s = s.replace("\n",r"\n") - s = s.replace("\t",r"\t") + s = s.replace(c, _bslash + c) + s = s.replace("\n", r"\n") + s = s.replace("\t", r"\t") return _ustr(s) def oneOf(strs, caseless=False, useRegex=True, asKeyword=False): @@ -5453,13 +5456,13 @@ def oneOf(strs, caseless=False, useRegex=True, asKeyword=False): parseElementClass = Keyword if asKeyword else Literal symbols = [] - if isinstance(strs,basestring): + if isinstance(strs, basestring): symbols = strs.split() elif isinstance(strs, Iterable): symbols = list(strs) else: warnings.warn("Invalid argument to oneOf, expected string or iterable", - SyntaxWarning, stacklevel=2) + SyntaxWarning, stacklevel=2) if not symbols: return NoMatch() @@ -5467,26 +5470,26 @@ def oneOf(strs, caseless=False, useRegex=True, asKeyword=False): # if not producing keywords, need to reorder to take care to avoid masking # longer choices with shorter ones i = 0 - while i < len(symbols)-1: + while i < len(symbols) - 1: cur = symbols[i] - for j, other in enumerate(symbols[i+1:]): - if (isequal(other, cur)): - del symbols[i+j+1] + for j, other in enumerate(symbols[i + 1:]): + if isequal(other, cur): + del symbols[i + j + 1] break - elif (masks(cur, other)): - del symbols[i+j+1] - symbols.insert(i,other) + elif masks(cur, other): + del symbols[i + j + 1] + symbols.insert(i, other) break else: i += 1 if not (caseless or asKeyword) and useRegex: - #~ print (strs,"->", "|".join( [ _escapeRegexChars(sym) for sym in symbols] )) + # ~ print (strs, "->", "|".join([_escapeRegexChars(sym) for sym in symbols])) try: - if len(symbols)==len("".join(symbols)): - return Regex( "[%s]" % "".join(_escapeRegexRangeChars(sym) for sym in symbols) ).setName(' | '.join(symbols)) + if len(symbols) == len("".join(symbols)): + return Regex("[%s]" % "".join(_escapeRegexRangeChars(sym) for sym in symbols)).setName(' | '.join(symbols)) else: - return Regex( "|".join(re.escape(sym) for sym in symbols) ).setName(' | '.join(symbols)) + return Regex("|".join(re.escape(sym) for sym in symbols)).setName(' | '.join(symbols)) except Exception: warnings.warn("Exception creating Regex for oneOf, building MatchFirst", SyntaxWarning, stacklevel=2) @@ -5494,7 +5497,7 @@ def oneOf(strs, caseless=False, useRegex=True, asKeyword=False): # last resort, just use MatchFirst return MatchFirst(parseElementClass(sym) for sym in symbols).setName(' | '.join(symbols)) -def dictOf( key, value ): +def dictOf(key, value): """Helper to easily and clearly define a dictionary by specifying the respective patterns for the key and value. Takes care of defining the :class:`Dict`, :class:`ZeroOrMore`, and @@ -5552,8 +5555,8 @@ def originalTextFor(expr, asString=True): Example:: src = "this is test <b> bold <i>text</i> </b> normal text " - for tag in ("b","i"): - opener,closer = makeHTMLTags(tag) + for tag in ("b", "i"): + opener, closer = makeHTMLTags(tag) patt = originalTextFor(opener + SkipTo(closer) + closer) print(patt.searchString(src)[0]) @@ -5562,14 +5565,14 @@ def originalTextFor(expr, asString=True): ['<b> bold <i>text</i> </b>'] ['<i>text</i>'] """ - locMarker = Empty().setParseAction(lambda s,loc,t: loc) + locMarker = Empty().setParseAction(lambda s, loc, t: loc) endlocMarker = locMarker.copy() endlocMarker.callPreparse = False matchExpr = locMarker("_original_start") + expr + endlocMarker("_original_end") if asString: - extractText = lambda s,l,t: s[t._original_start:t._original_end] + extractText = lambda s, l, t: s[t._original_start: t._original_end] else: - def extractText(s,l,t): + def extractText(s, l, t): t[:] = [s[t.pop('_original_start'):t.pop('_original_end')]] matchExpr.setParseAction(extractText) matchExpr.ignoreExprs = expr.ignoreExprs @@ -5579,7 +5582,7 @@ def ungroup(expr): """Helper to undo pyparsing's default grouping of And expressions, even if all but one are non-empty. """ - return TokenConverter(expr).addParseAction(lambda t:t[0]) + return TokenConverter(expr).addParseAction(lambda t: t[0]) def locatedExpr(expr): """Helper to decorate a returned token with its starting and ending @@ -5606,7 +5609,7 @@ def locatedExpr(expr): [[8, 'lksdjjf', 15]] [[18, 'lkkjj', 23]] """ - locator = Empty().setParseAction(lambda s,l,t: l) + locator = Empty().setParseAction(lambda s, l, t: l) return Group(locator("locn_start") + expr("value") + locator.copy().leaveWhitespace()("locn_end")) @@ -5617,12 +5620,12 @@ lineEnd = LineEnd().setName("lineEnd") stringStart = StringStart().setName("stringStart") stringEnd = StringEnd().setName("stringEnd") -_escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1]) -_escapedHexChar = Regex(r"\\0?[xX][0-9a-fA-F]+").setParseAction(lambda s,l,t:unichr(int(t[0].lstrip(r'\0x'),16))) -_escapedOctChar = Regex(r"\\0[0-7]+").setParseAction(lambda s,l,t:unichr(int(t[0][1:],8))) +_escapedPunc = Word(_bslash, r"\[]-*.$+^?()~ ", exact=2).setParseAction(lambda s, l, t: t[0][1]) +_escapedHexChar = Regex(r"\\0?[xX][0-9a-fA-F]+").setParseAction(lambda s, l, t: unichr(int(t[0].lstrip(r'\0x'), 16))) +_escapedOctChar = Regex(r"\\0[0-7]+").setParseAction(lambda s, l, t: unichr(int(t[0][1:], 8))) _singleChar = _escapedPunc | _escapedHexChar | _escapedOctChar | CharsNotIn(r'\]', exact=1) _charRange = Group(_singleChar + Suppress("-") + _singleChar) -_reBracketExpr = Literal("[") + Optional("^").setResultsName("negate") + Group( OneOrMore( _charRange | _singleChar ) ).setResultsName("body") + "]" +_reBracketExpr = Literal("[") + Optional("^").setResultsName("negate") + Group(OneOrMore(_charRange | _singleChar)).setResultsName("body") + "]" def srange(s): r"""Helper to easily define string ranges for use in Word @@ -5650,7 +5653,7 @@ def srange(s): - any combination of the above (``'aeiouy'``, ``'a-zA-Z0-9_$'``, etc.) """ - _expanded = lambda p: p if not isinstance(p,ParseResults) else ''.join(unichr(c) for c in range(ord(p[0]),ord(p[1])+1)) + _expanded = lambda p: p if not isinstance(p, ParseResults) else ''.join(unichr(c) for c in range(ord(p[0]), ord(p[1]) + 1)) try: return "".join(_expanded(part) for part in _reBracketExpr.parseString(s).body) except Exception: @@ -5660,9 +5663,9 @@ def matchOnlyAtCol(n): """Helper method for defining parse actions that require matching at a specific column in the input text. """ - def verifyCol(strg,locn,toks): - if col(locn,strg) != n: - raise ParseException(strg,locn,"matched token not at column %d" % n) + def verifyCol(strg, locn, toks): + if col(locn, strg) != n: + raise ParseException(strg, locn, "matched token not at column %d" % n) return verifyCol def replaceWith(replStr): @@ -5678,9 +5681,9 @@ def replaceWith(replStr): OneOrMore(term).parseString("324 234 N/A 234") # -> [324, 234, nan, 234] """ - return lambda s,l,t: [replStr] + return lambda s, l, t: [replStr] -def removeQuotes(s,l,t): +def removeQuotes(s, l, t): """Helper parse action for removing quotation marks from parsed quoted strings. @@ -5731,7 +5734,7 @@ def tokenMap(func, *args): now is the winter of our discontent made glorious summer by this sun of york ['Now Is The Winter Of Our Discontent Made Glorious Summer By This Sun Of York'] """ - def pa(s,l,t): + def pa(s, l, t): return [func(tokn, *args) for tokn in t] try: @@ -5755,34 +5758,34 @@ def _makeTags(tagStr, xml, suppress_LT=Suppress("<"), suppress_GT=Suppress(">")): """Internal helper to construct opening and closing tag expressions, given a tag name""" - if isinstance(tagStr,basestring): + if isinstance(tagStr, basestring): resname = tagStr tagStr = Keyword(tagStr, caseless=not xml) else: resname = tagStr.name - tagAttrName = Word(alphas,alphanums+"_-:") - if (xml): - tagAttrValue = dblQuotedString.copy().setParseAction( removeQuotes ) + tagAttrName = Word(alphas, alphanums + "_-:") + if xml: + tagAttrValue = dblQuotedString.copy().setParseAction(removeQuotes) openTag = (suppress_LT + tagStr("tag") - + Dict(ZeroOrMore(Group(tagAttrName + Suppress("=") + tagAttrValue ))) - + Optional("/", default=[False])("empty").setParseAction(lambda s,l,t:t[0]=='/') + + Dict(ZeroOrMore(Group(tagAttrName + Suppress("=") + tagAttrValue))) + + Optional("/", default=[False])("empty").setParseAction(lambda s, l, t: t[0] == '/') + suppress_GT) else: - tagAttrValue = quotedString.copy().setParseAction( removeQuotes ) | Word(printables, excludeChars=">") + tagAttrValue = quotedString.copy().setParseAction(removeQuotes) | Word(printables, excludeChars=">") openTag = (suppress_LT + tagStr("tag") + Dict(ZeroOrMore(Group(tagAttrName.setParseAction(downcaseTokens) + Optional(Suppress("=") + tagAttrValue)))) - + Optional("/",default=[False])("empty").setParseAction(lambda s,l,t:t[0]=='/') + + Optional("/", default=[False])("empty").setParseAction(lambda s, l, t: t[0] == '/') + suppress_GT) closeTag = Combine(_L("</") + tagStr + ">", adjacent=False) openTag.setName("<%s>" % resname) # add start<tagname> results name in parse action now that ungrouped names are not reported at two levels - openTag.addParseAction(lambda t: t.__setitem__("start"+"".join(resname.replace(":"," ").title().split()), t.copy())) - closeTag = closeTag("end"+"".join(resname.replace(":"," ").title().split())).setName("</%s>" % resname) + openTag.addParseAction(lambda t: t.__setitem__("start" + "".join(resname.replace(":", " ").title().split()), t.copy())) + closeTag = closeTag("end" + "".join(resname.replace(":", " ").title().split())).setName("</%s>" % resname) openTag.tag = resname closeTag.tag = resname openTag.tag_body = SkipTo(closeTag()) @@ -5798,7 +5801,7 @@ def makeHTMLTags(tagStr): text = '<td>More info at the <a href="https://github.com/pyparsing/pyparsing/wiki">pyparsing</a> wiki page</td>' # makeHTMLTags returns pyparsing expressions for the opening and # closing tags as a 2-tuple - a,a_end = makeHTMLTags("A") + a, a_end = makeHTMLTags("A") link_expr = a + SkipTo(a_end)("link_text") + a_end for link in link_expr.searchString(text): @@ -5810,7 +5813,7 @@ def makeHTMLTags(tagStr): pyparsing -> https://github.com/pyparsing/pyparsing/wiki """ - return _makeTags( tagStr, False ) + return _makeTags(tagStr, False) def makeXMLTags(tagStr): """Helper to construct opening and closing tag expressions for XML, @@ -5818,9 +5821,9 @@ def makeXMLTags(tagStr): Example: similar to :class:`makeHTMLTags` """ - return _makeTags( tagStr, True ) + return _makeTags(tagStr, True) -def withAttribute(*args,**attrDict): +def withAttribute(*args, **attrDict): """Helper to create a validating parse action to be used with start tags created with :class:`makeXMLTags` or :class:`makeHTMLTags`. Use ``withAttribute`` to qualify @@ -5833,7 +5836,7 @@ def withAttribute(*args,**attrDict): - keyword arguments, as in ``(align="right")``, or - as an explicit dict with ``**`` operator, when an attribute name is also a Python reserved word, as in ``**{"class":"Customer", "align":"right"}`` - - a list of name-value tuples, as in ``(("ns1:class", "Customer"), ("ns2:align","right"))`` + - a list of name-value tuples, as in ``(("ns1:class", "Customer"), ("ns2:align", "right"))`` For attribute names with a namespace prefix, you must use the second form. Attribute names are matched insensitive to upper/lower case. @@ -5880,13 +5883,13 @@ def withAttribute(*args,**attrDict): attrs = args[:] else: attrs = attrDict.items() - attrs = [(k,v) for k,v in attrs] - def pa(s,l,tokens): - for attrName,attrValue in attrs: + attrs = [(k, v) for k, v in attrs] + def pa(s, l, tokens): + for attrName, attrValue in attrs: if attrName not in tokens: - raise ParseException(s,l,"no matching attribute " + attrName) + raise ParseException(s, l, "no matching attribute " + attrName) if attrValue != withAttribute.ANY_VALUE and tokens[attrName] != attrValue: - raise ParseException(s,l,"attribute '%s' has value '%s', must be '%s'" % + raise ParseException(s, l, "attribute '%s' has value '%s', must be '%s'" % (attrName, tokens[attrName], attrValue)) return pa withAttribute.ANY_VALUE = object() @@ -5927,13 +5930,13 @@ def withClass(classname, namespace=''): 1,3 2,3 1,1 """ classattr = "%s:class" % namespace if namespace else "class" - return withAttribute(**{classattr : classname}) + return withAttribute(**{classattr: classname}) opAssoc = SimpleNamespace() opAssoc.LEFT = object() opAssoc.RIGHT = object() -def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ): +def infixNotation(baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')')): """Helper method for constructing grammars of expressions made up of operators working in a precedence hierarchy. Operators may be unary or binary, left- or right-associative. Parse actions can also be @@ -6011,9 +6014,9 @@ def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ): return loc, [] ret = Forward() - lastExpr = baseExpr | ( lpar + ret + rpar ) - for i,operDef in enumerate(opList): - opExpr,arity,rightLeftAssoc,pa = (operDef + (None,))[:4] + lastExpr = baseExpr | (lpar + ret + rpar) + for i, operDef in enumerate(opList): + opExpr, arity, rightLeftAssoc, pa = (operDef + (None, ))[:4] termName = "%s term" % opExpr if arity < 3 else "%s%s term" % opExpr if arity == 3: if opExpr is None or len(opExpr) != 2: @@ -6023,15 +6026,15 @@ def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ): thisExpr = Forward().setName(termName) if rightLeftAssoc == opAssoc.LEFT: if arity == 1: - matchExpr = _FB(lastExpr + opExpr) + Group( lastExpr + OneOrMore( opExpr ) ) + matchExpr = _FB(lastExpr + opExpr) + Group(lastExpr + OneOrMore(opExpr)) elif arity == 2: if opExpr is not None: - matchExpr = _FB(lastExpr + opExpr + lastExpr) + Group( lastExpr + OneOrMore( opExpr + lastExpr ) ) + matchExpr = _FB(lastExpr + opExpr + lastExpr) + Group(lastExpr + OneOrMore(opExpr + lastExpr)) else: - matchExpr = _FB(lastExpr+lastExpr) + Group( lastExpr + OneOrMore(lastExpr) ) + matchExpr = _FB(lastExpr + lastExpr) + Group(lastExpr + OneOrMore(lastExpr)) elif arity == 3: - matchExpr = _FB(lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr) + \ - Group( lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr ) + matchExpr = (_FB(lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr) + + Group(lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr)) else: raise ValueError("operator must be unary (1), binary (2), or ternary (3)") elif rightLeftAssoc == opAssoc.RIGHT: @@ -6039,15 +6042,15 @@ def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ): # try to avoid LR with this extra test if not isinstance(opExpr, Optional): opExpr = Optional(opExpr) - matchExpr = _FB(opExpr.expr + thisExpr) + Group( opExpr + thisExpr ) + matchExpr = _FB(opExpr.expr + thisExpr) + Group(opExpr + thisExpr) elif arity == 2: if opExpr is not None: - matchExpr = _FB(lastExpr + opExpr + thisExpr) + Group( lastExpr + OneOrMore( opExpr + thisExpr ) ) + matchExpr = _FB(lastExpr + opExpr + thisExpr) + Group(lastExpr + OneOrMore(opExpr + thisExpr)) else: - matchExpr = _FB(lastExpr + thisExpr) + Group( lastExpr + OneOrMore( thisExpr ) ) + matchExpr = _FB(lastExpr + thisExpr) + Group(lastExpr + OneOrMore(thisExpr)) elif arity == 3: - matchExpr = _FB(lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr) + \ - Group( lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr ) + matchExpr = (_FB(lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr) + + Group(lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr)) else: raise ValueError("operator must be unary (1), binary (2), or ternary (3)") else: @@ -6057,7 +6060,7 @@ def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ): matchExpr.setParseAction(*pa) else: matchExpr.setParseAction(pa) - thisExpr <<= ( matchExpr.setName(termName) | lastExpr ) + thisExpr <<= (matchExpr.setName(termName) | lastExpr) lastExpr = thisExpr ret <<= lastExpr return ret @@ -6066,10 +6069,10 @@ operatorPrecedence = infixNotation """(Deprecated) Former name of :class:`infixNotation`, will be dropped in a future release.""" -dblQuotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*')+'"').setName("string enclosed in double quotes") -sglQuotedString = Combine(Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*")+"'").setName("string enclosed in single quotes") -quotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*')+'"'| - Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*")+"'").setName("quotedString using single or double quotes") +dblQuotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*') + '"').setName("string enclosed in double quotes") +sglQuotedString = Combine(Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*") + "'").setName("string enclosed in single quotes") +quotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*') + '"' + | Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*") + "'").setName("quotedString using single or double quotes") unicodeString = Combine(_L('u') + quotedString.copy()).setName("unicode string literal") def nestedExpr(opener="(", closer=")", content=None, ignoreExpr=quotedString.copy()): @@ -6105,7 +6108,7 @@ def nestedExpr(opener="(", closer=")", content=None, ignoreExpr=quotedString.cop ident = Word(alphas+'_', alphanums+'_') number = pyparsing_common.number arg = Group(decl_data_type + ident) - LPAR,RPAR = map(Suppress, "()") + LPAR, RPAR = map(Suppress, "()") code_body = nestedExpr('{', '}', ignoreExpr=(quotedString | cStyleComment)) @@ -6140,33 +6143,40 @@ def nestedExpr(opener="(", closer=")", content=None, ignoreExpr=quotedString.cop if opener == closer: raise ValueError("opening and closing strings cannot be the same") if content is None: - if isinstance(opener,basestring) and isinstance(closer,basestring): - if len(opener) == 1 and len(closer)==1: + if isinstance(opener, basestring) and isinstance(closer, basestring): + if len(opener) == 1 and len(closer) == 1: if ignoreExpr is not None: - content = (Combine(OneOrMore(~ignoreExpr + - CharsNotIn(opener+closer+ParserElement.DEFAULT_WHITE_CHARS,exact=1)) - ).setParseAction(lambda t:t[0].strip())) + content = (Combine(OneOrMore(~ignoreExpr + + CharsNotIn(opener + + closer + + ParserElement.DEFAULT_WHITE_CHARS, exact=1) + ) + ).setParseAction(lambda t: t[0].strip())) else: - content = (empty.copy()+CharsNotIn(opener+closer+ParserElement.DEFAULT_WHITE_CHARS - ).setParseAction(lambda t:t[0].strip())) + content = (empty.copy() + CharsNotIn(opener + + closer + + ParserElement.DEFAULT_WHITE_CHARS + ).setParseAction(lambda t: t[0].strip())) else: if ignoreExpr is not None: - content = (Combine(OneOrMore(~ignoreExpr + - ~Literal(opener) + ~Literal(closer) + - CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS,exact=1)) - ).setParseAction(lambda t:t[0].strip())) + content = (Combine(OneOrMore(~ignoreExpr + + ~Literal(opener) + + ~Literal(closer) + + CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS, exact=1)) + ).setParseAction(lambda t: t[0].strip())) else: - content = (Combine(OneOrMore(~Literal(opener) + ~Literal(closer) + - CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS,exact=1)) - ).setParseAction(lambda t:t[0].strip())) + content = (Combine(OneOrMore(~Literal(opener) + + ~Literal(closer) + + CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS, exact=1)) + ).setParseAction(lambda t: t[0].strip())) else: raise ValueError("opening and closing arguments must be strings if no content expression is given") ret = Forward() if ignoreExpr is not None: - ret <<= Group( Suppress(opener) + ZeroOrMore( ignoreExpr | ret | content ) + Suppress(closer) ) + ret <<= Group(Suppress(opener) + ZeroOrMore(ignoreExpr | ret | content) + Suppress(closer)) else: - ret <<= Group( Suppress(opener) + ZeroOrMore( ret | content ) + Suppress(closer) ) - ret.setName('nested %s%s expression' % (opener,closer)) + ret <<= Group(Suppress(opener) + ZeroOrMore(ret | content) + Suppress(closer)) + ret.setName('nested %s%s expression' % (opener, closer)) return ret def indentedBlock(blockStatementExpr, indentStack, indent=True): @@ -6214,15 +6224,15 @@ def indentedBlock(blockStatementExpr, indentStack, indent=True): stmt = Forward() identifier = Word(alphas, alphanums) - funcDecl = ("def" + identifier + Group( "(" + Optional( delimitedList(identifier) ) + ")" ) + ":") + funcDecl = ("def" + identifier + Group("(" + Optional(delimitedList(identifier)) + ")") + ":") func_body = indentedBlock(stmt, indentStack) - funcDef = Group( funcDecl + func_body ) + funcDef = Group(funcDecl + func_body) rvalue = Forward() funcCall = Group(identifier + "(" + Optional(delimitedList(rvalue)) + ")") rvalue << (funcCall | identifier | Word(nums)) assignment = Group(identifier + "=" + rvalue) - stmt << ( funcDef | assignment | identifier ) + stmt << (funcDef | assignment | identifier) module_body = OneOrMore(stmt) @@ -6255,26 +6265,26 @@ def indentedBlock(blockStatementExpr, indentStack, indent=True): def reset_stack(): indentStack[:] = backup_stack - def checkPeerIndent(s,l,t): + def checkPeerIndent(s, l, t): if l >= len(s): return - curCol = col(l,s) + curCol = col(l, s) if curCol != indentStack[-1]: if curCol > indentStack[-1]: - raise ParseException(s,l,"illegal nesting") - raise ParseException(s,l,"not a peer entry") + raise ParseException(s, l, "illegal nesting") + raise ParseException(s, l, "not a peer entry") - def checkSubIndent(s,l,t): - curCol = col(l,s) + def checkSubIndent(s, l, t): + curCol = col(l, s) if curCol > indentStack[-1]: - indentStack.append( curCol ) + indentStack.append(curCol) else: - raise ParseException(s,l,"not a subentry") + raise ParseException(s, l, "not a subentry") - def checkUnindent(s,l,t): + def checkUnindent(s, l, t): if l >= len(s): return - curCol = col(l,s) + curCol = col(l, s) if not(indentStack and curCol in indentStack): - raise ParseException(s,l,"not an unindent") + raise ParseException(s, l, "not an unindent") if curCol < indentStack[-1]: indentStack.pop() @@ -6298,8 +6308,8 @@ def indentedBlock(blockStatementExpr, indentStack, indent=True): alphas8bit = srange(r"[\0xc0-\0xd6\0xd8-\0xf6\0xf8-\0xff]") punc8bit = srange(r"[\0xa1-\0xbf\0xd7\0xf7]") -anyOpenTag,anyCloseTag = makeHTMLTags(Word(alphas,alphanums+"_:").setName('any tag')) -_htmlEntityMap = dict(zip("gt lt amp nbsp quot apos".split(),'><& "\'')) +anyOpenTag, anyCloseTag = makeHTMLTags(Word(alphas, alphanums + "_:").setName('any tag')) +_htmlEntityMap = dict(zip("gt lt amp nbsp quot apos".split(), '><& "\'')) commonHTMLEntity = Regex('&(?P<entity>' + '|'.join(_htmlEntityMap.keys()) +");").setName("common HTML entity") def replaceHTMLEntity(t): """Helper parser action to replace common HTML entities with their special characters""" @@ -6316,7 +6326,7 @@ restOfLine = Regex(r".*").leaveWhitespace().setName("rest of line") dblSlashComment = Regex(r"//(?:\\\n|[^\n])*").setName("// comment") "Comment of the form ``// ... (to end of line)``" -cppStyleComment = Combine(Regex(r"/\*(?:[^*]|\*(?!/))*") + '*/'| dblSlashComment).setName("C++ style comment") +cppStyleComment = Combine(Regex(r"/\*(?:[^*]|\*(?!/))*") + '*/' | dblSlashComment).setName("C++ style comment") "Comment of either form :class:`cStyleComment` or :class:`dblSlashComment`" javaStyleComment = cppStyleComment @@ -6325,10 +6335,10 @@ javaStyleComment = cppStyleComment pythonStyleComment = Regex(r"#.*").setName("Python style comment") "Comment of the form ``# ... (to end of line)``" -_commasepitem = Combine(OneOrMore(Word(printables, excludeChars=',') + - Optional( Word(" \t") + - ~Literal(",") + ~LineEnd() ) ) ).streamline().setName("commaItem") -commaSeparatedList = delimitedList( Optional( quotedString.copy() | _commasepitem, default="") ).setName("commaSeparatedList") +_commasepitem = Combine(OneOrMore(Word(printables, excludeChars=',') + + Optional(Word(" \t") + + ~Literal(",") + ~LineEnd()))).streamline().setName("commaItem") +commaSeparatedList = delimitedList(Optional(quotedString.copy() | _commasepitem, default="")).setName("commaSeparatedList") """(Deprecated) Predefined expression of 1 or more printable words or quoted strings, separated by commas. @@ -6494,7 +6504,7 @@ class pyparsing_common: integer = Word(nums).setName("integer").setParseAction(convertToInteger) """expression that parses an unsigned integer, returns an int""" - hex_integer = Word(hexnums).setName("hex integer").setParseAction(tokenMap(int,16)) + hex_integer = Word(hexnums).setName("hex integer").setParseAction(tokenMap(int, 16)) """expression that parses a hexadecimal integer, returns an int""" signed_integer = Regex(r'[+-]?\d+').setName("signed integer").setParseAction(convertToInteger) @@ -6522,15 +6532,18 @@ class pyparsing_common: fnumber = Regex(r'[+-]?\d+\.?\d*([eE][+-]?\d+)?').setName("fnumber").setParseAction(convertToFloat) """any int or real number, returned as float""" - identifier = Word(alphas+'_', alphanums+'_').setName("identifier") + identifier = Word(alphas + '_', alphanums + '_').setName("identifier") """typical code identifier (leading alpha or '_', followed by 0 or more alphas, nums, or '_')""" ipv4_address = Regex(r'(25[0-5]|2[0-4][0-9]|1?[0-9]{1,2})(\.(25[0-5]|2[0-4][0-9]|1?[0-9]{1,2})){3}').setName("IPv4 address") "IPv4 address (``0.0.0.0 - 255.255.255.255``)" _ipv6_part = Regex(r'[0-9a-fA-F]{1,4}').setName("hex_integer") - _full_ipv6_address = (_ipv6_part + (':' + _ipv6_part)*7).setName("full IPv6 address") - _short_ipv6_address = (Optional(_ipv6_part + (':' + _ipv6_part)*(0,6)) + "::" + Optional(_ipv6_part + (':' + _ipv6_part)*(0,6))).setName("short IPv6 address") + _full_ipv6_address = (_ipv6_part + (':' + _ipv6_part) * 7).setName("full IPv6 address") + _short_ipv6_address = (Optional(_ipv6_part + (':' + _ipv6_part) * (0, 6)) + + "::" + + Optional(_ipv6_part + (':' + _ipv6_part) * (0, 6)) + ).setName("short IPv6 address") _short_ipv6_address.addCondition(lambda t: sum(1 for tt in t if pyparsing_common._ipv6_part.matches(tt)) < 8) _mixed_ipv6_address = ("::ffff:" + ipv4_address).setName("mixed IPv6 address") ipv6_address = Combine((_full_ipv6_address | _mixed_ipv6_address | _short_ipv6_address).setName("IPv6 address")).setName("IPv6 address") @@ -6557,7 +6570,7 @@ class pyparsing_common: [datetime.date(1999, 12, 31)] """ - def cvt_fn(s,l,t): + def cvt_fn(s, l, t): try: return datetime.strptime(t[0], fmt).date() except ValueError as ve: @@ -6582,7 +6595,7 @@ class pyparsing_common: [datetime.datetime(1999, 12, 31, 23, 59, 59, 999000)] """ - def cvt_fn(s,l,t): + def cvt_fn(s, l, t): try: return datetime.strptime(t[0], fmt) except ValueError as ve: @@ -6607,7 +6620,7 @@ class pyparsing_common: # strip HTML links from normal text text = '<td>More info at the <a href="https://github.com/pyparsing/pyparsing/wiki">pyparsing</a> wiki page</td>' - td,td_end = makeHTMLTags("TD") + td, td_end = makeHTMLTags("TD") table_text = td + SkipTo(td_end).setParseAction(pyparsing_common.stripHTMLTags)("body") + td_end print(table_text.parseString(text).body) @@ -6617,9 +6630,13 @@ class pyparsing_common: """ return pyparsing_common._html_stripper.transformString(tokens[0]) - _commasepitem = Combine(OneOrMore(~Literal(",") + ~LineEnd() + Word(printables, excludeChars=',') - + Optional( White(" \t") ) ) ).streamline().setName("commaItem") - comma_separated_list = delimitedList( Optional( quotedString.copy() | _commasepitem, default="") ).setName("comma separated list") + _commasepitem = Combine(OneOrMore(~Literal(",") + + ~LineEnd() + + Word(printables, excludeChars=',') + + Optional(White(" \t")))).streamline().setName("commaItem") + comma_separated_list = delimitedList(Optional(quotedString.copy() + | _commasepitem, default='') + ).setName("comma separated list") """Predefined expression of 1 or more printable words or quoted strings, separated by commas.""" upcaseTokens = staticmethod(tokenMap(lambda t: _ustr(t).upper())) @@ -6638,7 +6655,8 @@ class _lazyclassproperty(object): def __get__(self, obj, cls): if cls is None: cls = type(obj) - if not hasattr(cls, '_intern') or any(cls._intern is getattr(superclass, '_intern', []) for superclass in cls.__mro__[1:]): + if not hasattr(cls, '_intern') or any(cls._intern is getattr(superclass, '_intern', []) + for superclass in cls.__mro__[1:]): cls._intern = {} attrname = self.fn.__name__ if attrname not in cls._intern: @@ -6669,7 +6687,7 @@ class unicode_set(object): if cc is unicode_set: break for rr in cc._ranges: - ret.extend(range(rr[0], rr[-1]+1)) + ret.extend(range(rr[0], rr[-1] + 1)) return [unichr(c) for c in sorted(set(ret))] @_lazyclassproperty @@ -6725,27 +6743,27 @@ class pyparsing_unicode(unicode_set): class Chinese(unicode_set): "Unicode set for Chinese Unicode Character Range" - _ranges = [(0x4e00, 0x9fff), (0x3000, 0x303f), ] + _ranges = [(0x4e00, 0x9fff), (0x3000, 0x303f),] class Japanese(unicode_set): "Unicode set for Japanese Unicode Character Range, combining Kanji, Hiragana, and Katakana ranges" - _ranges = [ ] + _ranges = [] class Kanji(unicode_set): "Unicode set for Kanji Unicode Character Range" - _ranges = [(0x4E00, 0x9Fbf), (0x3000, 0x303f), ] + _ranges = [(0x4E00, 0x9Fbf), (0x3000, 0x303f),] class Hiragana(unicode_set): "Unicode set for Hiragana Unicode Character Range" - _ranges = [(0x3040, 0x309f), ] + _ranges = [(0x3040, 0x309f),] class Katakana(unicode_set): "Unicode set for Katakana Unicode Character Range" - _ranges = [(0x30a0, 0x30ff), ] + _ranges = [(0x30a0, 0x30ff),] class Korean(unicode_set): "Unicode set for Korean Unicode Character Range" - _ranges = [(0xac00, 0xd7af), (0x1100, 0x11ff), (0x3130, 0x318f), (0xa960, 0xa97f), (0xd7b0, 0xd7ff), (0x3000, 0x303f), ] + _ranges = [(0xac00, 0xd7af), (0x1100, 0x11ff), (0x3130, 0x318f), (0xa960, 0xa97f), (0xd7b0, 0xd7ff), (0x3000, 0x303f),] class CJK(Chinese, Japanese, Korean): "Unicode set for combined Chinese, Japanese, and Korean (CJK) Unicode Character Range" @@ -6753,15 +6771,15 @@ class pyparsing_unicode(unicode_set): class Thai(unicode_set): "Unicode set for Thai Unicode Character Range" - _ranges = [(0x0e01, 0x0e3a), (0x0e3f, 0x0e5b), ] + _ranges = [(0x0e01, 0x0e3a), (0x0e3f, 0x0e5b),] class Arabic(unicode_set): "Unicode set for Arabic Unicode Character Range" - _ranges = [(0x0600, 0x061b), (0x061e, 0x06ff), (0x0700, 0x077f), ] + _ranges = [(0x0600, 0x061b), (0x061e, 0x06ff), (0x0700, 0x077f),] class Hebrew(unicode_set): "Unicode set for Hebrew Unicode Character Range" - _ranges = [(0x0590, 0x05ff), ] + _ranges = [(0x0590, 0x05ff),] class Devanagari(unicode_set): "Unicode set for Devanagari Unicode Character Range" @@ -6773,18 +6791,18 @@ pyparsing_unicode.Japanese._ranges = (pyparsing_unicode.Japanese.Kanji._ranges # define ranges in language character sets if PY_3: - setattr(pyparsing_unicode, "العربية", pyparsing_unicode.Arabic) - setattr(pyparsing_unicode, "中文", pyparsing_unicode.Chinese) - setattr(pyparsing_unicode, "кириллица", pyparsing_unicode.Cyrillic) - setattr(pyparsing_unicode, "Ελληνικά", pyparsing_unicode.Greek) - setattr(pyparsing_unicode, "עִברִית", pyparsing_unicode.Hebrew) - setattr(pyparsing_unicode, "日本語", pyparsing_unicode.Japanese) - setattr(pyparsing_unicode.Japanese, "漢字", pyparsing_unicode.Japanese.Kanji) - setattr(pyparsing_unicode.Japanese, "カタカナ", pyparsing_unicode.Japanese.Katakana) - setattr(pyparsing_unicode.Japanese, "ひらがな", pyparsing_unicode.Japanese.Hiragana) - setattr(pyparsing_unicode, "한국어", pyparsing_unicode.Korean) - setattr(pyparsing_unicode, "ไทย", pyparsing_unicode.Thai) - setattr(pyparsing_unicode, "देवनागरी", pyparsing_unicode.Devanagari) + setattr(pyparsing_unicode, u"العربية", pyparsing_unicode.Arabic) + setattr(pyparsing_unicode, u"中文", pyparsing_unicode.Chinese) + setattr(pyparsing_unicode, u"кириллица", pyparsing_unicode.Cyrillic) + setattr(pyparsing_unicode, u"Ελληνικά", pyparsing_unicode.Greek) + setattr(pyparsing_unicode, u"עִברִית", pyparsing_unicode.Hebrew) + setattr(pyparsing_unicode, u"日本語", pyparsing_unicode.Japanese) + setattr(pyparsing_unicode.Japanese, u"漢字", pyparsing_unicode.Japanese.Kanji) + setattr(pyparsing_unicode.Japanese, u"カタカナ", pyparsing_unicode.Japanese.Katakana) + setattr(pyparsing_unicode.Japanese, u"ひらがな", pyparsing_unicode.Japanese.Hiragana) + setattr(pyparsing_unicode, u"한국어", pyparsing_unicode.Korean) + setattr(pyparsing_unicode, u"ไทย", pyparsing_unicode.Thai) + setattr(pyparsing_unicode, u"देवनागरी", pyparsing_unicode.Devanagari) if __name__ == "__main__": |