summaryrefslogtreecommitdiff
path: root/giscanner/annotationparser.py
Commit message (Collapse)AuthorAgeFilesLines
* giscanner: fix c0e748e1cdf8cf0803266f94c3c5ad154df504a8Dieter Verfaillie2013-10-081-1/+2
| | | | | | | | | | | When encountering /**/ in the source, parse_gtk_doc_comment() would be executed (due to the /** part starting a GTK-Doc comment block) and would happily consume the / and everything up until the next comment block was closed or EOF, thus consuming a whole block of C code... Encoutered in the wild here: https://git.gnome.org/browse/clutter-gst/tree/clutter-gst/clutter-gst-player.c?id=03f0d8f48bd7f60e582e6185cb73a07038e8d55d#n1926
* giscanner: complain about text before the ' * 'Dieter Verfaillie2013-10-081-0/+9
| | | | | | Makes our GTK-Doc comment block rewriting tool halt on such issues, requireing user intervention instead of writing back even more bogus data.
* giscanner: return namedtuples where appropriateDieter Verfaillie2013-10-081-65/+77
| | | | | makes _parse_annotations() and _parse_fields() callers slightly more readable
* giscanner: Fix GtkDocBlock.position...Dieter Verfaillie2013-10-081-59/+119
| | | | | ... so it points to the start of the GTK-Doc comment block instead of the position of the identifier field.
* giscanner: refactor GTK-Doc comment block serializationDieter Verfaillie2013-10-081-103/+195
|
* giscanner: make some warnings errorsDieter Verfaillie2013-10-081-23/+23
| | | | | | | GTK-Doc comment block fixer tool will refuse to rewrite source files that generated errors (indicating a source->parse tree->source would result in information being lost), but will happily continue on warnings (which do not result in information being lost).
* giscanner: honor line and paragraph breaks in description fieldsDieter Verfaillie2013-10-081-14/+34
| | | | | | | | | GTK-Doc parameter description fields are allowed to span multiple lines, tag description fields are allowed to span multiple lines and paragraphs. A tool fixing/rewriting GTK-Doc comment blocks in source files would need to have description fields parsed and stored (almost) exactly as they appear in the source file.
* giscanner: improve state trackingDieter Verfaillie2013-10-081-19/+12
|
* giscanner: better lineno handlingDieter Verfaillie2013-10-081-12/+13
| | | | | | No need to enumerate the comment lines list as we already receive the lineno of the very first line as a parameter. Simply increment that one when looping over the comment lines list.
* giscanner: store indentation before the '*' of each lineDieter Verfaillie2013-10-081-14/+20
|
* giscanner: store code before and after comment blockDieter Verfaillie2013-10-081-32/+76
| | | | | | so we can later use them to re-write source files containing broken GTK-Doc comment blocks where /** is preceded by and/or */ is followed by code...
* giscanner: accept different forms of line breaksDieter Verfaillie2013-10-081-4/+4
|
* giscanner: merge _parse_comment_block() into parse_comment_block()Dieter Verfaillie2013-10-081-26/+0
|
* giscanner: refactor annotation validationDieter Verfaillie2013-10-081-260/+385
| | | | | | | | - annotations on the identifier (formerly g-i specific tags) have never been validated before, so fix this - removes duplicate validation code from GtkDocTag and GtkDocParameter - remove repeated validation code doing the same thing as annotationparser from maintransformer...
* giscanner: give parameters their own storage classDieter Verfaillie2013-10-081-40/+221
|
* giscanner: deprecate GObject-Introspection GTK-Doc tagsDieter Verfaillie2013-10-081-32/+53
| | | | | | | | | | | | | | | | | | | | | After discussing a different issue with the GTK-Doc maintainers, we learned that our g-i specific top level tags should never have existed in the first place. The prefered notation for annotations that apply to the identifier should be written on the identifier line, for example like what we already do with (skip). As a result, this patch deprecates g-i specific top level tags and implements them as annotations on the identifier instead but still keeps support for malformed comment blocks using g-i specific top level tags. This means that all annotated code "out there" will continue to work just fine with this version of g-i, but when a developer decides to fix deprecation warnings in his/her comment blocks, the dependency on g-i needs to be raised to a version that contains at least this patch. #676133 https://bugzilla.gnome.org/show_bug.cgi?id=676133
* giscanner: flesh out annotation parsing and storageDieter Verfaillie2013-10-081-367/+611
| | | | | | | | | | - remove annotations regex, restore proper parens parsing - drop weird DocOption() storage class and use lists/dicts as appropriate - make GtkDocAnnotations a simple OrderedDict subclass instead of a weird hybrid dict/list storage class - Deprecate Attribute: tag, replace with (attributes) annotation on the identifier
* giscanner: extract tag valuesDieter Verfaillie2013-10-081-9/+54
|
* giscanner: use Position and warn directlyDieter Verfaillie2013-10-081-75/+75
|
* giscanner: remove unused GtkDocCommentBlock.valueDieter Verfaillie2013-10-081-2/+1
|
* giscanner: remove unused backrefsDieter Verfaillie2013-10-081-10/+8
|
* giscanner: bring some order into our constantsDieter Verfaillie2013-10-081-47/+60
| | | | | Write them in alphabetical order and explain what they are used for a bit more clearly.
* giscanner: fix GTK-Doc section parsingDieter Verfaillie2013-10-081-6/+4
|
* giscanner: fix GTK-Doc identifier parsingDieter Verfaillie2013-10-081-51/+33
| | | | | | No need to track what type of identifier we've found as we don't use that information later on. Continue searching for an identifier if it's not found on the first line like gtkdoc-mkdb does.
* giscanner: rename DocAnnotations to GtkDocAnnotationsDieter Verfaillie2013-10-081-44/+44
| | | | and move it below DocOption for easier reading
* giscanner: rename DocTag to GtkDocTagDieter Verfaillie2013-10-081-85/+85
| | | | | and move it downwards after DocOption and DocAnnotations for easier reading
* giscanner: rename DocBlock to GtkDocCommentBlockDieter Verfaillie2013-10-081-64/+77
| | | | | and move it downwards after DocTag, DocOptions etc for easier reading
* giscanner: restore support for 'Return:' and 'Returns value:' tagsDieter Verfaillie2013-10-081-5/+13
| | | | | Accepted by old version of annotationparser.py and actually encountered in the wild. Internaly normalize to 'Returns:'.
* giscanner: only warn about missing delimiter when necessaryDieter Verfaillie2013-10-081-9/+9
| | | | | | | | Turns out gtkdoc-mkdb has always accepted identifier parts with and without a delimiter after the identifier_name field, so it is safe to only require it when annotations are present or in other words, when there are actually multiple fields to be seperated...
* giscanner: give pointer to original comment block...Dieter Verfaillie2013-10-081-8/+10
| | | | | ...when complaining about multiple comment blocks documenting the same identifier.
* giscanner: complete vararg parameter parsingDieter Verfaillie2013-10-081-3/+9
| | | | | | | | | | Normalize deprecated @Varargs [1] and named __VA_ARGS__ [2] notations in as single location, add tests. Remove special casing in maintransformer as it is no longer a special case (tests/warn/unknown-parameter.h passes). [1] https://git.gnome.org/browse/gtk-doc/commit/gtkdoc-mkdb.in?id=4c9d4db0e264d403b301dad9be1c290134846d03 [1] https://git.gnome.org/browse/gtk-doc/commit/gtkdoc-mkdb.in?id=55c78181d6ae1606bdf7885e9caed70a484127ec
* giscanner: define __slots__ on some classesDieter Verfaillie2013-10-081-0/+9
| | | | | | For example, when creating Gtk-3.0.gir, thousands of instances of these classes get created. This makes us waste less space for these instances by not having their __dict__ being created.
* giscanner: always use re.UNICODE modeDieter Verfaillie2013-10-081-16/+12
| | | | | should prevent us from running into locale specific issues wrt what's whitespace, alpha, numeric, etc.
* giscanner: use "\s" instead of "[^\S\n\r]"Dieter Verfaillie2013-10-081-48/+48
| | | | | The original idea was to only match non breaking space, but there's really no reason to do things differently than gtkdoc-mkdb.
* giscanner: expand parse_comment_block() parametersDieter Verfaillie2013-10-081-7/+7
| | | | | Makes it consistent with the parse_comment_blocks() and _parse_comment_block() methods.
* giscanner: rename the parse() method to parse_comment_blocks()Dieter Verfaillie2013-10-081-1/+1
| | | | | | We already have a parse_comment_block() method parsing a single GTK-Doc comment block so it only seems natural to have a plural parse_comment_blocks() to go along with that.
* giscanner: rename AnnotationParser() to GtkDocCommentBlockParser()Dieter Verfaillie2013-10-081-2/+2
| | | | | | Clarify the purpose of what up until now was know as the AnnotationParser() class, as it does more than just extracting annotations, it parses the complete GTK-Doc comment block.
* giscanner: rename "comment" to "description"Dieter Verfaillie2013-10-081-21/+21
| | | | Simply because they are called description fields on GTK-Doc parts...
* giscanner: rename "colon" to "delimiter"Dieter Verfaillie2013-10-081-19/+19
| | | | | | | It serves as a GTK-Doc "field" separator, so delimiter seems like a better name. This also makes the distinction between the colon used in property/signal names (which is not a "field" separator) and the GTK-Doc "field" separator a bit more clear.
* giscanner: rename "options" to "annotations"Dieter Verfaillie2013-10-081-143/+141
| | | | | | | Makes it easier to understand the difference between an annotation and an option, which are written as "(annotation option1=value1 option2=value2 ...)" in GTK-Doc comment blocks.
* giscanner: reindent re documentationDieter Verfaillie2013-10-081-101/+101
|
* giscanner: improve inline docsDieter Verfaillie2013-10-081-23/+106
|
* tests: Update misc/pep8.py to 1.4.6Dieter Verfaillie2013-10-081-28/+19
| | | | | Updated from: https://raw.github.com/jcrocholl/pep8/1.4.6/pep8.py
* tests: Update misc/pep8.py to 1.4.5Dieter Verfaillie2013-05-071-27/+38
| | | | | | | | | | | | | | | | | Version in our tree is a wee bit outdated. For example, later work will introduce an utf8 encoded python source file which our old pep8.py does not yet understand (yeah, it really was *that* ancient)... Updated from: https://raw.github.com/jcrocholl/pep8/1.4.5/pep8.py Takes 552c1f1525e37a30376790151c1ba437776682c5, f941537d1c0a40f0906490ed160db6c79af572d3, 5a4afe2a77d0ff7d9fea13dd93c3304a6ca993de and a17f157e19bd6792c00321c8020dca5e5a281f45 into account... https://bugzilla.gnome.org/show_bug.cgi?id=699535
* annotationparser: Remove get_tag/get_paramJasper St. Pierre2013-05-071-6/+0
| | | | | | They're useless if we can just access the dict directly. https://bugzilla.gnome.org/show_bug.cgi?id=688897
* giscanner: move odict.OrderedDict into a collections packageDieter Verfaillie2013-05-071-1/+1
| | | | | | because we'll add more related code to collections later on https://bugzilla.gnome.org/show_bug.cgi?id=699536
* giscanner: use SqlAlchemy's OrderedDict implementationDieter Verfaillie2013-04-091-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | g-ir-scanner can be a bit on the slow side. While true we now do a bit more work parsing GTK-Doc comment blocks and more is still to come, one of the biggest hotspots besides what's going on in _giscanner.SourceScanner() comes from the OrderedDict implementations we have been using. For example, time needed to build Gtk-3.0.gir on a relatively slow machine using "python2 -m cProfile -o $prefix/bin/g-ir-scanner ...": 1) Our original DictMixin sublass: 92,79867 seconds 2) Python's collections.OrderedDict class: 88,65786 seconds 3) Larosa/Foord implementation from http://www.voidspace.org.uk/python/odict.html : 71,64323 seconds 4) SqlAlchemy's implementation: 66,12449 seconds Looks like we have a clear winner with the SqlAclchemy implementation, which comes in at around 20 seconds without profiling on the same machine. Not bad. https://bugzilla.gnome.org/show_bug.cgi?id=697620
* giscanner: fix string formatting tuplesDieter Verfaillie2013-04-091-10/+10
| | | | | | | | | | | | | | | | | | | | During review Colin Walters mentioned: """ One thing I should probably explain is my habit of *always* passing a tuple for %. The reason is pretty simple...a long time ago I had an error handler that did: print "unexpected data: %s" % foo. And I'd changed the type of "foo" from a string to a tuple at some point. But this obviously caused "print" to fail because it was expecting multiple format strings. Basically always using tuples is safer in case you change the type of a variable to a tuple. """ This fixes violations of this rule introduced up until now. https://bugzilla.gnome.org/show_bug.cgi?id=697614
* giscanner: emit a warning when we fail to parse a GTK-Doc comment blockDieter Verfaillie2013-01-091-1/+7
| | | | | | | | | | | Instead of going down with a for the user inexplicable backtrace, emit a warning asking said user to file a bug including the comment block in question. Thanks to Alexandre Rostovtsev <tetromino@gentoo.org> for proposing something similar in bug #690850 https://bugzilla.gnome.org/show_bug.cgi?id=690850
* giscanner: Don't fail on empty GTK-Doc comment blocksDieter Verfaillie2013-01-091-11/+14
| | | | | | | | | | A completely empty GTK-Doc comment block (/**\n*/) resulted in an unfriendly backtrace, complaining about an "AttributeError: 'NoneType' object has no attribute 'comment'" This fixes the issue and adds a test case. https://bugzilla.gnome.org/show_bug.cgi?id=690850
View Lorry Controller Status