diff options
| author | Andy Li <andy@onthewings.net> | 2013-02-19 02:30:44 +0800 |
|---|---|---|
| committer | Andy Li <andy@onthewings.net> | 2013-02-19 02:30:44 +0800 |
| commit | 565a8ec40162f20a004b826ad339c655f696b45e (patch) | |
| tree | a4ba4fe5b8874e3305941613034e6f6548bdb6a9 | |
| parent | 25273ed077997a0ed8ca8bcf34be7b94e248c5cb (diff) | |
| parent | 054c464c70c115f476608a51bb0b4a931d1fa400 (diff) | |
| download | pygments-565a8ec40162f20a004b826ad339c655f696b45e.tar.gz | |
Merge upstream changes.
136 files changed, 10308 insertions, 1429 deletions
@@ -9,7 +9,8 @@ Other contributors, listed alphabetically, are: * Kumar Appaiah -- Debian control lexer * Ali Afshar -- image formatter * Andreas Amann -- AppleScript lexer -* Jeffrey Arnold -- R/S, Rd and BUGS lexers +* Timothy Armstrong -- Dart lexer fixes +* Jeffrey Arnold -- R/S, Rd, BUGS, Jags, and Stan lexers * Jeremy Ashkenas -- CoffeeScript lexer * Stefan Matthias Aust -- Smalltalk lexer * Ben Bangert -- Mako lexers @@ -25,6 +26,7 @@ Other contributors, listed alphabetically, are: * Hiram Chirino -- Scaml and Jade lexers * Ian Cooper -- VGL lexer * Leaf Corcoran -- MoonScript lexer +* Christian Jann -- ShellSession lexer * Christopher Creutzig -- MuPAD lexer * Pete Curry -- bugfixes * Owen Durni -- Haxe lexer @@ -33,29 +35,34 @@ Other contributors, listed alphabetically, are: * Artem Egorkine -- terminal256 formatter * James H. Fisher -- PostScript lexer * Carlos Galdino -- Elixir and Elixir Console lexers +* Michael Galloy -- IDL lexer * Naveen Garg -- Autohotkey lexer * Laurent Gautier -- R/S lexer * Alex Gaynor -- PyPy log lexer +* Alain Gilbert -- TypeScript lexer * Bertrand Goetzmann -- Groovy lexer * Krzysiek Goj -- Scala lexer * Matt Good -- Genshi, Cheetah lexers * Patrick Gotthardt -- PHP namespaces support * Olivier Guibe -- Asymptote lexer +* Jordi Gutiérrez Hermoso -- Octave lexer * Martin Harriman -- SNOBOL lexer * Matthew Harrison -- SVG formatter * Steven Hazel -- Tcl lexer * Aslak Hellesøy -- Gherkin lexer * Greg Hendershott -- Racket lexer -* Jordi Gutiérrez Hermoso -- Octave lexer * David Hess, Fish Software, Inc. -- Objective-J lexer * Varun Hiremath -- Debian control lexer * Doug Hogan -- Mscgen lexer * Ben Hollis -- Mason lexer +* Dustin Howett -- Logos lexer +* Alastair Houghton -- Lexer inheritance facility * Tim Howard -- BlitzMax lexer * Ivan Inozemtsev -- Fantom lexer * Brian R. Jackson -- Tea lexer * Dennis Kaarsemaker -- sources.list lexer * Igor Kalnitsky -- vhdl lexer +* Pekka Klärck -- Robot Framework lexer * Eric Knibbe -- Lasso lexer * Adam Koprowski -- Opa lexer * Benjamin Kowarsch -- Modula-2 lexer @@ -63,19 +70,23 @@ Other contributors, listed alphabetically, are: * Marek Kubica -- Scheme lexer * Jochen Kupperschmidt -- Markdown processor * Gerd Kurzbach -- Modelica lexer +* Jon Larimer, Google Inc. -- Smali lexer * Olov Lassus -- Dart lexer * Sylvestre Ledru -- Scilab lexer * Mark Lee -- Vala lexer * Ben Mabey -- Gherkin lexer +* Angus MacArthur -- QML lexer * Simone Margaritelli -- Hybris lexer * Kirk McDonald -- D lexer * Gordon McGregor -- SystemVerilog lexer * Stephen McKamey -- Duel/JBST lexer * Brian McKenna -- F# lexer +* Charles McLaughlin -- Puppet lexer * Lukas Meuser -- BBCode formatter, Lua lexer * Paul Miller -- LiveScript lexer * Hong Minhee -- HTTP lexer * Michael Mior -- Awk lexer +* Bruce Mitchener -- Dylan lexer rewrite * Reuben Morais -- SourcePawn lexer * Jon Morton -- Rust lexer * Paulo Moura -- Logtalk lexer @@ -103,17 +114,22 @@ Other contributors, listed alphabetically, are: * Joe Schafer -- Ada lexer * Ken Schutte -- Matlab lexers * Tassilo Schweyer -- Io, MOOCode lexers +* Ted Shaw -- AutoIt lexer * Joerg Sieker -- ABAP lexer * Robert Simmons -- Standard ML lexer * Kirill Simonov -- YAML lexer +* Alexander Smishlajev -- Visual FoxPro lexer * Steve Spigarelli -- XQuery lexer * Jerome St-Louis -- eC lexer * James Strachan -- Kotlin lexer +* Tom Stuart -- Treetop lexer * Tiberius Teng -- default style overhaul * Jeremy Thurgood -- Erlang, Squid config lexers +* Brian Tiffin -- OpenCOBOL lexer * Erick Tryzelaar -- Felix lexer * Daniele Varrazzo -- PostgreSQL lexers * Abe Voelker -- OpenEdge ABL lexer +* Pepijn de Vos -- HTML formatter CTags support * Whitney Young -- ObjectiveC lexer * Matthias Vallentin -- Bro lexer * Nathan Weizenbaum -- Haml and Sass lexers @@ -2,34 +2,94 @@ Pygments changelog ================== Issue numbers refer to the tracker at -http://bitbucket.org/birkenfeld/pygments-main/issues. +<http://bitbucket.org/birkenfeld/pygments-main/issues>, +pull request numbers to the requests at +<http://bitbucket.org/birkenfeld/pygments-main/pull-requests/merged>. Version 1.6 ----------- -(in development) +(released Feb 3, 2013) - Lexers added: - * Julia (PR#61) - * Croc (new name for MiniD) - * LiveScript (PR#84) - * Lasso (PR#95) + * Dylan console (PR#149) + * Logos (PR#150) + * Shell sessions (PR#158) + +- Fix guessed lexers not receiving lexer options (#838). + +- Fix unquoted HTML attribute lexing in Opa (#841). + +- Fixes to the Dart lexer (PR#160). + + +Version 1.6rc1 +-------------- +(released Jan 9, 2013) + +- Lexers added: + + * AspectJ (PR#90) + * AutoIt (PR#122) * BUGS-like languages (PR#89) - * Rust (PR#67) + * Ceylon (PR#86) + * Croc (new name for MiniD) * CUDA (PR#75) - * Xtend (PR#68) - * Mscgen (PR#80) + * Dg (PR#116) + * IDL (PR#115) + * Jags (PR#89) + * Julia (PR#61) * Kconfig (#711) - * AspectJ (PR#90) + * Lasso (PR#95, PR#113) + * LiveScript (PR#84) + * Monkey (PR#117) + * Mscgen (PR#80) + * NSIS scripts (PR#136) + * OpenCOBOL (PR#72) + * QML (PR#123) + * Puppet (PR#133) * Racket (PR#94) * Rdoc (PR#99) - * VGL (PR#12) + * Robot Framework (PR#137) + * RPM spec files (PR#124) + * Rust (PR#67) + * Smali (Dalvik assembly) * SourcePawn (PR#39) - * Ceylon (PR#86) + * Stan (PR#89) + * Treetop (PR#125) + * TypeScript (PR#114) + * VGL (PR#12) + * Visual FoxPro (#762) + * Windows Registry (#819) + * Xtend (PR#68) + +- The HTML formatter now supports linking to tags using CTags files, when the + python-ctags package is installed (PR#87). + +- The HTML formatter now has a "linespans" option that wraps every line in a + <span> tag with a specific id (PR#82). + +- When deriving a lexer from another lexer with token definitions, definitions + for states not in the child lexer are now inherited. If you override a state + in the child lexer, an "inherit" keyword has been added to insert the base + state at that position (PR#141). + +- The C family lexers now inherit token definitions from a common base class, + removing code duplication (PR#141). + +- Use "colorama" on Windows for console color output (PR#142). + +- Fix Template Haskell highlighting (PR#63). + +- Fix some S/R lexer errors (PR#91). + +- Fix a bug in the Prolog lexer with names that start with 'is' (#810). + +- Rewrite Dylan lexer, add Dylan LID lexer (PR#147). -- Fix Template Haskell highlighting (PR#63) +- Add a Java quickstart document (PR#146). -- Fix some S/R lexer errors (PR#91) +- Add a "external/autopygmentize" file that can be used as .lessfilter (#802). Version 1.5 @@ -1,4 +1,4 @@ -Copyright (c) 2006-2012 by the respective authors (see AUTHORS file). +Copyright (c) 2006-2013 by the respective authors (see AUTHORS file). All rights reserved. Redistribution and use in source and binary forms, with or without @@ -4,7 +4,7 @@ # # Combines scripts for common tasks. # -# :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. +# :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. # :license: BSD, see LICENSE for details. # diff --git a/docs/generate.py b/docs/generate.py index 0ceeb58d..f5405074 100755 --- a/docs/generate.py +++ b/docs/generate.py @@ -6,7 +6,7 @@ Generates a bunch of html files containing the documentation. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/docs/src/api.txt b/docs/src/api.txt index b8159379..4276eea2 100644 --- a/docs/src/api.txt +++ b/docs/src/api.txt @@ -64,7 +64,7 @@ def `guess_lexer(text, **options):` def `guess_lexer_for_filename(filename, text, **options):` As `guess_lexer()`, but only lexers which have a pattern in `filenames` or `alias_filenames` that matches `filename` are taken into consideration. - + `pygments.util.ClassNotFound` is raised if no lexer thinks it can handle the content. diff --git a/docs/src/index.txt b/docs/src/index.txt index 1fad0f03..b1e099c7 100644 --- a/docs/src/index.txt +++ b/docs/src/index.txt @@ -37,7 +37,7 @@ Welcome to the Pygments documentation. - `Write your own lexer <lexerdevelopment.txt>`_ - `Write your own formatter <formatterdevelopment.txt>`_ - + - `Write your own filter <filterdevelopment.txt>`_ - `Register plugins <plugins.txt>`_ diff --git a/docs/src/integrate.txt b/docs/src/integrate.txt index 51a3dac4..6f8c1253 100644 --- a/docs/src/integrate.txt +++ b/docs/src/integrate.txt @@ -41,3 +41,8 @@ Bash completion The source distribution contains a file ``external/pygments.bashcomp`` that sets up completion for the ``pygmentize`` command in bash. + +Java +---- + +See the `Java quickstart <java.txt>`_ document. diff --git a/docs/src/java.txt b/docs/src/java.txt new file mode 100644 index 00000000..5eb6196a --- /dev/null +++ b/docs/src/java.txt @@ -0,0 +1,70 @@ +===================== +Use Pygments in Java +===================== + +Thanks to `Jython <http://www.jython.org>`__ it is possible to use Pygments in +Java. + +This page is a simple tutorial to get an idea of how this is working. You can +then look at the `Jython documentation <http://www.jython.org/docs/>`__ for more +advanced use. + +Since version 1.5, Pygments is deployed on `Maven Central +<http://repo1.maven.org/maven2/org/pygments/pygments/>`__ as a JAR so is Jython +which makes it a lot easier to create the Java project. + +Here is an example of a `Maven <http://www.maven.org>`__ ``pom.xml`` file for a +project running Pygments: + +.. sourcecode:: xml + + <?xml version="1.0" encoding="UTF-8"?> + + <project xmlns="http://maven.apache.org/POM/4.0.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 + http://maven.apache.org/maven-v4_0_0.xsd"> + <modelVersion>4.0.0</modelVersion> + <groupId>example</groupId> + <artifactId>example</artifactId> + <version>1.0-SNAPSHOT</version> + <dependencies> + <dependency> + <groupId>org.python</groupId> + <artifactId>jython-standalone</artifactId> + <version>2.5.3</version> + </dependency> + <dependency> + <groupId>org.pygments</groupId> + <artifactId>pygments</artifactId> + <version>1.5</version> + <scope>runtime</scope> + </dependency> + </dependencies> + </project> + +The following Java example: + +.. sourcecode:: java + + PythonInterpreter interpreter = new PythonInterpreter(); + + // Set a variable with the content you want to work with + interpreter.set("code", code); + + // Simple use Pygments as you would in Python + interpreter.exec("from pygments import highlight\n" + + "from pygments.lexers import PythonLexer\n" + + "from pygments.formatters import HtmlFormatter\n" + + "\nresult = highlight(code, PythonLexer(), HtmlFormatter())"); + + // Get the result that has been set in a variable + System.out.println(interpreter.get("result", String.class)); + +will print something like: + +.. sourcecode:: html + + <div class="highlight"> + <pre><span class="k">print</span> <span class="s">"Hello World"</span></pre> + </div> diff --git a/external/autopygmentize b/external/autopygmentize new file mode 100755 index 00000000..85c8dfd2 --- /dev/null +++ b/external/autopygmentize @@ -0,0 +1,64 @@ +#!/bin/sh +# Best effort auto-pygmentization with transparent decompression +# (c) Reuben Thomas 2012-2013 +# This program is in the public domain. + +# Strategy: first see if pygmentize can find a lexer; if not, ask file; if that finds nothing, fail +# Set the environment variable PYGMENTIZE_OPTS to configure pygments. + +# This program can be used as a .lessfilter for the less pager to auto-color less's output + +lexer=`pygmentize -N "$1"` +if [ "$lexer" = "text" ]; then + file_common_opts="--brief --dereference --uncompress" + + unset lexer + case `file --mime-type $file_common_opts "$1"` in + application/xml|image/svg+xml) lexer=xml;; + text/html) lexer=html;; + text/troff) lexer=nroff;; + text/x-asm) lexer=nasm;; + text/x-awk) lexer=awk;; + text/x-c) lexer=c;; + text/x-c++) lexer=cpp;; + text/x-diff) lexer=diff;; + text/x-fortran) lexer=fortran;; + text/x-gawk) lexer=gawk;; + text/x-java) lexer=java;; + text/x-lisp) lexer=common-lisp;; + text/x-lua) lexer=lua;; + text/x-makefile) lexer=make;; + text/x-msdos-batch) lexer=bat;; + text/x-nawk) lexer=nawk;; + text/x-pascal) lexer=pascal;; + text/x-perl) lexer=perl;; + text/x-php) lexer=php;; + text/x-po) lexer=po;; + text/x-python) lexer=python;; + text/x-ruby) lexer=ruby;; + text/x-shellscript) lexer=sh;; + text/x-tcl) lexer=tcl;; + text/x-tex|text/x-texinfo) lexer=latex;; # FIXME: texinfo really needs its own lexer + + # Types that file outputs which pygmentize didn't support as of file 5.11, pygments 1.6rc1 + # text/calendar + # text/PGP + # text/rtf + # text/texmacs + # text/x-bcpl + # text/x-info + # text/x-m4 + # text/x-vcard + # text/x-xmcd + esac +fi + +encoding=`file --brief --mime-encoding $file_common_opts "$1"` + +if [ -n "$lexer" ]; then + # FIXME: Specify input encoding rather than output encoding https://bitbucket.org/birkenfeld/pygments-main/issue/800 + zcat "$1" | pygmentize -O encoding=$encoding,outencoding=UTF-8 $PYGMENTIZE_OPTS -l $lexer + exit 0 +fi + +exit 1 diff --git a/external/markdown-processor.py b/external/markdown-processor.py index 0d939b67..12e64680 100644 --- a/external/markdown-processor.py +++ b/external/markdown-processor.py @@ -27,7 +27,7 @@ .. _Markdown: http://www.freewisdom.org/projects/python-markdown/ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -64,4 +64,4 @@ class CodeBlockPreprocessor(TextPreprocessor): code = code.replace('\n\n', '\n \n').replace('\n', '<br />') return '\n\n<div class="code">%s</div>\n\n' % code return self.pattern.sub( - repl, lines)
\ No newline at end of file + repl, lines) diff --git a/external/moin-parser.py b/external/moin-parser.py index 0bb9791c..6544da1b 100644 --- a/external/moin-parser.py +++ b/external/moin-parser.py @@ -31,7 +31,7 @@ If you do not want to do that and are willing to accept larger HTML output, you can set the INLINESTYLES option below to True. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/external/rst-directive-old.py b/external/rst-directive-old.py index 8e703faa..a074536f 100644 --- a/external/rst-directive-old.py +++ b/external/rst-directive-old.py @@ -31,7 +31,7 @@ .. _directive documentation: http://docutils.sourceforge.net/docs/howto/rst-directives.html - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/external/rst-directive.py b/external/rst-directive.py index b54b03ef..5c04038d 100644 --- a/external/rst-directive.py +++ b/external/rst-directive.py @@ -31,7 +31,7 @@ .. _directive documentation: http://docutils.sourceforge.net/docs/howto/rst-directives.html - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/__init__.py b/pygments/__init__.py index 0ecc167d..2bfd8ba5 100644 --- a/pygments/__init__.py +++ b/pygments/__init__.py @@ -22,11 +22,11 @@ .. _Pygments tip: http://bitbucket.org/birkenfeld/pygments-main/get/tip.zip#egg=Pygments-dev - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ -__version__ = '1.5' +__version__ = '1.6' __docformat__ = 'restructuredtext' __all__ = ['lex', 'format', 'highlight'] diff --git a/pygments/cmdline.py b/pygments/cmdline.py index 1f14cf5d..c25204bf 100644 --- a/pygments/cmdline.py +++ b/pygments/cmdline.py @@ -5,7 +5,7 @@ Command line interface. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import sys @@ -192,6 +192,14 @@ def main(args=sys.argv): usage = USAGE % ((args[0],) * 6) + if sys.platform in ['win32', 'cygwin']: + try: + # Provide coloring under Windows, if possible + import colorama + colorama.init() + except ImportError: + pass + try: popts, args = getopt.getopt(args[1:], "l:f:F:o:O:P:LS:a:N:hVHg") except getopt.GetoptError, err: @@ -219,7 +227,7 @@ def main(args=sys.argv): return 0 if opts.pop('-V', None) is not None: - print 'Pygments version %s, (c) 2006-2012 by Georg Brandl.' % __version__ + print 'Pygments version %s, (c) 2006-2013 by Georg Brandl.' % __version__ return 0 # handle ``pygmentize -L`` @@ -370,9 +378,9 @@ def main(args=sys.argv): except ClassNotFound, err: if '-g' in opts: try: - lexer = guess_lexer(code) + lexer = guess_lexer(code, **parsed_opts) except ClassNotFound: - lexer = TextLexer() + lexer = TextLexer(**parsed_opts) else: print >>sys.stderr, 'Error:', err return 1 @@ -384,9 +392,9 @@ def main(args=sys.argv): if '-g' in opts: code = sys.stdin.read() try: - lexer = guess_lexer(code) + lexer = guess_lexer(code, **parsed_opts) except ClassNotFound: - lexer = TextLexer() + lexer = TextLexer(**parsed_opts) elif not lexer: print >>sys.stderr, 'Error: no lexer name given and reading ' + \ 'from stdin (try using -g or -l <lexer>)' diff --git a/pygments/console.py b/pygments/console.py index 3901bd18..c8dfbd1f 100644 --- a/pygments/console.py +++ b/pygments/console.py @@ -5,7 +5,7 @@ Format colored console output. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/filter.py b/pygments/filter.py index de1bf79c..0b9224f2 100644 --- a/pygments/filter.py +++ b/pygments/filter.py @@ -5,7 +5,7 @@ Module that implements the default filter. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/filters/__init__.py b/pygments/filters/__init__.py index b816c58c..f12d025c 100644 --- a/pygments/filters/__init__.py +++ b/pygments/filters/__init__.py @@ -6,7 +6,7 @@ Module containing filter lookup functions and default filters. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -331,7 +331,6 @@ class TokenMergeFilter(Filter): Filter.__init__(self, **options) def filter(self, lexer, stream): - output = [] current_type = None current_value = None for ttype, value in stream: diff --git a/pygments/formatter.py b/pygments/formatter.py index 73124846..4b69f2a4 100644 --- a/pygments/formatter.py +++ b/pygments/formatter.py @@ -5,7 +5,7 @@ Base formatter class. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/__init__.py b/pygments/formatters/__init__.py index da0fd51e..d842b96b 100644 --- a/pygments/formatters/__init__.py +++ b/pygments/formatters/__init__.py @@ -5,7 +5,7 @@ Pygments formatters. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import os.path diff --git a/pygments/formatters/_mapping.py b/pygments/formatters/_mapping.py index 9d3d7823..a423ba50 100755 --- a/pygments/formatters/_mapping.py +++ b/pygments/formatters/_mapping.py @@ -9,7 +9,7 @@ Do not alter the FORMATTERS dictionary by hand. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/bbcode.py b/pygments/formatters/bbcode.py index a0d50553..15faff6c 100644 --- a/pygments/formatters/bbcode.py +++ b/pygments/formatters/bbcode.py @@ -5,7 +5,7 @@ BBcode formatter. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/html.py b/pygments/formatters/html.py index 85a16ff8..06096930 100644 --- a/pygments/formatters/html.py +++ b/pygments/formatters/html.py @@ -5,18 +5,23 @@ Formatter for HTML output. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import os import sys +import os.path import StringIO from pygments.formatter import Formatter from pygments.token import Token, Text, STANDARD_TYPES from pygments.util import get_bool_opt, get_int_opt, get_list_opt, bytes +try: + import ctags +except ImportError: + ctags = None __all__ = ['HtmlFormatter'] @@ -141,6 +146,12 @@ class HtmlFormatter(Formatter): the style definitions inside a ``<style>`` tag, or in a separate file if the `cssfile` option is given. + When `tagsfile` is set to the path of a ctags index file, it is used to + generate hyperlinks from names to their definition. You must enable + `anchorlines` and run ctags with the `-n` option for this to work. The + `python-ctags` module from PyPI must be installed to use this feature; + otherwise a `RuntimeError` will be raised. + The `get_style_defs(arg='')` method of a `HtmlFormatter` returns a string containing CSS rules for the CSS classes used by the formatter. The argument `arg` can be used to specify additional CSS selectors that @@ -282,10 +293,27 @@ class HtmlFormatter(Formatter): output line in an anchor tag with a ``name`` of ``foo-linenumber``. This allows easy linking to certain lines. *New in Pygments 0.9.* + `linespans` + If set to a nonempty string, e.g. ``foo``, the formatter will wrap each + output line in a span tag with an ``id`` of ``foo-linenumber``. + This allows easy access to lines via javascript. *New in Pygments 1.6.* + `anchorlinenos` If set to `True`, will wrap line numbers in <a> tags. Used in combination with `linenos` and `lineanchors`. + `tagsfile` + If set to the path of a ctags file, wrap names in anchor tags that + link to their definitions. `lineanchors` should be used, and the + tags file should specify line numbers (see the `-n` option to ctags). + *New in Pygments 1.6.* + + `tagurlformat` + A string formatting pattern used to generate links to ctags definitions. + Available variables are `%(path)s`, `%(fname)s` and `%(fext)s`. + Defaults to an empty string, resulting in just `#prefix-number` links. + *New in Pygments 1.6.* + **Subclassing the HTML formatter** @@ -351,6 +379,14 @@ class HtmlFormatter(Formatter): self.prestyles = self._decodeifneeded(options.get('prestyles', '')) self.cssfile = self._decodeifneeded(options.get('cssfile', '')) self.noclobber_cssfile = get_bool_opt(options, 'noclobber_cssfile', False) + self.tagsfile = self._decodeifneeded(options.get('tagsfile', '')) + self.tagurlformat = self._decodeifneeded(options.get('tagurlformat', '')) + + if self.tagsfile: + if not ctags: + raise RuntimeError('The "ctags" package must to be installed ' + 'to be able to use the "tagsfile" feature.') + self._ctags = ctags.CTags(self.tagsfile) linenos = options.get('linenos', False) if linenos == 'inline': @@ -366,6 +402,7 @@ class HtmlFormatter(Formatter): self.nobackground = get_bool_opt(options, 'nobackground', False) self.lineseparator = options.get('lineseparator', '\n') self.lineanchors = options.get('lineanchors', '') + self.linespans = options.get('linespans', '') self.anchorlinenos = options.get('anchorlinenos', False) self.hl_lines = set() for lineno in get_list_opt(options, 'hl_lines', []): @@ -596,7 +633,8 @@ class HtmlFormatter(Formatter): def _wrap_lineanchors(self, inner): s = self.lineanchors - i = self.linenostart - 1 # subtract 1 since we have to increment i *before* yielding + i = self.linenostart - 1 # subtract 1 since we have to increment i + # *before* yielding for t, line in inner: if t: i += 1 @@ -604,6 +642,16 @@ class HtmlFormatter(Formatter): else: yield 0, line + def _wrap_linespans(self, inner): + s = self.linespans + i = self.linenostart - 1 + for t, line in inner: + if t: + i += 1 + yield 1, '<span id="%s-%d">%s</span>' % (s, i, line) + else: + yield 0, line + def _wrap_div(self, inner): style = [] if (self.noclasses and not self.nobackground and @@ -643,6 +691,7 @@ class HtmlFormatter(Formatter): getcls = self.ttype2class.get c2s = self.class2style escape_table = _escape_html_table + tagsfile = self.tagsfile lspan = '' line = '' @@ -659,6 +708,19 @@ class HtmlFormatter(Formatter): parts = value.translate(escape_table).split('\n') + if tagsfile and ttype in Token.Name: + filename, linenumber = self._lookup_ctag(value) + if linenumber: + base, filename = os.path.split(filename) + if base: + base += '/' + filename, extension = os.path.splitext(filename) + url = self.tagurlformat % {'path': base, 'fname': filename, + 'fext': extension} + parts[0] = "<a href=\"%s#%s-%d\">%s" % \ + (url, self.lineanchors, linenumber, parts[0]) + parts[-1] = parts[-1] + "</a>" + # for all but the last line for part in parts[:-1]: if line: @@ -688,6 +750,13 @@ class HtmlFormatter(Formatter): if line: yield 1, line + (lspan and '</span>') + lsep + def _lookup_ctag(self, token): + entry = ctags.TagEntry() + if self._ctags.find(entry, token, 0): + return entry['file'], entry['lineNumber'] + else: + return None, None + def _highlight_lines(self, tokensource): """ Highlighted the lines specified in the `hl_lines` option by @@ -740,6 +809,8 @@ class HtmlFormatter(Formatter): source = self._wrap_inlinelinenos(source) if self.lineanchors: source = self._wrap_lineanchors(source) + if self.linespans: + source = self._wrap_linespans(source) source = self.wrap(source, outfile) if self.linenos == 1: source = self._wrap_tablelinenos(source) diff --git a/pygments/formatters/img.py b/pygments/formatters/img.py index 609aa54c..394c3b6a 100644 --- a/pygments/formatters/img.py +++ b/pygments/formatters/img.py @@ -5,7 +5,7 @@ Formatter for Pixmap output. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/latex.py b/pygments/formatters/latex.py index fcd5f379..47fd1239 100644 --- a/pygments/formatters/latex.py +++ b/pygments/formatters/latex.py @@ -5,7 +5,7 @@ Formatter for LaTeX fancyvrb output. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/other.py b/pygments/formatters/other.py index a19e4ce0..1029a7a7 100644 --- a/pygments/formatters/other.py +++ b/pygments/formatters/other.py @@ -5,7 +5,7 @@ Other formatters: NullFormatter, RawTokenFormatter. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -102,8 +102,6 @@ class RawTokenFormatter(Formatter): outfile.write(text.encode()) flush = outfile.flush - lasttype = None - lastval = u'' if self.error_color: for ttype, value in tokensource: line = "%s\t%r\n" % (ttype, value) diff --git a/pygments/formatters/rtf.py b/pygments/formatters/rtf.py index 5627d070..3efda284 100644 --- a/pygments/formatters/rtf.py +++ b/pygments/formatters/rtf.py @@ -5,7 +5,7 @@ A formatter that generates RTF files. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/svg.py b/pygments/formatters/svg.py index f08e6570..271f22a7 100644 --- a/pygments/formatters/svg.py +++ b/pygments/formatters/svg.py @@ -5,7 +5,7 @@ Formatter for SVG output. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/terminal.py b/pygments/formatters/terminal.py index dae00157..94e078f2 100644 --- a/pygments/formatters/terminal.py +++ b/pygments/formatters/terminal.py @@ -5,7 +5,7 @@ Formatter for terminal output with ANSI sequences. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/formatters/terminal256.py b/pygments/formatters/terminal256.py index cbd30be1..772ed423 100644 --- a/pygments/formatters/terminal256.py +++ b/pygments/formatters/terminal256.py @@ -11,7 +11,7 @@ Formatter version 1. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexer.py b/pygments/lexer.py index ad2c72d1..8f88dfda 100644 --- a/pygments/lexer.py +++ b/pygments/lexer.py @@ -5,10 +5,10 @@ Base lexer classes. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ -import re +import re, itertools from pygments.filter import apply_filters, Filter from pygments.filters import get_filter_by_name @@ -18,7 +18,7 @@ from pygments.util import get_bool_opt, get_int_opt, get_list_opt, \ __all__ = ['Lexer', 'RegexLexer', 'ExtendedRegexLexer', 'DelegatingLexer', - 'LexerContext', 'include', 'bygroups', 'using', 'this'] + 'LexerContext', 'include', 'inherit', 'bygroups', 'using', 'this'] _encoding_map = [('\xef\xbb\xbf', 'utf-8'), @@ -72,15 +72,18 @@ class Lexer(object): #: Shortcuts for the lexer aliases = [] - #: fn match rules + #: File name globs filenames = [] - #: fn alias filenames + #: Secondary file name globs alias_filenames = [] - #: mime types + #: MIME types mimetypes = [] + #: Priority, should multiple lexers match and no content is provided + priority = 0 + __metaclass__ = LexerMeta def __init__(self, **options): @@ -163,6 +166,10 @@ class Lexer(object): text = decoded else: text = text.decode(self.encoding) + else: + if text.startswith(u'\ufeff'): + text = text[len(u'\ufeff'):] + # text now *is* a unicode string text = text.replace('\r\n', '\n') text = text.replace('\r', '\n') @@ -238,6 +245,16 @@ class include(str): pass +class _inherit(object): + """ + Indicates the a state should inherit from its superclass. + """ + def __repr__(self): + return 'inherit' + +inherit = _inherit() + + class combined(tuple): """ Indicates a state combined from multiple states. @@ -428,6 +445,9 @@ class RegexLexerMeta(LexerMeta): tokens.extend(cls._process_state(unprocessed, processed, str(tdef))) continue + if isinstance(tdef, _inherit): + # processed already + continue assert type(tdef) is tuple, "wrong rule def %r" % tdef @@ -456,6 +476,49 @@ class RegexLexerMeta(LexerMeta): cls._process_state(tokendefs, processed, state) return processed + def get_tokendefs(cls): + """ + Merge tokens from superclasses in MRO order, returning a single tokendef + dictionary. + + Any state that is not defined by a subclass will be inherited + automatically. States that *are* defined by subclasses will, by + default, override that state in the superclass. If a subclass wishes to + inherit definitions from a superclass, it can use the special value + "inherit", which will cause the superclass' state definition to be + included at that point in the state. + """ + tokens = {} + inheritable = {} + for c in itertools.chain((cls,), cls.__mro__): + toks = c.__dict__.get('tokens', {}) + + for state, items in toks.iteritems(): + curitems = tokens.get(state) + if curitems is None: + tokens[state] = items + try: + inherit_ndx = items.index(inherit) + except ValueError: + continue + inheritable[state] = inherit_ndx + continue + + inherit_ndx = inheritable.pop(state, None) + if inherit_ndx is None: + continue + + # Replace the "inherit" value with the items + curitems[inherit_ndx:inherit_ndx+1] = items + try: + new_inh_ndx = items.index(inherit) + except ValueError: + pass + else: + inheritable[state] = inherit_ndx + new_inh_ndx + + return tokens + def __call__(cls, *args, **kwds): """Instantiate cls after preprocessing its token definitions.""" if '_tokens' not in cls.__dict__: @@ -465,7 +528,7 @@ class RegexLexerMeta(LexerMeta): # don't process yet pass else: - cls._tokens = cls.process_tokendef('', cls.tokens) + cls._tokens = cls.process_tokendef('', cls.get_tokendefs()) return type.__call__(cls, *args, **kwds) @@ -606,7 +669,13 @@ class ExtendedRegexLexer(RegexLexer): if new_state is not None: # state transition if isinstance(new_state, tuple): - ctx.stack.extend(new_state) + for state in new_state: + if state == '#pop': + ctx.stack.pop() + elif state == '#push': + ctx.stack.append(statestack[-1]) + else: + ctx.stack.append(state) elif isinstance(new_state, int): # pop del ctx.stack[new_state:] @@ -622,10 +691,10 @@ class ExtendedRegexLexer(RegexLexer): break if text[ctx.pos] == '\n': # at EOL, reset state to "root" - ctx.pos += 1 ctx.stack = ['root'] statetokens = tokendefs['root'] yield ctx.pos, Text, u'\n' + ctx.pos += 1 continue yield ctx.pos, Error, text[ctx.pos] ctx.pos += 1 @@ -694,4 +763,3 @@ def do_insertions(insertions, tokens): except StopIteration: insleft = False break # not strictly necessary - diff --git a/pygments/lexers/__init__.py b/pygments/lexers/__init__.py index eb8609d4..9af6ce68 100644 --- a/pygments/lexers/__init__.py +++ b/pygments/lexers/__init__.py @@ -5,7 +5,7 @@ Pygments lexers. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -113,7 +113,7 @@ def get_lexer_for_filename(_fn, code=None, **options): # to find lexers which need it overridden. if code: return cls.analyse_text(code) + bonus - return bonus + return cls.priority + bonus if matches: matches.sort(key=get_rating) diff --git a/pygments/lexers/_asybuiltins.py b/pygments/lexers/_asybuiltins.py index d2edace6..108fa199 100644 --- a/pygments/lexers/_asybuiltins.py +++ b/pygments/lexers/_asybuiltins.py @@ -10,7 +10,7 @@ TODO: perl/python script in Asymptote SVN similar to asy-list.pl but only for function and variable names. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/_clbuiltins.py b/pygments/lexers/_clbuiltins.py index 3a3b3dfc..59f948ba 100644 --- a/pygments/lexers/_clbuiltins.py +++ b/pygments/lexers/_clbuiltins.py @@ -5,7 +5,7 @@ ANSI Common Lisp builtins. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/_lassobuiltins.py b/pygments/lexers/_lassobuiltins.py index e609daf7..08b65f37 100644 --- a/pygments/lexers/_lassobuiltins.py +++ b/pygments/lexers/_lassobuiltins.py @@ -1,9 +1,12 @@ # -*- coding: utf-8 -*- """ pygments.lexers._lassobuiltins - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Built-in Lasso types, traits, and methods. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. """ BUILTINS = { diff --git a/pygments/lexers/_luabuiltins.py b/pygments/lexers/_luabuiltins.py index 27b39822..069c44fd 100644 --- a/pygments/lexers/_luabuiltins.py +++ b/pygments/lexers/_luabuiltins.py @@ -9,7 +9,7 @@ Do not edit the MODULES dict by hand. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/_mapping.py b/pygments/lexers/_mapping.py index ca0d0c79..82be5f4e 100644 --- a/pygments/lexers/_mapping.py +++ b/pygments/lexers/_mapping.py @@ -9,7 +9,7 @@ Do not alter the LEXERS dictionary by hand. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -31,6 +31,7 @@ LEXERS = { 'AppleScriptLexer': ('pygments.lexers.other', 'AppleScript', ('applescript',), ('*.applescript',), ()), 'AspectJLexer': ('pygments.lexers.jvm', 'AspectJ', ('aspectj',), ('*.aj',), ('text/x-aspectj',)), 'AsymptoteLexer': ('pygments.lexers.other', 'Asymptote', ('asy', 'asymptote'), ('*.asy',), ('text/x-asymptote',)), + 'AutoItLexer': ('pygments.lexers.other', 'AutoIt', ('autoit', 'Autoit'), ('*.au3',), ('text/x-autoit',)), 'AutohotkeyLexer': ('pygments.lexers.other', 'autohotkey', ('ahk',), ('*.ahk', '*.ahkl'), ('text/x-autohotkey',)), 'AwkLexer': ('pygments.lexers.other', 'Awk', ('awk', 'gawk', 'mawk', 'nawk'), ('*.awk',), ('application/x-awk',)), 'BBCodeLexer': ('pygments.lexers.text', 'BBCode', ('bbcode',), (), ('text/x-bbcode',)), @@ -49,7 +50,8 @@ LEXERS = { 'CObjdumpLexer': ('pygments.lexers.asm', 'c-objdump', ('c-objdump',), ('*.c-objdump',), ('text/x-c-objdump',)), 'CSharpAspxLexer': ('pygments.lexers.dotnet', 'aspx-cs', ('aspx-cs',), ('*.aspx', '*.asax', '*.ascx', '*.ashx', '*.asmx', '*.axd'), ()), 'CSharpLexer': ('pygments.lexers.dotnet', 'C#', ('csharp', 'c#'), ('*.cs',), ('text/x-csharp',)), - 'CUDALexer': ('pygments.lexers.compiled', 'CUDA', ('cuda', 'cu'), ('*.cu', '*.cuh'), ('text/x-cuda',)), + 'Ca65Lexer': ('pygments.lexers.asm', 'ca65', ('ca65',), ('*.s',), ()), + 'CbmBasicV2Lexer': ('pygments.lexers.other', 'CBM BASIC V2', ('cbmbas',), ('*.bas',), ()), 'CeylonLexer': ('pygments.lexers.jvm', 'Ceylon', ('ceylon',), ('*.ceylon',), ('text/x-ceylon',)), 'Cfengine3Lexer': ('pygments.lexers.other', 'CFEngine3', ('cfengine3', 'cf3'), ('*.cf',), ()), 'CheetahHtmlLexer': ('pygments.lexers.templates', 'HTML+Cheetah', ('html+cheetah', 'html+spitfire'), (), ('text/html+cheetah', 'text/html+spitfire')), @@ -57,12 +59,14 @@ LEXERS = { 'CheetahLexer': ('pygments.lexers.templates', 'Cheetah', ('cheetah', 'spitfire'), ('*.tmpl', '*.spt'), ('application/x-cheetah', 'application/x-spitfire')), 'CheetahXmlLexer': ('pygments.lexers.templates', 'XML+Cheetah', ('xml+cheetah', 'xml+spitfire'), (), ('application/xml+cheetah', 'application/xml+spitfire')), 'ClojureLexer': ('pygments.lexers.jvm', 'Clojure', ('clojure', 'clj'), ('*.clj',), ('text/x-clojure', 'application/x-clojure')), + 'CobolFreeformatLexer': ('pygments.lexers.compiled', 'COBOLFree', ('cobolfree',), ('*.cbl', '*.CBL'), ()), + 'CobolLexer': ('pygments.lexers.compiled', 'COBOL', ('cobol',), ('*.cob', '*.COB', '*.cpy', '*.CPY'), ('text/x-cobol',)), 'CoffeeScriptLexer': ('pygments.lexers.web', 'CoffeeScript', ('coffee-script', 'coffeescript'), ('*.coffee',), ('text/coffeescript',)), 'ColdfusionHtmlLexer': ('pygments.lexers.templates', 'Coldfusion HTML', ('cfm',), ('*.cfm', '*.cfml', '*.cfc'), ('application/x-coldfusion',)), 'ColdfusionLexer': ('pygments.lexers.templates', 'cfstatement', ('cfs',), (), ()), 'CommonLispLexer': ('pygments.lexers.functional', 'Common Lisp', ('common-lisp', 'cl'), ('*.cl', '*.lisp', '*.el'), ('text/x-common-lisp',)), 'CoqLexer': ('pygments.lexers.functional', 'Coq', ('coq',), ('*.v',), ('text/x-coq',)), - 'CppLexer': ('pygments.lexers.compiled', 'C++', ('cpp', 'c++'), ('*.cpp', '*.hpp', '*.c++', '*.h++', '*.cc', '*.hh', '*.cxx', '*.hxx'), ('text/x-c++hdr', 'text/x-c++src')), + 'CppLexer': ('pygments.lexers.compiled', 'C++', ('cpp', 'c++'), ('*.cpp', '*.hpp', '*.c++', '*.h++', '*.cc', '*.hh', '*.cxx', '*.hxx', '*.C', '*.H', '*.cp', '*.CPP'), ('text/x-c++hdr', 'text/x-c++src')), 'CppObjdumpLexer': ('pygments.lexers.asm', 'cpp-objdump', ('cpp-objdump', 'c++-objdumb', 'cxx-objdump'), ('*.cpp-objdump', '*.c++-objdump', '*.cxx-objdump'), ('text/x-cpp-objdump',)), 'CrocLexer': ('pygments.lexers.agile', 'Croc', ('croc',), ('*.croc',), ('text/x-crocsrc',)), 'CssDjangoLexer': ('pygments.lexers.templates', 'CSS+Django/Jinja', ('css+django', 'css+jinja'), (), ('text/css+django', 'text/css+jinja')), @@ -71,6 +75,7 @@ LEXERS = { 'CssLexer': ('pygments.lexers.web', 'CSS', ('css',), ('*.css',), ('text/css',)), 'CssPhpLexer': ('pygments.lexers.templates', 'CSS+PHP', ('css+php',), (), ('text/css+php',)), 'CssSmartyLexer': ('pygments.lexers.templates', 'CSS+Smarty', ('css+smarty',), (), ('text/css+smarty',)), + 'CudaLexer': ('pygments.lexers.compiled', 'CUDA', ('cuda', 'cu'), ('*.cu', '*.cuh'), ('text/x-cuda',)), 'CythonLexer': ('pygments.lexers.compiled', 'Cython', ('cython', 'pyx'), ('*.pyx', '*.pxd', '*.pxi'), ('text/x-cython', 'application/x-cython')), 'DLexer': ('pygments.lexers.compiled', 'D', ('d',), ('*.d', '*.di'), ('text/x-dsrc',)), 'DObjdumpLexer': ('pygments.lexers.asm', 'd-objdump', ('d-objdump',), ('*.d-objdump',), ('text/x-d-objdump',)), @@ -78,11 +83,14 @@ LEXERS = { 'DartLexer': ('pygments.lexers.web', 'Dart', ('dart',), ('*.dart',), ('text/x-dart',)), 'DebianControlLexer': ('pygments.lexers.text', 'Debian Control file', ('control',), ('control',), ()), 'DelphiLexer': ('pygments.lexers.compiled', 'Delphi', ('delphi', 'pas', 'pascal', 'objectpascal'), ('*.pas',), ('text/x-pascal',)), + 'DgLexer': ('pygments.lexers.agile', 'dg', ('dg',), ('*.dg',), ('text/x-dg',)), 'DiffLexer': ('pygments.lexers.text', 'Diff', ('diff', 'udiff'), ('*.diff', '*.patch'), ('text/x-diff', 'text/x-patch')), 'DjangoLexer': ('pygments.lexers.templates', 'Django/Jinja', ('django', 'jinja'), (), ('application/x-django-templating', 'application/x-jinja')), 'DtdLexer': ('pygments.lexers.web', 'DTD', ('dtd',), ('*.dtd',), ('application/xml-dtd',)), 'DuelLexer': ('pygments.lexers.web', 'Duel', ('duel', 'Duel Engine', 'Duel View', 'JBST', 'jbst', 'JsonML+BST'), ('*.duel', '*.jbst'), ('text/x-duel', 'text/x-jbst')), - 'DylanLexer': ('pygments.lexers.compiled', 'Dylan', ('dylan',), ('*.dylan', '*.dyl'), ('text/x-dylan',)), + 'DylanConsoleLexer': ('pygments.lexers.compiled', 'Dylan session', ('dylan-console', 'dylan-repl'), ('*.dylan-console',), ('text/x-dylan-console',)), + 'DylanLexer': ('pygments.lexers.compiled', 'Dylan', ('dylan',), ('*.dylan', '*.dyl', '*.intr'), ('text/x-dylan',)), + 'DylanLidLexer': ('pygments.lexers.compiled', 'DylanLID', ('dylan-lid', 'lid'), ('*.lid', '*.hdp'), ('text/x-dylan-lid',)), 'ECLLexer': ('pygments.lexers.other', 'ECL', ('ecl',), ('*.ecl',), ('application/x-ecl',)), 'ECLexer': ('pygments.lexers.compiled', 'eC', ('ec',), ('*.ec', '*.eh'), ('text/x-echdr', 'text/x-ecsrc')), 'ElixirConsoleLexer': ('pygments.lexers.functional', 'Elixir iex session', ('iex',), (), ('text/x-elixir-shellsession',)), @@ -99,6 +107,7 @@ LEXERS = { 'FantomLexer': ('pygments.lexers.compiled', 'Fantom', ('fan',), ('*.fan',), ('application/x-fantom',)), 'FelixLexer': ('pygments.lexers.compiled', 'Felix', ('felix', 'flx'), ('*.flx', '*.flxh'), ('text/x-felix',)), 'FortranLexer': ('pygments.lexers.compiled', 'Fortran', ('fortran',), ('*.f', '*.f90', '*.F', '*.F90'), ('text/x-fortran',)), + 'FoxProLexer': ('pygments.lexers.foxpro', 'FoxPro', ('Clipper', 'XBase'), ('*.PRG', '*.prg'), ()), 'GLShaderLexer': ('pygments.lexers.compiled', 'GLSL', ('glsl',), ('*.vert', '*.frag', '*.geo'), ('text/x-glslsrc',)), 'GasLexer': ('pygments.lexers.asm', 'GAS', ('gas',), ('*.s', '*.S'), ('text/x-gas',)), 'GenshiLexer': ('pygments.lexers.templates', 'Genshi', ('genshi', 'kid', 'xml+genshi', 'xml+kid'), ('*.kid',), ('application/x-genshi', 'application/x-kid')), @@ -121,7 +130,9 @@ LEXERS = { 'HtmlPhpLexer': ('pygments.lexers.templates', 'HTML+PHP', ('html+php',), ('*.phtml',), ('application/x-php', 'application/x-httpd-php', 'application/x-httpd-php3', 'application/x-httpd-php4', 'application/x-httpd-php5')), 'HtmlSmartyLexer': ('pygments.lexers.templates', 'HTML+Smarty', ('html+smarty',), (), ('text/html+smarty',)), 'HttpLexer': ('pygments.lexers.text', 'HTTP', ('http',), (), ()), + 'HxmlLexer': ('pygments.lexers.text', 'Hxml', ('haxeml', 'hxml'), ('*.hxml',), ()), 'HybrisLexer': ('pygments.lexers.other', 'Hybris', ('hybris', 'hy'), ('*.hy', '*.hyb'), ('text/x-hybris', 'application/x-hybris')), + 'IDLLexer': ('pygments.lexers.math', 'IDL', ('idl',), ('*.pro',), ('text/idl',)), 'IniLexer': ('pygments.lexers.text', 'INI', ('ini', 'cfg'), ('*.ini', '*.cfg'), ('text/x-ini',)), 'IoLexer': ('pygments.lexers.agile', 'Io', ('io',), ('*.io',), ('text/x-iosrc',)), 'IokeLexer': ('pygments.lexers.jvm', 'Ioke', ('ioke', 'ik'), ('*.ik',), ('text/x-iokesrc',)), @@ -151,6 +162,7 @@ LEXERS = { 'LiterateHaskellLexer': ('pygments.lexers.functional', 'Literate Haskell', ('lhs', 'literate-haskell'), ('*.lhs',), ('text/x-literate-haskell',)), 'LiveScriptLexer': ('pygments.lexers.web', 'LiveScript', ('live-script', 'livescript'), ('*.ls',), ('text/livescript',)), 'LlvmLexer': ('pygments.lexers.asm', 'LLVM', ('llvm',), ('*.ll',), ('text/x-llvm',)), + 'LogosLexer': ('pygments.lexers.compiled', 'Logos', ('logos',), ('*.x', '*.xi', '*.xm', '*.xmi'), ('text/x-logos',)), 'LogtalkLexer': ('pygments.lexers.other', 'Logtalk', ('logtalk',), ('*.lgt',), ('text/x-logtalk',)), 'LuaLexer': ('pygments.lexers.agile', 'Lua', ('lua',), ('*.lua', '*.wlua'), ('text/x-lua', 'application/x-lua')), 'MOOCodeLexer': ('pygments.lexers.other', 'MOOCode', ('moocode',), ('*.moo',), ('text/x-moocode',)), @@ -168,6 +180,7 @@ LEXERS = { 'ModelicaLexer': ('pygments.lexers.other', 'Modelica', ('modelica',), ('*.mo',), ('text/x-modelica',)), 'Modula2Lexer': ('pygments.lexers.compiled', 'Modula-2', ('modula2', 'm2'), ('*.def', '*.mod'), ('text/x-modula2',)), 'MoinWikiLexer': ('pygments.lexers.text', 'MoinMoin/Trac Wiki markup', ('trac-wiki', 'moin'), (), ('text/x-trac-wiki',)), + 'MonkeyLexer': ('pygments.lexers.compiled', 'Monkey', ('monkey',), ('*.monkey',), ('text/x-monkey',)), 'MoonScriptLexer': ('pygments.lexers.agile', 'MoonScript', ('moon', 'moonscript'), ('*.moon',), ('text/x-moonscript', 'application/x-moonscript')), 'MscgenLexer': ('pygments.lexers.other', 'Mscgen', ('mscgen', 'msc'), ('*.msc',), ()), 'MuPADLexer': ('pygments.lexers.math', 'MuPAD', ('mupad',), ('*.mu',), ()), @@ -178,6 +191,7 @@ LEXERS = { 'MyghtyJavascriptLexer': ('pygments.lexers.templates', 'JavaScript+Myghty', ('js+myghty', 'javascript+myghty'), (), ('application/x-javascript+myghty', 'text/x-javascript+myghty', 'text/javascript+mygthy')), 'MyghtyLexer': ('pygments.lexers.templates', 'Myghty', ('myghty',), ('*.myt', 'autodelegate'), ('application/x-myghty',)), 'MyghtyXmlLexer': ('pygments.lexers.templates', 'XML+Myghty', ('xml+myghty',), (), ('application/xml+myghty',)), + 'NSISLexer': ('pygments.lexers.other', 'NSIS', ('nsis', 'nsi', 'nsh'), ('*.nsi', '*.nsh'), ('text/x-nsis',)), 'NasmLexer': ('pygments.lexers.asm', 'NASM', ('nasm',), ('*.asm', '*.ASM'), ('text/x-nasm',)), 'NemerleLexer': ('pygments.lexers.dotnet', 'Nemerle', ('nemerle',), ('*.n',), ('text/x-nemerle',)), 'NewLispLexer': ('pygments.lexers.functional', 'NewLisp', ('newlisp',), ('*.lsp', '*.nl'), ('text/x-newlisp', 'application/x-newlisp')), @@ -186,7 +200,8 @@ LEXERS = { 'NimrodLexer': ('pygments.lexers.compiled', 'Nimrod', ('nimrod', 'nim'), ('*.nim', '*.nimrod'), ('text/x-nimrod',)), 'NumPyLexer': ('pygments.lexers.math', 'NumPy', ('numpy',), (), ()), 'ObjdumpLexer': ('pygments.lexers.asm', 'objdump', ('objdump',), ('*.objdump',), ('text/x-objdump',)), - 'ObjectiveCLexer': ('pygments.lexers.compiled', 'Objective-C', ('objective-c', 'objectivec', 'obj-c', 'objc'), ('*.m',), ('text/x-objective-c',)), + 'ObjectiveCLexer': ('pygments.lexers.compiled', 'Objective-C', ('objective-c', 'objectivec', 'obj-c', 'objc'), ('*.m', '*.h'), ('text/x-objective-c',)), + 'ObjectiveCppLexer': ('pygments.lexers.compiled', 'Objective-C++', ('objective-c++', 'objectivec++', 'obj-c++', 'objc++'), ('*.mm', '*.hh'), ('text/x-objective-c++',)), 'ObjectiveJLexer': ('pygments.lexers.web', 'Objective-J', ('objective-j', 'objectivej', 'obj-j', 'objj'), ('*.j',), ('text/x-objective-j',)), 'OcamlLexer': ('pygments.lexers.functional', 'OCaml', ('ocaml',), ('*.ml', '*.mli', '*.mll', '*.mly'), ('text/x-ocaml',)), 'OctaveLexer': ('pygments.lexers.math', 'Octave', ('octave',), ('*.m',), ('text/octave',)), @@ -194,7 +209,7 @@ LEXERS = { 'OpaLexer': ('pygments.lexers.functional', 'Opa', ('opa',), ('*.opa',), ('text/x-opa',)), 'OpenEdgeLexer': ('pygments.lexers.other', 'OpenEdge ABL', ('openedge', 'abl', 'progress'), ('*.p', '*.cls'), ('text/x-openedge', 'application/x-openedge')), 'PerlLexer': ('pygments.lexers.agile', 'Perl', ('perl', 'pl'), ('*.pl', '*.pm'), ('text/x-perl', 'application/x-perl')), - 'PhpLexer': ('pygments.lexers.web', 'PHP', ('php', 'php3', 'php4', 'php5'), ('*.php', '*.php[345]'), ('text/x-php',)), + 'PhpLexer': ('pygments.lexers.web', 'PHP', ('php', 'php3', 'php4', 'php5'), ('*.php', '*.php[345]', '*.inc'), ('text/x-php',)), 'PlPgsqlLexer': ('pygments.lexers.sql', 'PL/pgSQL', ('plpgsql',), (), ('text/x-plpgsql',)), 'PostScriptLexer': ('pygments.lexers.other', 'PostScript', ('postscript',), ('*.ps', '*.eps'), ('application/postscript',)), 'PostgresConsoleLexer': ('pygments.lexers.sql', 'PostgreSQL console (psql)', ('psql', 'postgresql-console', 'postgres-console'), (), ('text/x-postgresql-psql',)), @@ -204,13 +219,16 @@ LEXERS = { 'PrologLexer': ('pygments.lexers.compiled', 'Prolog', ('prolog',), ('*.prolog', '*.pro', '*.pl'), ('text/x-prolog',)), 'PropertiesLexer': ('pygments.lexers.text', 'Properties', ('properties',), ('*.properties',), ('text/x-java-properties',)), 'ProtoBufLexer': ('pygments.lexers.other', 'Protocol Buffer', ('protobuf',), ('*.proto',), ()), + 'PuppetLexer': ('pygments.lexers.other', 'Puppet', ('puppet',), ('*.pp',), ()), 'PyPyLogLexer': ('pygments.lexers.text', 'PyPy Log', ('pypylog', 'pypy'), ('*.pypylog',), ('application/x-pypylog',)), 'Python3Lexer': ('pygments.lexers.agile', 'Python 3', ('python3', 'py3'), (), ('text/x-python3', 'application/x-python3')), 'Python3TracebackLexer': ('pygments.lexers.agile', 'Python 3.0 Traceback', ('py3tb',), ('*.py3tb',), ('text/x-python3-traceback',)), 'PythonConsoleLexer': ('pygments.lexers.agile', 'Python console session', ('pycon',), (), ('text/x-python-doctest',)), 'PythonLexer': ('pygments.lexers.agile', 'Python', ('python', 'py', 'sage'), ('*.py', '*.pyw', '*.sc', 'SConstruct', 'SConscript', '*.tac', '*.sage'), ('text/x-python', 'application/x-python')), 'PythonTracebackLexer': ('pygments.lexers.agile', 'Python Traceback', ('pytb',), ('*.pytb',), ('text/x-python-traceback',)), + 'QmlLexer': ('pygments.lexers.web', 'QML', ('qml', 'Qt Meta Language', 'Qt modeling Language'), ('*.qml',), ('application/x-qml',)), 'RConsoleLexer': ('pygments.lexers.math', 'RConsole', ('rconsole', 'rout'), ('*.Rout',), ()), + 'RPMSpecLexer': ('pygments.lexers.other', 'RPMSpec', ('spec',), ('*.spec',), ('text/x-rpm-spec',)), 'RacketLexer': ('pygments.lexers.functional', 'Racket', ('racket', 'rkt'), ('*.rkt', '*.rktl'), ('text/x-racket', 'application/x-racket')), 'RagelCLexer': ('pygments.lexers.parsers', 'Ragel in C Host', ('ragel-c',), ('*.rl',), ()), 'RagelCppLexer': ('pygments.lexers.parsers', 'Ragel in CPP Host', ('ragel-cpp',), ('*.rl',), ()), @@ -224,7 +242,9 @@ LEXERS = { 'RdLexer': ('pygments.lexers.math', 'Rd', ('rd',), ('*.Rd',), ('text/x-r-doc',)), 'RebolLexer': ('pygments.lexers.other', 'REBOL', ('rebol',), ('*.r', '*.r3'), ('text/x-rebol',)), 'RedcodeLexer': ('pygments.lexers.other', 'Redcode', ('redcode',), ('*.cw',), ()), + 'RegeditLexer': ('pygments.lexers.text', 'reg', ('registry',), ('*.reg',), ('text/x-windows-registry',)), 'RhtmlLexer': ('pygments.lexers.templates', 'RHTML', ('rhtml', 'html+erb', 'html+ruby'), ('*.rhtml',), ('text/html+ruby',)), + 'RobotFrameworkLexer': ('pygments.lexers.other', 'RobotFramework', ('RobotFramework', 'robotframework'), ('*.txt', '*.robot'), ('text/x-robotframework',)), 'RstLexer': ('pygments.lexers.text', 'reStructuredText', ('rst', 'rest', 'restructuredtext'), ('*.rst', '*.rest'), ('text/x-rst', 'text/prs.fallenstein.rst')), 'RubyConsoleLexer': ('pygments.lexers.agile', 'Ruby irb session', ('rbcon', 'irb'), (), ('text/x-ruby-shellsession',)), 'RubyLexer': ('pygments.lexers.agile', 'Ruby', ('rb', 'ruby', 'duby'), ('*.rb', '*.rbw', 'Rakefile', '*.rake', '*.gemspec', '*.rbx', '*.duby'), ('text/x-ruby', 'application/x-ruby')), @@ -237,6 +257,8 @@ LEXERS = { 'SchemeLexer': ('pygments.lexers.functional', 'Scheme', ('scheme', 'scm'), ('*.scm', '*.ss'), ('text/x-scheme', 'application/x-scheme')), 'ScilabLexer': ('pygments.lexers.math', 'Scilab', ('scilab',), ('*.sci', '*.sce', '*.tst'), ('text/scilab',)), 'ScssLexer': ('pygments.lexers.web', 'SCSS', ('scss',), ('*.scss',), ('text/x-scss',)), + 'ShellSessionLexer': ('pygments.lexers.shell', 'Shell Session', ('shell-session',), ('*.shell-session',), ('application/x-sh-session',)), + 'SmaliLexer': ('pygments.lexers.dalvik', 'Smali', ('smali',), ('*.smali',), ('text/smali',)), 'SmalltalkLexer': ('pygments.lexers.other', 'Smalltalk', ('smalltalk', 'squeak'), ('*.st',), ('text/x-smalltalk',)), 'SmartyLexer': ('pygments.lexers.templates', 'Smarty', ('smarty',), ('*.tpl',), ('application/x-smarty',)), 'SnobolLexer': ('pygments.lexers.other', 'Snobol', ('snobol',), ('*.snobol',), ('text/x-snobol',)), @@ -247,12 +269,14 @@ LEXERS = { 'SquidConfLexer': ('pygments.lexers.text', 'SquidConf', ('squidconf', 'squid.conf', 'squid'), ('squid.conf',), ('text/x-squidconf',)), 'SspLexer': ('pygments.lexers.templates', 'Scalate Server Page', ('ssp',), ('*.ssp',), ('application/x-ssp',)), 'StanLexer': ('pygments.lexers.math', 'Stan', ('stan',), ('*.stan',), ()), - 'SystemVerilogLexer': ('pygments.lexers.hdl', 'systemverilog', ('sv',), ('*.sv', '*.svh'), ('text/x-systemverilog',)), + 'SystemVerilogLexer': ('pygments.lexers.hdl', 'systemverilog', ('systemverilog', 'sv'), ('*.sv', '*.svh'), ('text/x-systemverilog',)), 'TclLexer': ('pygments.lexers.agile', 'Tcl', ('tcl',), ('*.tcl',), ('text/x-tcl', 'text/x-script.tcl', 'application/x-tcl')), 'TcshLexer': ('pygments.lexers.shell', 'Tcsh', ('tcsh', 'csh'), ('*.tcsh', '*.csh'), ('application/x-csh',)), 'TeaTemplateLexer': ('pygments.lexers.templates', 'Tea', ('tea',), ('*.tea',), ('text/x-tea',)), 'TexLexer': ('pygments.lexers.text', 'TeX', ('tex', 'latex'), ('*.tex', '*.aux', '*.toc'), ('text/x-tex', 'text/x-latex')), 'TextLexer': ('pygments.lexers.special', 'Text only', ('text',), ('*.txt',), ('text/plain',)), + 'TreetopLexer': ('pygments.lexers.parsers', 'Treetop', ('treetop',), ('*.treetop', '*.tt'), ()), + 'TypeScriptLexer': ('pygments.lexers.web', 'TypeScript', ('ts',), ('*.ts',), ('text/x-typescript',)), 'UrbiscriptLexer': ('pygments.lexers.other', 'UrbiScript', ('urbiscript',), ('*.u',), ('application/x-urbiscript',)), 'VGLLexer': ('pygments.lexers.other', 'VGL', ('vgl',), ('*.rpf',), ()), 'ValaLexer': ('pygments.lexers.compiled', 'Vala', ('vala', 'vapi'), ('*.vala', '*.vapi'), ('text/x-vala',)), @@ -261,7 +285,7 @@ LEXERS = { 'VelocityHtmlLexer': ('pygments.lexers.templates', 'HTML+Velocity', ('html+velocity',), (), ('text/html+velocity',)), 'VelocityLexer': ('pygments.lexers.templates', 'Velocity', ('velocity',), ('*.vm', '*.fhtml'), ()), 'VelocityXmlLexer': ('pygments.lexers.templates', 'XML+Velocity', ('xml+velocity',), (), ('application/xml+velocity',)), - 'VerilogLexer': ('pygments.lexers.hdl', 'verilog', ('v',), ('*.v',), ('text/x-verilog',)), + 'VerilogLexer': ('pygments.lexers.hdl', 'verilog', ('verilog', 'v'), ('*.v',), ('text/x-verilog',)), 'VhdlLexer': ('pygments.lexers.hdl', 'vhdl', ('vhdl',), ('*.vhdl', '*.vhd'), ('text/x-vhdl',)), 'VimLexer': ('pygments.lexers.text', 'VimL', ('vim',), ('*.vim', '.vimrc', '.exrc', '.gvimrc', '_vimrc', '_exrc', '_gvimrc', 'vimrc', 'gvimrc'), ('text/x-vim',)), 'XQueryLexer': ('pygments.lexers.web', 'XQuery', ('xquery', 'xqy', 'xq', 'xql', 'xqm'), ('*.xqy', '*.xquery', '*.xq', '*.xql', '*.xqm'), ('text/xquery', 'application/xquery')), @@ -309,9 +333,8 @@ if __name__ == '__main__': footer = content[content.find("if __name__ == '__main__':"):] # write new file - f = open(__file__, 'w') + f = open(__file__, 'wb') f.write(header) f.write('LEXERS = {\n %s,\n}\n\n' % ',\n '.join(found_lexers)) f.write(footer) f.close() - diff --git a/pygments/lexers/_openedgebuiltins.py b/pygments/lexers/_openedgebuiltins.py index 81cc17c9..4561b07b 100644 --- a/pygments/lexers/_openedgebuiltins.py +++ b/pygments/lexers/_openedgebuiltins.py @@ -1,3 +1,14 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers._openedgebuiltins + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Builtin list for the OpenEdgeLexer. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + OPENEDGEKEYWORDS = [ 'ABSOLUTE', 'ABS', 'ABSO', 'ABSOL', 'ABSOLU', 'ABSOLUT', 'ACCELERATOR', 'ACCUM', 'ACCUMULATE', 'ACCUM', 'ACCUMU', 'ACCUMUL', 'ACCUMULA', diff --git a/pygments/lexers/_phpbuiltins.py b/pygments/lexers/_phpbuiltins.py index cd1608f1..08eaaf2e 100644 --- a/pygments/lexers/_phpbuiltins.py +++ b/pygments/lexers/_phpbuiltins.py @@ -12,7 +12,7 @@ internet connection. don't run that at home, use a server ;-) - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -3784,4 +3784,4 @@ if __name__ == '__main__': regenerate(__file__, modules) shutil.rmtree(PHP_MANUAL_DIR) - run()
\ No newline at end of file + run() diff --git a/pygments/lexers/_postgres_builtins.py b/pygments/lexers/_postgres_builtins.py index 45b3f22d..b2322137 100644 --- a/pygments/lexers/_postgres_builtins.py +++ b/pygments/lexers/_postgres_builtins.py @@ -1,10 +1,11 @@ +# -*- coding: utf-8 -*- """ pygments.lexers._postgres_builtins ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Self-updating data files for PostgreSQL lexer. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -41,7 +42,6 @@ def parse_keywords(f): def parse_datatypes(f): dt = set() - re_entry = re.compile('\s*<entry><type>([^<]+)</type></entry>') for line in f: if '<sect1' in line: break @@ -57,7 +57,8 @@ def parse_datatypes(f): line = re.sub("<[^>]+>", "", line) # Drop the parts containing braces - for tmp in [ t for tmp in line.split('[') for t in tmp.split(']') if "(" not in t ]: + for tmp in [t for tmp in line.split('[') + for t in tmp.split(']') if "(" not in t]: for t in tmp.split(','): t = t.strip() if not t: continue diff --git a/pygments/lexers/_robotframeworklexer.py b/pygments/lexers/_robotframeworklexer.py new file mode 100644 index 00000000..0192d289 --- /dev/null +++ b/pygments/lexers/_robotframeworklexer.py @@ -0,0 +1,557 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers._robotframeworklexer + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Lexer for Robot Framework. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +# Copyright 2012 Nokia Siemens Networks Oyj +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import re + +from pygments.lexer import Lexer +from pygments.token import Token + + +HEADING = Token.Generic.Heading +SETTING = Token.Keyword.Namespace +IMPORT = Token.Name.Namespace +TC_KW_NAME = Token.Generic.Subheading +KEYWORD = Token.Name.Function +ARGUMENT = Token.String +VARIABLE = Token.Name.Variable +COMMENT = Token.Comment +SEPARATOR = Token.Punctuation +SYNTAX = Token.Punctuation +GHERKIN = Token.Generic.Emph +ERROR = Token.Error + + +def normalize(string, remove=''): + string = string.lower() + for char in remove + ' ': + if char in string: + string = string.replace(char, '') + return string + + +class RobotFrameworkLexer(Lexer): + """ + For `Robot Framework <http://robotframework.org>`_ test data. + + Supports both space and pipe separated plain text formats. + + *New in Pygments 1.6.* + """ + name = 'RobotFramework' + aliases = ['RobotFramework', 'robotframework'] + filenames = ['*.txt', '*.robot'] + mimetypes = ['text/x-robotframework'] + + def __init__(self, **options): + options['tabsize'] = 2 + options['encoding'] = 'UTF-8' + Lexer.__init__(self, **options) + + def get_tokens_unprocessed(self, text): + row_tokenizer = RowTokenizer() + var_tokenizer = VariableTokenizer() + index = 0 + for row in text.splitlines(): + for value, token in row_tokenizer.tokenize(row): + for value, token in var_tokenizer.tokenize(value, token): + if value: + yield index, token, unicode(value) + index += len(value) + + +class VariableTokenizer(object): + + def tokenize(self, string, token): + var = VariableSplitter(string, identifiers='$@%') + if var.start < 0 or token in (COMMENT, ERROR): + yield string, token + return + for value, token in self._tokenize(var, string, token): + if value: + yield value, token + + def _tokenize(self, var, string, orig_token): + before = string[:var.start] + yield before, orig_token + yield var.identifier + '{', SYNTAX + for value, token in self.tokenize(var.base, VARIABLE): + yield value, token + yield '}', SYNTAX + if var.index: + yield '[', SYNTAX + for value, token in self.tokenize(var.index, VARIABLE): + yield value, token + yield ']', SYNTAX + for value, token in self.tokenize(string[var.end:], orig_token): + yield value, token + + +class RowTokenizer(object): + + def __init__(self): + self._table = UnknownTable() + self._splitter = RowSplitter() + testcases = TestCaseTable() + settings = SettingTable(testcases.set_default_template) + variables = VariableTable() + keywords = KeywordTable() + self._tables = {'settings': settings, 'setting': settings, + 'metadata': settings, + 'variables': variables, 'variable': variables, + 'testcases': testcases, 'testcase': testcases, + 'keywords': keywords, 'keyword': keywords, + 'userkeywords': keywords, 'userkeyword': keywords} + + def tokenize(self, row): + commented = False + heading = False + for index, value in enumerate(self._splitter.split(row)): + # First value, and every second after that, is a separator. + index, separator = divmod(index-1, 2) + if value.startswith('#'): + commented = True + elif index == 0 and value.startswith('*'): + self._table = self._start_table(value) + heading = True + for value, token in self._tokenize(value, index, commented, + separator, heading): + yield value, token + self._table.end_row() + + def _start_table(self, header): + name = normalize(header, remove='*') + return self._tables.get(name, UnknownTable()) + + def _tokenize(self, value, index, commented, separator, heading): + if commented: + yield value, COMMENT + elif separator: + yield value, SEPARATOR + elif heading: + yield value, HEADING + else: + for value, token in self._table.tokenize(value, index): + yield value, token + + +class RowSplitter(object): + _space_splitter = re.compile('( {2,})') + _pipe_splitter = re.compile('((?:^| +)\|(?: +|$))') + + def split(self, row): + splitter = (row.startswith('| ') and self._split_from_pipes + or self._split_from_spaces) + for value in splitter(row.rstrip()): + yield value + yield '\n' + + def _split_from_spaces(self, row): + yield '' # Start with (pseudo)separator similarly as with pipes + for value in self._space_splitter.split(row): + yield value + + def _split_from_pipes(self, row): + _, separator, rest = self._pipe_splitter.split(row, 1) + yield separator + while self._pipe_splitter.search(rest): + cell, separator, rest = self._pipe_splitter.split(rest, 1) + yield cell + yield separator + yield rest + + +class Tokenizer(object): + _tokens = None + + def __init__(self): + self._index = 0 + + def tokenize(self, value): + values_and_tokens = self._tokenize(value, self._index) + self._index += 1 + if isinstance(values_and_tokens, type(Token)): + values_and_tokens = [(value, values_and_tokens)] + return values_and_tokens + + def _tokenize(self, value, index): + index = min(index, len(self._tokens) - 1) + return self._tokens[index] + + def _is_assign(self, value): + if value.endswith('='): + value = value[:-1].strip() + var = VariableSplitter(value, identifiers='$@') + return var.start == 0 and var.end == len(value) + + +class Comment(Tokenizer): + _tokens = (COMMENT,) + + +class Setting(Tokenizer): + _tokens = (SETTING, ARGUMENT) + _keyword_settings = ('suitesetup', 'suiteprecondition', 'suiteteardown', + 'suitepostcondition', 'testsetup', 'testprecondition', + 'testteardown', 'testpostcondition', 'testtemplate') + _import_settings = ('library', 'resource', 'variables') + _other_settings = ('documentation', 'metadata', 'forcetags', 'defaulttags', + 'testtimeout') + _custom_tokenizer = None + + def __init__(self, template_setter=None): + Tokenizer.__init__(self) + self._template_setter = template_setter + + def _tokenize(self, value, index): + if index == 1 and self._template_setter: + self._template_setter(value) + if index == 0: + normalized = normalize(value) + if normalized in self._keyword_settings: + self._custom_tokenizer = KeywordCall(support_assign=False) + elif normalized in self._import_settings: + self._custom_tokenizer = ImportSetting() + elif normalized not in self._other_settings: + return ERROR + elif self._custom_tokenizer: + return self._custom_tokenizer.tokenize(value) + return Tokenizer._tokenize(self, value, index) + + +class ImportSetting(Tokenizer): + _tokens = (IMPORT, ARGUMENT) + + +class TestCaseSetting(Setting): + _keyword_settings = ('setup', 'precondition', 'teardown', 'postcondition', + 'template') + _import_settings = () + _other_settings = ('documentation', 'tags', 'timeout') + + def _tokenize(self, value, index): + if index == 0: + type = Setting._tokenize(self, value[1:-1], index) + return [('[', SYNTAX), (value[1:-1], type), (']', SYNTAX)] + return Setting._tokenize(self, value, index) + + +class KeywordSetting(TestCaseSetting): + _keyword_settings = ('teardown',) + _other_settings = ('documentation', 'arguments', 'return', 'timeout') + + +class Variable(Tokenizer): + _tokens = (SYNTAX, ARGUMENT) + + def _tokenize(self, value, index): + if index == 0 and not self._is_assign(value): + return ERROR + return Tokenizer._tokenize(self, value, index) + + +class KeywordCall(Tokenizer): + _tokens = (KEYWORD, ARGUMENT) + + def __init__(self, support_assign=True): + Tokenizer.__init__(self) + self._keyword_found = not support_assign + self._assigns = 0 + + def _tokenize(self, value, index): + if not self._keyword_found and self._is_assign(value): + self._assigns += 1 + return SYNTAX # VariableTokenizer tokenizes this later. + if self._keyword_found: + return Tokenizer._tokenize(self, value, index - self._assigns) + self._keyword_found = True + return GherkinTokenizer().tokenize(value, KEYWORD) + + +class GherkinTokenizer(object): + _gherkin_prefix = re.compile('^(Given|When|Then|And) ', re.IGNORECASE) + + def tokenize(self, value, token): + match = self._gherkin_prefix.match(value) + if not match: + return [(value, token)] + end = match.end() + return [(value[:end], GHERKIN), (value[end:], token)] + + +class TemplatedKeywordCall(Tokenizer): + _tokens = (ARGUMENT,) + + +class ForLoop(Tokenizer): + + def __init__(self): + Tokenizer.__init__(self) + self._in_arguments = False + + def _tokenize(self, value, index): + token = self._in_arguments and ARGUMENT or SYNTAX + if value.upper() in ('IN', 'IN RANGE'): + self._in_arguments = True + return token + + +class _Table(object): + _tokenizer_class = None + + def __init__(self, prev_tokenizer=None): + self._tokenizer = self._tokenizer_class() + self._prev_tokenizer = prev_tokenizer + self._prev_values_on_row = [] + + def tokenize(self, value, index): + if self._continues(value, index): + self._tokenizer = self._prev_tokenizer + yield value, SYNTAX + else: + for value_and_token in self._tokenize(value, index): + yield value_and_token + self._prev_values_on_row.append(value) + + def _continues(self, value, index): + return value == '...' and all(self._is_empty(t) + for t in self._prev_values_on_row) + + def _is_empty(self, value): + return value in ('', '\\') + + def _tokenize(self, value, index): + return self._tokenizer.tokenize(value) + + def end_row(self): + self.__init__(prev_tokenizer=self._tokenizer) + + +class UnknownTable(_Table): + _tokenizer_class = Comment + + def _continues(self, value, index): + return False + + +class VariableTable(_Table): + _tokenizer_class = Variable + + +class SettingTable(_Table): + _tokenizer_class = Setting + + def __init__(self, template_setter, prev_tokenizer=None): + _Table.__init__(self, prev_tokenizer) + self._template_setter = template_setter + + def _tokenize(self, value, index): + if index == 0 and normalize(value) == 'testtemplate': + self._tokenizer = Setting(self._template_setter) + return _Table._tokenize(self, value, index) + + def end_row(self): + self.__init__(self._template_setter, prev_tokenizer=self._tokenizer) + + +class TestCaseTable(_Table): + _setting_class = TestCaseSetting + _test_template = None + _default_template = None + + @property + def _tokenizer_class(self): + if self._test_template or (self._default_template and + self._test_template is not False): + return TemplatedKeywordCall + return KeywordCall + + def _continues(self, value, index): + return index > 0 and _Table._continues(self, value, index) + + def _tokenize(self, value, index): + if index == 0: + if value: + self._test_template = None + return GherkinTokenizer().tokenize(value, TC_KW_NAME) + if index == 1 and self._is_setting(value): + if self._is_template(value): + self._test_template = False + self._tokenizer = self._setting_class(self.set_test_template) + else: + self._tokenizer = self._setting_class() + if index == 1 and self._is_for_loop(value): + self._tokenizer = ForLoop() + if index == 1 and self._is_empty(value): + return [(value, SYNTAX)] + return _Table._tokenize(self, value, index) + + def _is_setting(self, value): + return value.startswith('[') and value.endswith(']') + + def _is_template(self, value): + return normalize(value) == '[template]' + + def _is_for_loop(self, value): + return value.startswith(':') and normalize(value, remove=':') == 'for' + + def set_test_template(self, template): + self._test_template = self._is_template_set(template) + + def set_default_template(self, template): + self._default_template = self._is_template_set(template) + + def _is_template_set(self, template): + return normalize(template) not in ('', '\\', 'none', '${empty}') + + +class KeywordTable(TestCaseTable): + _tokenizer_class = KeywordCall + _setting_class = KeywordSetting + + def _is_template(self, value): + return False + + +# Following code copied directly from Robot Framework 2.7.5. + +class VariableSplitter: + + def __init__(self, string, identifiers): + self.identifier = None + self.base = None + self.index = None + self.start = -1 + self.end = -1 + self._identifiers = identifiers + self._may_have_internal_variables = False + try: + self._split(string) + except ValueError: + pass + else: + self._finalize() + + def get_replaced_base(self, variables): + if self._may_have_internal_variables: + return variables.replace_string(self.base) + return self.base + + def _finalize(self): + self.identifier = self._variable_chars[0] + self.base = ''.join(self._variable_chars[2:-1]) + self.end = self.start + len(self._variable_chars) + if self._has_list_variable_index(): + self.index = ''.join(self._list_variable_index_chars[1:-1]) + self.end += len(self._list_variable_index_chars) + + def _has_list_variable_index(self): + return self._list_variable_index_chars\ + and self._list_variable_index_chars[-1] == ']' + + def _split(self, string): + start_index, max_index = self._find_variable(string) + self.start = start_index + self._open_curly = 1 + self._state = self._variable_state + self._variable_chars = [string[start_index], '{'] + self._list_variable_index_chars = [] + self._string = string + start_index += 2 + for index, char in enumerate(string[start_index:]): + index += start_index # Giving start to enumerate only in Py 2.6+ + try: + self._state(char, index) + except StopIteration: + return + if index == max_index and not self._scanning_list_variable_index(): + return + + def _scanning_list_variable_index(self): + return self._state in [self._waiting_list_variable_index_state, + self._list_variable_index_state] + + def _find_variable(self, string): + max_end_index = string.rfind('}') + if max_end_index == -1: + raise ValueError('No variable end found') + if self._is_escaped(string, max_end_index): + return self._find_variable(string[:max_end_index]) + start_index = self._find_start_index(string, 1, max_end_index) + if start_index == -1: + raise ValueError('No variable start found') + return start_index, max_end_index + + def _find_start_index(self, string, start, end): + index = string.find('{', start, end) - 1 + if index < 0: + return -1 + if self._start_index_is_ok(string, index): + return index + return self._find_start_index(string, index+2, end) + + def _start_index_is_ok(self, string, index): + return string[index] in self._identifiers\ + and not self._is_escaped(string, index) + + def _is_escaped(self, string, index): + escaped = False + while index > 0 and string[index-1] == '\\': + index -= 1 + escaped = not escaped + return escaped + + def _variable_state(self, char, index): + self._variable_chars.append(char) + if char == '}' and not self._is_escaped(self._string, index): + self._open_curly -= 1 + if self._open_curly == 0: + if not self._is_list_variable(): + raise StopIteration + self._state = self._waiting_list_variable_index_state + elif char in self._identifiers: + self._state = self._internal_variable_start_state + + def _is_list_variable(self): + return self._variable_chars[0] == '@' + + def _internal_variable_start_state(self, char, index): + self._state = self._variable_state + if char == '{': + self._variable_chars.append(char) + self._open_curly += 1 + self._may_have_internal_variables = True + else: + self._variable_state(char, index) + + def _waiting_list_variable_index_state(self, char, index): + if char != '[': + raise StopIteration + self._list_variable_index_chars.append(char) + self._state = self._list_variable_index_state + + def _list_variable_index_state(self, char, index): + self._list_variable_index_chars.append(char) + if char == ']': + raise StopIteration diff --git a/pygments/lexers/_scilab_builtins.py b/pygments/lexers/_scilab_builtins.py index 7e5530d2..ed0dc819 100644 --- a/pygments/lexers/_scilab_builtins.py +++ b/pygments/lexers/_scilab_builtins.py @@ -1,3 +1,14 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers._scilab_builtins + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Builtin list for the ScilabLexer. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + # These lists are generated automatically. # Run the following in a Scilab script: # diff --git a/pygments/lexers/_sourcemodbuiltins.py b/pygments/lexers/_sourcemodbuiltins.py index 19c783a0..0f6b4770 100644 --- a/pygments/lexers/_sourcemodbuiltins.py +++ b/pygments/lexers/_sourcemodbuiltins.py @@ -1,14 +1,14 @@ # -*- coding: utf-8 -*- """ pygments.lexers._sourcemodbuiltins - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This file contains the names of SourceMod functions. It is able to re-generate itself. Do not edit the FUNCTIONS list by hand. - :copyright: Copyright 2006-2011 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/_stan_builtins.py b/pygments/lexers/_stan_builtins.py index 7ce82e53..69d8ce75 100644 --- a/pygments/lexers/_stan_builtins.py +++ b/pygments/lexers/_stan_builtins.py @@ -1,13 +1,13 @@ # -*- coding: utf-8 -*- """ -pygments.lexers._stan_builtins -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + pygments.lexers._stan_builtins + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This file contains the names of functions for Stan used by -``pygments.lexers.math.StanLexer. + This file contains the names of functions for Stan used by + ``pygments.lexers.math.StanLexer. -:copyright: Copyright 2012 by the Pygments team, see AUTHORS. -:license: BSD, see LICENSE for details. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. """ CONSTANTS=[ 'e', diff --git a/pygments/lexers/agile.py b/pygments/lexers/agile.py index 85e157fb..8bcb1d46 100644 --- a/pygments/lexers/agile.py +++ b/pygments/lexers/agile.py @@ -5,7 +5,7 @@ Lexers for agile languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -22,7 +22,8 @@ from pygments import unistring as uni __all__ = ['PythonLexer', 'PythonConsoleLexer', 'PythonTracebackLexer', 'Python3Lexer', 'Python3TracebackLexer', 'RubyLexer', 'RubyConsoleLexer', 'PerlLexer', 'LuaLexer', 'MoonScriptLexer', - 'CrocLexer', 'MiniDLexer', 'IoLexer', 'TclLexer', 'FactorLexer', 'FancyLexer'] + 'CrocLexer', 'MiniDLexer', 'IoLexer', 'TclLexer', 'FactorLexer', + 'FancyLexer', 'DgLexer'] # b/w compatibility from pygments.lexers.functional import SchemeLexer @@ -392,7 +393,7 @@ class PythonTracebackLexer(RegexLexer): bygroups(Text, using(PythonLexer), Text)), (r'^([ \t]*)(\.\.\.)(\n)', bygroups(Text, Comment, Text)), # for doctests... - (r'^(.+)(: )(.+)(\n)', + (r'^([^:]+)(: )(.+)(\n)', bygroups(Generic.Error, Text, Name, Text), '#pop'), (r'^([a-zA-Z_][a-zA-Z0-9_]*)(:?\n)', bygroups(Generic.Error, Text), '#pop') @@ -428,7 +429,7 @@ class Python3TracebackLexer(RegexLexer): bygroups(Text, using(Python3Lexer), Text)), (r'^([ \t]*)(\.\.\.)(\n)', bygroups(Text, Comment, Text)), # for doctests... - (r'^(.+)(: )(.+)(\n)', + (r'^([^:]+)(: )(.+)(\n)', bygroups(Generic.Error, Text, Name, Text), '#pop'), (r'^([a-zA-Z_][a-zA-Z0-9_]*)(:?\n)', bygroups(Generic.Error, Text), '#pop') @@ -520,6 +521,8 @@ class RubyLexer(ExtendedRegexLexer): (r":'(\\\\|\\'|[^'])*'", String.Symbol), (r"'(\\\\|\\'|[^'])*'", String.Single), (r':"', String.Symbol, 'simple-sym'), + (r'([a-zA-Z_][a-zA-Z0-9]*)(:)', + bygroups(String.Symbol, Punctuation)), # Since Ruby 1.9 (r'"', String.Double, 'simple-string'), (r'(?<!\.)`', String.Backtick, 'simple-backtick'), ] @@ -604,7 +607,8 @@ class RubyLexer(ExtendedRegexLexer): r'rescue|raise|retry|return|super|then|undef|unless|until|when|' r'while|yield)\b', Keyword), # start of function, class and module names - (r'(module)(\s+)([a-zA-Z_][a-zA-Z0-9_]*(::[a-zA-Z_][a-zA-Z0-9_]*)*)', + (r'(module)(\s+)([a-zA-Z_][a-zA-Z0-9_]*' + r'(?:::[a-zA-Z_][a-zA-Z0-9_]*)*)', bygroups(Keyword, Text, Name.Namespace)), (r'(def)(\s+)', bygroups(Keyword, Text), 'funcname'), (r'def(?=[*%&^`~+-/\[<>=])', Keyword, 'funcname'), @@ -649,7 +653,7 @@ class RubyLexer(ExtendedRegexLexer): (r'(<<-?)("|\')()(\2)(.*?\n)', heredoc_callback), (r'__END__', Comment.Preproc, 'end-part'), # multiline regex (after keywords or assignments) - (r'(?:^|(?<=[=<>~!])|' + (r'(?:^|(?<=[=<>~!:])|' r'(?<=(?:\s|;)when\s)|' r'(?<=(?:\s|;)or\s)|' r'(?<=(?:\s|;)and\s)|' @@ -1221,7 +1225,8 @@ class CrocLexer(RegexLexer): r'|this|throw|try|vararg|while|with|yield)\b', Keyword), (r'(false|true|null)\b', Keyword.Constant), # FloatLiteral - (r'([0-9][0-9_]*)(?=[.eE])(\.[0-9][0-9_]*)?([eE][+\-]?[0-9_]+)?', Number.Float), + (r'([0-9][0-9_]*)(?=[.eE])(\.[0-9][0-9_]*)?([eE][+\-]?[0-9_]+)?', + Number.Float), # IntegerLiteral # -- Binary (r'0[bB][01][01_]*', Number), @@ -1815,3 +1820,98 @@ class FancyLexer(RegexLexer): (r'\d+', Number.Integer) ] } + + +class DgLexer(RegexLexer): + """ + Lexer for `dg <http://pyos.github.com/dg>`_, + a functional and object-oriented programming language + running on the CPython 3 VM. + + *New in Pygments 1.6.* + """ + name = 'dg' + aliases = ['dg'] + filenames = ['*.dg'] + mimetypes = ['text/x-dg'] + + tokens = { + 'root': [ + # Whitespace: + (r'\s+', Text), + (r'#.*?$', Comment.Single), + # Lexemes: + # Numbers + (r'0[bB][01]+', Number.Bin), + (r'0[oO][0-7]+', Number.Oct), + (r'0[xX][\da-fA-F]+', Number.Hex), + (r'[+-]?\d+\.\d+([eE][+-]?\d+)?[jJ]?', Number.Float), + (r'[+-]?\d+[eE][+-]?\d+[jJ]?', Number.Float), + (r'[+-]?\d+[jJ]?', Number.Integer), + # Character/String Literals + (r"[br]*'''", String, combined('stringescape', 'tsqs', 'string')), + (r'[br]*"""', String, combined('stringescape', 'tdqs', 'string')), + (r"[br]*'", String, combined('stringescape', 'sqs', 'string')), + (r'[br]*"', String, combined('stringescape', 'dqs', 'string')), + # Operators + (r"`\w+'*`", Operator), # Infix links + # Reserved infix links + (r'\b(or|and|if|else|where|is|in)\b', Operator.Word), + (r'[!$%&*+\-./:<-@\\^|~;,]+', Operator), + # Identifiers + # Python 3 types + (r"(?<!\.)(bool|bytearray|bytes|classmethod|complex|dict'?|" + r"float|frozenset|int|list'?|memoryview|object|property|range|" + r"set'?|slice|staticmethod|str|super|tuple'?|type)" + r"(?!['\w])", Name.Builtin), + # Python 3 builtins + some more + (r'(?<!\.)(__import__|abs|all|any|bin|bind|chr|cmp|compile|complex|' + r'delattr|dir|divmod|drop|dropwhile|enumerate|eval|filter|flip|' + r'foldl1?|format|fst|getattr|globals|hasattr|hash|head|hex|id|' + r'init|input|isinstance|issubclass|iter|iterate|last|len|locals|' + r'map|max|min|next|oct|open|ord|pow|print|repr|reversed|round|' + r'setattr|scanl1?|snd|sorted|sum|tail|take|takewhile|vars|zip)' + r"(?!['\w])", Name.Builtin), + (r"(?<!\.)(self|Ellipsis|NotImplemented|None|True|False)(?!['\w])", + Name.Builtin.Pseudo), + (r"(?<!\.)[A-Z]\w*(Error|Exception|Warning)'*(?!['\w])", + Name.Exception), + (r"(?<!\.)(KeyboardInterrupt|SystemExit|StopIteration|" + r"GeneratorExit)(?!['\w])", Name.Exception), + # Compiler-defined identifiers + (r"(?<![\.\w])(import|inherit|for|while|switch|not|raise|unsafe|" + r"yield|with)(?!['\w])", Keyword.Reserved), + # Other links + (r"[A-Z_']+\b", Name), + (r"[A-Z][\w']*\b", Keyword.Type), + (r"\w+'*", Name), + # Blocks + (r'[()]', Punctuation), + ], + 'stringescape': [ + (r'\\([\\abfnrtv"\']|\n|N{.*?}|u[a-fA-F0-9]{4}|' + r'U[a-fA-F0-9]{8}|x[a-fA-F0-9]{2}|[0-7]{1,3})', String.Escape) + ], + 'string': [ + (r'%(\([a-zA-Z0-9_]+\))?[-#0 +]*([0-9]+|[*])?(\.([0-9]+|[*]))?' + '[hlL]?[diouxXeEfFgGcrs%]', String.Interpol), + (r'[^\\\'"%\n]+', String), + # quotes, percents and backslashes must be parsed one at a time + (r'[\'"\\]', String), + # unhandled string formatting sign + (r'%', String), + (r'\n', String) + ], + 'dqs': [ + (r'"', String, '#pop') + ], + 'sqs': [ + (r"'", String, '#pop') + ], + 'tdqs': [ + (r'"""', String, '#pop') + ], + 'tsqs': [ + (r"'''", String, '#pop') + ], + } diff --git a/pygments/lexers/asm.py b/pygments/lexers/asm.py index c1d46bcb..7ff64bcc 100644 --- a/pygments/lexers/asm.py +++ b/pygments/lexers/asm.py @@ -5,7 +5,7 @@ Lexers for assembly languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -17,7 +17,7 @@ from pygments.token import Text, Name, Number, String, Comment, Punctuation, \ Other, Keyword, Operator __all__ = ['GasLexer', 'ObjdumpLexer','DObjdumpLexer', 'CppObjdumpLexer', - 'CObjdumpLexer', 'LlvmLexer', 'NasmLexer'] + 'CObjdumpLexer', 'LlvmLexer', 'NasmLexer', 'Ca65Lexer'] class GasLexer(RegexLexer): @@ -240,8 +240,8 @@ class LlvmLexer(RegexLexer): r'|linkonce_odr|weak|weak_odr|appending|dllimport|dllexport' r'|common|default|hidden|protected|extern_weak|external' r'|thread_local|zeroinitializer|undef|null|to|tail|target|triple' - r'|deplibs|datalayout|volatile|nuw|nsw|exact|inbounds|align' - r'|addrspace|section|alias|module|asm|sideeffect|gc|dbg' + r'|datalayout|volatile|nuw|nsw|nnan|ninf|nsz|arcp|fast|exact|inbounds' + r'|align|addrspace|section|alias|module|asm|sideeffect|gc|dbg' r'|ccc|fastcc|coldcc|x86_stdcallcc|x86_fastcallcc|arm_apcscc' r'|arm_aapcscc|arm_aapcs_vfpcc' @@ -358,3 +358,41 @@ class NasmLexer(RegexLexer): (type, Keyword.Type) ], } + + +class Ca65Lexer(RegexLexer): + """ + For ca65 assembler sources. + + *New in Pygments 1.6.* + """ + name = 'ca65' + aliases = ['ca65'] + filenames = ['*.s'] + + flags = re.IGNORECASE + + tokens = { + 'root': [ + (r';.*', Comment.Single), + (r'\s+', Text), + (r'[a-z_.@$][\w.@$]*:', Name.Label), + (r'((ld|st)[axy]|(in|de)[cxy]|asl|lsr|ro[lr]|adc|sbc|cmp|cp[xy]' + r'|cl[cvdi]|se[cdi]|jmp|jsr|bne|beq|bpl|bmi|bvc|bvs|bcc|bcs' + r'|p[lh][ap]|rt[is]|brk|nop|ta[xy]|t[xy]a|txs|tsx|and|ora|eor' + r'|bit)\b', Keyword), + (r'\.[a-z0-9_]+', Keyword.Pseudo), + (r'[-+~*/^&|!<>=]', Operator), + (r'"[^"\n]*.', String), + (r"'[^'\n]*.", String.Char), + (r'\$[0-9a-f]+|[0-9a-f]+h\b', Number.Hex), + (r'\d+|%[01]+', Number.Integer), + (r'[#,.:()=]', Punctuation), + (r'[a-z_.@$][\w.@$]*', Name), + ] + } + + def analyse_text(self, text): + # comments in GAS start with "#" + if re.match(r'^\s*;', text, re.MULTILINE): + return 0.9 diff --git a/pygments/lexers/compiled.py b/pygments/lexers/compiled.py index 450a9c4d..7513a4e1 100644 --- a/pygments/lexers/compiled.py +++ b/pygments/lexers/compiled.py @@ -5,7 +5,7 @@ Lexers for compiled languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -13,31 +13,30 @@ import re from string import Template from pygments.lexer import Lexer, RegexLexer, include, bygroups, using, \ - this, combined + this, combined, inherit, do_insertions from pygments.util import get_bool_opt, get_list_opt from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ - Number, Punctuation, Error, Literal + Number, Punctuation, Error, Literal, Generic from pygments.scanner import Scanner # backwards compatibility from pygments.lexers.functional import OcamlLexer from pygments.lexers.jvm import JavaLexer, ScalaLexer -__all__ = ['CLexer', 'CppLexer', 'DLexer', 'DelphiLexer', 'ECLexer', - 'DylanLexer', 'ObjectiveCLexer', 'FortranLexer', 'GLShaderLexer', +__all__ = ['CLexer', 'CppLexer', 'DLexer', 'DelphiLexer', 'ECLexer', 'DylanLexer', + 'ObjectiveCLexer', 'ObjectiveCppLexer', 'FortranLexer', 'GLShaderLexer', 'PrologLexer', 'CythonLexer', 'ValaLexer', 'OocLexer', 'GoLexer', 'FelixLexer', 'AdaLexer', 'Modula2Lexer', 'BlitzMaxLexer', - 'NimrodLexer', 'FantomLexer', 'RustLexer', 'CUDALexer'] + 'NimrodLexer', 'FantomLexer', 'RustLexer', 'CudaLexer', 'MonkeyLexer', + 'DylanLidLexer', 'DylanConsoleLexer', 'CobolLexer', + 'CobolFreeformatLexer', 'LogosLexer'] -class CLexer(RegexLexer): +class CFamilyLexer(RegexLexer): """ - For C source code with preprocessor directives. + For C family source code. This is used as a base class to avoid repetitious + definitions. """ - name = 'C' - aliases = ['c'] - filenames = ['*.c', '*.h', '*.idc'] - mimetypes = ['text/x-chdr', 'text/x-csrc'] #: optional Comment or Whitespace _ws = r'(?:\s|//.*?\n|/[*].*?[*]/)+' @@ -76,12 +75,17 @@ class CLexer(RegexLexer): (r'\b(case)(.+?)(:)', bygroups(Keyword, using(this), Text)), (r'(auto|break|case|const|continue|default|do|else|enum|extern|' r'for|goto|if|register|restricted|return|sizeof|static|struct|' - r'switch|typedef|union|volatile|virtual|while)\b', Keyword), - (r'(int|long|float|short|double|char|unsigned|signed|void)\b', + r'switch|typedef|union|volatile|while)\b', Keyword), + (r'(bool|int|long|float|short|double|char|unsigned|signed|void|' + r'[a-z_][a-z0-9_]*_t)\b', Keyword.Type), (r'(_{0,2}inline|naked|restrict|thread|typename)\b', Keyword.Reserved), + # Vector intrinsics + (r'(__(m128i|m128d|m128|m64))\b', Keyword.Reserved), + # Microsoft-isms (r'__(asm|int8|based|except|int16|stdcall|cdecl|fastcall|int32|' - r'declspec|finally|int64|try|leave)\b', Keyword.Reserved), + r'declspec|finally|int64|try|leave|wchar_t|w64|unaligned|' + r'raise|noop|identifier|forceinline|assume)\b', Keyword.Reserved), (r'(true|false|NULL)\b', Name.Builtin), ('[a-zA-Z_][a-zA-Z0-9_]*', Name), ], @@ -119,7 +123,8 @@ class CLexer(RegexLexer): ], 'string': [ (r'"', String, '#pop'), - (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})', String.Escape), + (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|' + r'u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8}|[0-7]{1,3})', String.Escape), (r'[^\\"\n]+', String), # all other characters (r'\\\n', String), # line continuation (r'\\', String), # stray backslash @@ -141,16 +146,17 @@ class CLexer(RegexLexer): } stdlib_types = ['size_t', 'ssize_t', 'off_t', 'wchar_t', 'ptrdiff_t', - 'sig_atomic_t', 'fpos_t', 'clock_t', 'time_t', 'va_list', - 'jmp_buf', 'FILE', 'DIR', 'div_t', 'ldiv_t', 'mbstate_t', - 'wctrans_t', 'wint_t', 'wctype_t'] + 'sig_atomic_t', 'fpos_t', 'clock_t', 'time_t', 'va_list', + 'jmp_buf', 'FILE', 'DIR', 'div_t', 'ldiv_t', 'mbstate_t', + 'wctrans_t', 'wint_t', 'wctype_t'] c99_types = ['_Bool', '_Complex', 'int8_t', 'int16_t', 'int32_t', 'int64_t', - 'uint8_t', 'uint16_t', 'uint32_t', 'uint64_t', 'int_least8_t', - 'int_least16_t', 'int_least32_t', 'int_least64_t', - 'uint_least8_t', 'uint_least16_t', 'uint_least32_t', - 'uint_least64_t', 'int_fast8_t', 'int_fast16_t', 'int_fast32_t', - 'int_fast64_t', 'uint_fast8_t', 'uint_fast16_t', 'uint_fast32_t', - 'uint_fast64_t', 'intptr_t', 'uintptr_t', 'intmax_t', 'uintmax_t'] + 'uint8_t', 'uint16_t', 'uint32_t', 'uint64_t', 'int_least8_t', + 'int_least16_t', 'int_least32_t', 'int_least64_t', + 'uint_least8_t', 'uint_least16_t', 'uint_least32_t', + 'uint_least64_t', 'int_fast8_t', 'int_fast16_t', 'int_fast32_t', + 'int_fast64_t', 'uint_fast8_t', 'uint_fast16_t', 'uint_fast32_t', + 'uint_fast64_t', 'intptr_t', 'uintptr_t', 'intmax_t', + 'uintmax_t'] def __init__(self, **options): self.stdlibhighlighting = get_bool_opt(options, @@ -169,99 +175,63 @@ class CLexer(RegexLexer): token = Keyword.Type yield index, token, value -class CppLexer(RegexLexer): + +class CLexer(CFamilyLexer): + """ + For C source code with preprocessor directives. + """ + name = 'C' + aliases = ['c'] + filenames = ['*.c', '*.h', '*.idc'] + mimetypes = ['text/x-chdr', 'text/x-csrc'] + priority = 0.1 + + def analyse_text(text): + return 0.1 + + +class CppLexer(CFamilyLexer): """ For C++ source code with preprocessor directives. """ name = 'C++' aliases = ['cpp', 'c++'] filenames = ['*.cpp', '*.hpp', '*.c++', '*.h++', - '*.cc', '*.hh', '*.cxx', '*.hxx'] + '*.cc', '*.hh', '*.cxx', '*.hxx', + '*.C', '*.H', '*.cp', '*.CPP'] mimetypes = ['text/x-c++hdr', 'text/x-c++src'] - - #: optional Comment or Whitespace - _ws = r'(?:\s|//.*?\n|/[*].*?[*]/)+' - #: only one /* */ style comment - _ws1 = r':\s*/[*].*?[*]/\s*' + priority = 0.1 tokens = { - 'root': [ - # preprocessor directives: without whitespace - ('^#if\s+0', Comment.Preproc, 'if0'), - ('^#', Comment.Preproc, 'macro'), - # or with whitespace - ('^(' + _ws1 + r')(#if\s+0)', - bygroups(using(this), Comment.Preproc), 'if0'), - ('^(' + _ws1 + ')(#)', - bygroups(using(this), Comment.Preproc), 'macro'), - (r'\n', Text), - (r'\s+', Text), - (r'\\\n', Text), # line continuation - (r'/(\\\n)?/(\n|(.|\n)*?[^\\]\n)', Comment.Single), - (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline), - (r'[{}]', Punctuation), - (r'L?"', String, 'string'), - (r"L?'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])'", String.Char), - (r'(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[LlUu]*', Number.Float), - (r'(\d+\.\d*|\.\d+|\d+[fF])[fF]?', Number.Float), - (r'0x[0-9a-fA-F]+[LlUu]*', Number.Hex), - (r'0[0-7]+[LlUu]*', Number.Oct), - (r'\d+[LlUu]*', Number.Integer), - (r'\*/', Error), - (r'[~!%^&*+=|?:<>/-]', Operator), - (r'[()\[\],.;]', Punctuation), - (r'(asm|auto|break|case|catch|const|const_cast|continue|' - r'default|delete|do|dynamic_cast|else|enum|explicit|export|' - r'extern|for|friend|goto|if|mutable|namespace|new|operator|' - r'private|protected|public|register|reinterpret_cast|return|' - r'restrict|sizeof|static|static_cast|struct|switch|template|' - r'this|throw|throws|try|typedef|typeid|typename|union|using|' - r'volatile|virtual|while)\b', Keyword), + 'statements': [ + (r'(asm|catch|const_cast|delete|dynamic_cast|explicit|' + r'export|friend|mutable|namespace|new|operator|' + r'private|protected|public|reinterpret_cast|' + r'restrict|static_cast|template|this|throw|throws|' + r'typeid|typename|using|virtual)\b', Keyword), (r'(class)(\s+)', bygroups(Keyword, Text), 'classname'), - (r'(bool|int|long|float|short|double|char|unsigned|signed|' - r'void|wchar_t)\b', Keyword.Type), - (r'(_{0,2}inline|naked|thread)\b', Keyword.Reserved), - (r'__(asm|int8|based|except|int16|stdcall|cdecl|fastcall|int32|' - r'declspec|finally|int64|try|leave|wchar_t|w64|virtual_inheritance|' - r'uuidof|unaligned|super|single_inheritance|raise|noop|' - r'multiple_inheritance|m128i|m128d|m128|m64|interface|' - r'identifier|forceinline|event|assume)\b', Keyword.Reserved), + inherit, + ], + 'root': [ + inherit, + # C++ Microsoft-isms + (r'__(virtual_inheritance|uuidof|super|single_inheritance|' + r'multiple_inheritance|interface|event)\b', Keyword.Reserved), # Offload C++ extensions, http://offload.codeplay.com/ (r'(__offload|__blockingoffload|__outer)\b', Keyword.Pseudo), - (r'(true|false)\b', Keyword.Constant), - (r'NULL\b', Name.Builtin), - ('[a-zA-Z_][a-zA-Z0-9_]*:(?!:)', Name.Label), - ('[a-zA-Z_][a-zA-Z0-9_]*', Name), ], 'classname': [ (r'[a-zA-Z_][a-zA-Z0-9_]*', Name.Class, '#pop'), # template specification (r'\s*(?=>)', Text, '#pop'), ], - 'string': [ - (r'"', String, '#pop'), - (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})', String.Escape), - (r'[^\\"\n]+', String), # all other characters - (r'\\\n', String), # line continuation - (r'\\', String), # stray backslash - ], - 'macro': [ - (r'[^/\n]+', Comment.Preproc), - (r'/[*](.|\n)*?[*]/', Comment.Multiline), - (r'//.*?\n', Comment.Single, '#pop'), - (r'/', Comment.Preproc), - (r'(?<=\\)\n', Comment.Preproc), - (r'\n', Comment.Preproc, '#pop'), - ], - 'if0': [ - (r'^\s*#if.*?(?<!\\)\n', Comment.Preproc, '#push'), - (r'^\s*#endif.*?(?<!\\)\n', Comment.Preproc, '#pop'), - (r'.*?\n', Comment), - ] } + def analyse_text(text): + return 0.1 -class ECLexer(RegexLexer): + +class ECLexer(CLexer): """ For eC source code with preprocessor directives. @@ -272,146 +242,29 @@ class ECLexer(RegexLexer): filenames = ['*.ec', '*.eh'] mimetypes = ['text/x-echdr', 'text/x-ecsrc'] - #: optional Comment or Whitespace - _ws = r'(?:\s|//.*?\n|/[*].*?[*]/)+' - #: only one /* */ style comment - _ws1 = r':\s*/[*].*?[*]/\s*' - tokens = { - 'whitespace': [ - # preprocessor directives: without whitespace - ('^#if\s+0', Comment.Preproc, 'if0'), - ('^#', Comment.Preproc, 'macro'), - # or with whitespace - ('^' + _ws1 + r'#if\s+0', Comment.Preproc, 'if0'), - ('^' + _ws1 + '#', Comment.Preproc, 'macro'), - (r'^(\s*)([a-zA-Z_][a-zA-Z0-9_]*:(?!:))', bygroups(Text, Name.Label)), - (r'\n', Text), - (r'\s+', Text), - (r'\\\n', Text), # line continuation - (r'//(\n|(.|\n)*?[^\\]\n)', Comment.Single), - (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline), - ], 'statements': [ - (r'L?"', String, 'string'), - (r"L?'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])'", String.Char), - (r'(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[LlUu]*', Number.Float), - (r'(\d+\.\d*|\.\d+|\d+[fF])[fF]?', Number.Float), - (r'0x[0-9a-fA-F]+[LlUu]*', Number.Hex), - (r'0[0-7]+[LlUu]*', Number.Oct), - (r'\d+[LlUu]*', Number.Integer), - (r'\*/', Error), - (r'[~!%^&*+=|?:<>/-]', Operator), - (r'[()\[\],.]', Punctuation), - (r'\b(case)(.+?)(:)', bygroups(Keyword, using(this), Text)), - (r'(auto|break|case|const|continue|default|do|else|enum|extern|' - r'for|goto|if|register|restricted|return|sizeof|static|struct|' - r'switch|typedef|union|volatile|virtual|while|class|private|public|' - r'property|import|delete|new|new0|renew|renew0|define|get|set|remote|dllexport|dllimport|stdcall|' - r'subclass|__on_register_module|namespace|using|typed_object|any_object|incref|register|watch|' - r'stopwatching|firewatchers|watchable|class_designer|class_fixed|class_no_expansion|isset|' - r'class_default_property|property_category|class_data|class_property|virtual|thisclass|' + (r'(virtual|class|private|public|property|import|delete|new|new0|' + r'renew|renew0|define|get|set|remote|dllexport|dllimport|stdcall|' + r'subclass|__on_register_module|namespace|using|typed_object|' + r'any_object|incref|register|watch|stopwatching|firewatchers|' + r'watchable|class_designer|class_fixed|class_no_expansion|isset|' + r'class_default_property|property_category|class_data|' + r'class_property|virtual|thisclass|' r'dbtable|dbindex|database_open|dbfield)\b', Keyword), - (r'(int|long|float|short|double|char|unsigned|signed|void)\b', - Keyword.Type), (r'(uint|uint16|uint32|uint64|bool|byte|unichar|int64)\b', Keyword.Type), (r'(class)(\s+)', bygroups(Keyword, Text), 'classname'), - (r'(_{0,2}inline|naked|restrict|thread|typename)\b', Keyword.Reserved), - (r'__(asm|int8|based|except|int16|stdcall|cdecl|fastcall|int32|' - r'declspec|finally|int64|try|leave)\b', Keyword.Reserved), - (r'(true|false|null|value|this|NULL)\b', Name.Builtin), - ('[a-zA-Z_][a-zA-Z0-9_]*', Name), - ], - 'root': [ - include('whitespace'), - # functions - (r'((?:[a-zA-Z0-9_*\s])+?(?:\s|[*]))' # return arguments - r'([a-zA-Z_][a-zA-Z0-9_]*)' # method name - r'(\s*\([^;]*?\))' # signature - r'(' + _ws + r')?({)', - bygroups(using(this), Name.Function, using(this), using(this), - Punctuation), - 'function'), - # function declarations - (r'((?:[a-zA-Z0-9_*\s])+?(?:\s|[*]))' # return arguments - r'([a-zA-Z_][a-zA-Z0-9_]*)' # method name - r'(\s*\([^;]*?\))' # signature - r'(' + _ws + r')?(;)', - bygroups(using(this), Name.Function, using(this), using(this), - Punctuation)), - ('', Text, 'statement'), + (r'(null|value|this)\b', Name.Builtin), + inherit, ], 'classname': [ (r'[a-zA-Z_][a-zA-Z0-9_]*', Name.Class, '#pop'), # template specification (r'\s*(?=>)', Text, '#pop'), ], - 'statement' : [ - include('whitespace'), - include('statements'), - ('[{}]', Punctuation), - (';', Punctuation, '#pop'), - ], - 'function': [ - include('whitespace'), - include('statements'), - (';', Punctuation), - ('{', Punctuation, '#push'), - ('}', Punctuation, '#pop'), - ], - 'string': [ - (r'"', String, '#pop'), - (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})', String.Escape), - (r'[^\\"\n]+', String), # all other characters - (r'\\\n', String), # line continuation - (r'\\', String), # stray backslash - ], - 'macro': [ - (r'[^/\n]+', Comment.Preproc), - (r'/[*](.|\n)*?[*]/', Comment.Multiline), - (r'//.*?\n', Comment.Single, '#pop'), - (r'/', Comment.Preproc), - (r'(?<=\\)\n', Comment.Preproc), - (r'\n', Comment.Preproc, '#pop'), - ], - 'if0': [ - (r'^\s*#if.*?(?<!\\)\n', Comment.Preproc, '#push'), - (r'^\s*#el(?:se|if).*\n', Comment.Preproc, '#pop'), - (r'^\s*#endif.*?(?<!\\)\n', Comment.Preproc, '#pop'), - (r'.*?\n', Comment), - ] } - stdlib_types = ['size_t', 'ssize_t', 'off_t', 'wchar_t', 'ptrdiff_t', - 'sig_atomic_t', 'fpos_t', 'clock_t', 'time_t', 'va_list', - 'jmp_buf', 'FILE', 'DIR', 'div_t', 'ldiv_t', 'mbstate_t', - 'wctrans_t', 'wint_t', 'wctype_t'] - c99_types = ['_Bool', '_Complex', 'int8_t', 'int16_t', 'int32_t', 'int64_t', - 'uint8_t', 'uint16_t', 'uint32_t', 'uint64_t', 'int_least8_t', - 'int_least16_t', 'int_least32_t', 'int_least64_t', - 'uint_least8_t', 'uint_least16_t', 'uint_least32_t', - 'uint_least64_t', 'int_fast8_t', 'int_fast16_t', 'int_fast32_t', - 'int_fast64_t', 'uint_fast8_t', 'uint_fast16_t', 'uint_fast32_t', - 'uint_fast64_t', 'intptr_t', 'uintptr_t', 'intmax_t', 'uintmax_t'] - - def __init__(self, **options): - self.stdlibhighlighting = get_bool_opt(options, - 'stdlibhighlighting', True) - self.c99highlighting = get_bool_opt(options, - 'c99highlighting', True) - RegexLexer.__init__(self, **options) - - def get_tokens_unprocessed(self, text): - for index, token, value in \ - RegexLexer.get_tokens_unprocessed(self, text): - if token is Name: - if self.stdlibhighlighting and value in self.stdlib_types: - token = Keyword.Type - elif self.c99highlighting and value in self.c99_types: - token = Keyword.Type - yield index, token, value - class DLexer(RegexLexer): """ @@ -1056,40 +909,178 @@ class DylanLexer(RegexLexer): name = 'Dylan' aliases = ['dylan'] - filenames = ['*.dylan', '*.dyl'] + filenames = ['*.dylan', '*.dyl', '*.intr'] mimetypes = ['text/x-dylan'] - flags = re.DOTALL + flags = re.IGNORECASE + + builtins = set([ + 'subclass', 'abstract', 'block', 'concrete', 'constant', 'class', + 'compiler-open', 'compiler-sideways', 'domain', 'dynamic', + 'each-subclass', 'exception', 'exclude', 'function', 'generic', + 'handler', 'inherited', 'inline', 'inline-only', 'instance', + 'interface', 'import', 'keyword', 'library', 'macro', 'method', + 'module', 'open', 'primary', 'required', 'sealed', 'sideways', + 'singleton', 'slot', 'thread', 'variable', 'virtual']) + + keywords = set([ + 'above', 'afterwards', 'begin', 'below', 'by', 'case', 'cleanup', + 'create', 'define', 'else', 'elseif', 'end', 'export', 'finally', + 'for', 'from', 'if', 'in', 'let', 'local', 'otherwise', 'rename', + 'select', 'signal', 'then', 'to', 'unless', 'until', 'use', 'when', + 'while']) + + operators = set([ + '~', '+', '-', '*', '|', '^', '=', '==', '~=', '~==', '<', '<=', + '>', '>=', '&', '|']) + + functions = set([ + 'abort', 'abs', 'add', 'add!', 'add-method', 'add-new', 'add-new!', + 'all-superclasses', 'always', 'any?', 'applicable-method?', 'apply', + 'aref', 'aref-setter', 'as', 'as-lowercase', 'as-lowercase!', + 'as-uppercase', 'as-uppercase!', 'ash', 'backward-iteration-protocol', + 'break', 'ceiling', 'ceiling/', 'cerror', 'check-type', 'choose', + 'choose-by', 'complement', 'compose', 'concatenate', 'concatenate-as', + 'condition-format-arguments', 'condition-format-string', 'conjoin', + 'copy-sequence', 'curry', 'default-handler', 'dimension', 'dimensions', + 'direct-subclasses', 'direct-superclasses', 'disjoin', 'do', + 'do-handlers', 'element', 'element-setter', 'empty?', 'error', 'even?', + 'every?', 'false-or', 'fill!', 'find-key', 'find-method', 'first', + 'first-setter', 'floor', 'floor/', 'forward-iteration-protocol', + 'function-arguments', 'function-return-values', + 'function-specializers', 'gcd', 'generic-function-mandatory-keywords', + 'generic-function-methods', 'head', 'head-setter', 'identity', + 'initialize', 'instance?', 'integral?', 'intersection', + 'key-sequence', 'key-test', 'last', 'last-setter', 'lcm', 'limited', + 'list', 'logand', 'logbit?', 'logior', 'lognot', 'logxor', 'make', + 'map', 'map-as', 'map-into', 'max', 'member?', 'merge-hash-codes', + 'min', 'modulo', 'negative', 'negative?', 'next-method', + 'object-class', 'object-hash', 'odd?', 'one-of', 'pair', 'pop', + 'pop-last', 'positive?', 'push', 'push-last', 'range', 'rank', + 'rcurry', 'reduce', 'reduce1', 'remainder', 'remove', 'remove!', + 'remove-duplicates', 'remove-duplicates!', 'remove-key!', + 'remove-method', 'replace-elements!', 'replace-subsequence!', + 'restart-query', 'return-allowed?', 'return-description', + 'return-query', 'reverse', 'reverse!', 'round', 'round/', + 'row-major-index', 'second', 'second-setter', 'shallow-copy', + 'signal', 'singleton', 'size', 'size-setter', 'slot-initialized?', + 'sort', 'sort!', 'sorted-applicable-methods', 'subsequence-position', + 'subtype?', 'table-protocol', 'tail', 'tail-setter', 'third', + 'third-setter', 'truncate', 'truncate/', 'type-error-expected-type', + 'type-error-value', 'type-for-copy', 'type-union', 'union', 'values', + 'vector', 'zero?']) + + valid_name = '\\\\?[a-zA-Z0-9' + re.escape('!&*<>|^$%@_-+~?/=') + ']+' + + def get_tokens_unprocessed(self, text): + for index, token, value in RegexLexer.get_tokens_unprocessed(self, text): + if token is Name: + lowercase_value = value.lower() + if lowercase_value in self.builtins: + yield index, Name.Builtin, value + continue + if lowercase_value in self.keywords: + yield index, Keyword, value + continue + if lowercase_value in self.functions: + yield index, Name.Builtin, value + continue + if lowercase_value in self.operators: + yield index, Operator, value + continue + yield index, token, value tokens = { 'root': [ - (r'\b(subclass|abstract|block|c(on(crete|stant)|lass)|domain' - r'|ex(c(eption|lude)|port)|f(unction(al)?)|generic|handler' - r'|i(n(herited|line|stance|terface)|mport)|library|m(acro|ethod)' - r'|open|primary|sealed|si(deways|ngleton)|slot' - r'|v(ariable|irtual))\b', Name.Builtin), - (r'<\w+>', Keyword.Type), + # Whitespace + (r'\s+', Text), + + # single line comment (r'//.*?\n', Comment.Single), - (r'/\*[\w\W]*?\*/', Comment.Multiline), + + # lid header + (r'([A-Za-z0-9-]+)(:)([ \t]*)(.*(?:\n[ \t].+)*)', + bygroups(Name.Attribute, Operator, Text, String)), + + ('', Text, 'code') # no header match, switch to code + ], + 'code': [ + # Whitespace + (r'\s+', Text), + + # single line comment + (r'//.*?\n', Comment.Single), + + # multi-line comment + (r'/\*', Comment.Multiline, 'comment'), + + # strings and characters (r'"', String, 'string'), (r"'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])'", String.Char), - (r'=>|\b(a(bove|fterwards)|b(e(gin|low)|y)|c(ase|leanup|reate)' - r'|define|else(if)?|end|f(inally|or|rom)|i[fn]|l(et|ocal)|otherwise' - r'|rename|s(elect|ignal)|t(hen|o)|u(n(less|til)|se)|wh(en|ile))\b', - Keyword), - (r'([ \t])([!\$%&\*\/:<=>\?~_^a-zA-Z0-9.+\-]*:)', - bygroups(Text, Name.Variable)), - (r'([ \t]*)(\S+[^:])([ \t]*)(\()([ \t]*)', - bygroups(Text, Name.Function, Text, Punctuation, Text)), - (r'-?[0-9.]+', Number), - (r'[(),;]', Punctuation), - (r'\$[a-zA-Z0-9-]+', Name.Constant), - (r'[!$%&*/:<>=?~^.+\[\]{}-]+', Operator), - (r'\s+', Text), - (r'#"[a-zA-Z0-9-]+"', Keyword), + + # binary integer + (r'#[bB][01]+', Number), + + # octal integer + (r'#[oO][0-7]+', Number.Oct), + + # floating point + (r'[-+]?(\d*\.\d+(e[-+]?\d+)?|\d+(\.\d*)?e[-+]?\d+)', Number.Float), + + # decimal integer + (r'[-+]?\d+', Number.Integer), + + # hex integer + (r'#[xX][0-9a-fA-F]+', Number.Hex), + + # Macro parameters + (r'(\?' + valid_name + ')(:)' + r'(token|name|variable|expression|body|case-body|\*)', + bygroups(Name.Tag, Operator, Name.Builtin)), + (r'(\?)(:)(token|name|variable|expression|body|case-body|\*)', + bygroups(Name.Tag, Operator, Name.Builtin)), + (r'\?' + valid_name, Name.Tag), + + # Punctuation + (r'(=>|::|#\(|#\[|##|\?|\?\?|\?=|[(){}\[\],\.;])', Punctuation), + + # Most operators are picked up as names and then re-flagged. + # This one isn't valid in a name though, so we pick it up now. + (r':=', Operator), + + # Pick up #t / #f before we match other stuff with #. + (r'#[tf]', Literal), + + # #"foo" style keywords + (r'#"', String.Symbol, 'keyword'), + + # #rest, #key, #all-keys, etc. (r'#[a-zA-Z0-9-]+', Keyword), - (r'#(\(|\[)', Punctuation), - (r'[a-zA-Z0-9-_]+', Name.Variable), + + # required-init-keyword: style keywords. + (valid_name + ':', Keyword), + + # class names + (r'<' + valid_name + '>', Name.Class), + + # define variable forms. + (r'\*' + valid_name + '\*', Name.Variable.Global), + + # define constant forms. + (r'\$' + valid_name, Name.Constant), + + # everything else. We re-flag some of these in the method above. + (valid_name, Name), + ], + 'comment': [ + (r'[^*/]', Comment.Multiline), + (r'/\*', Comment.Multiline, '#push'), + (r'\*/', Comment.Multiline, '#pop'), + (r'[*/]', Comment.Multiline) + ], + 'keyword': [ + (r'"', String.Symbol, '#pop'), + (r'[^\\"]+', String.Symbol), # all other characters ], 'string': [ (r'"', String, '#pop'), @@ -1097,173 +1088,211 @@ class DylanLexer(RegexLexer): (r'[^\\"\n]+', String), # all other characters (r'\\\n', String), # line continuation (r'\\', String), # stray backslash - ], + ] } -class ObjectiveCLexer(RegexLexer): +class DylanLidLexer(RegexLexer): """ - For Objective-C source code with preprocessor directives. + For Dylan LID (Library Interchange Definition) files. + + *New in Pygments 1.6.* """ - name = 'Objective-C' - aliases = ['objective-c', 'objectivec', 'obj-c', 'objc'] - # XXX: objc has .h files too :-/ - filenames = ['*.m'] - mimetypes = ['text/x-objective-c'] + name = 'DylanLID' + aliases = ['dylan-lid', 'lid'] + filenames = ['*.lid', '*.hdp'] + mimetypes = ['text/x-dylan-lid'] - #: optional Comment or Whitespace - _ws = r'(?:\s|//.*?\n|/[*].*?[*]/)+' - #: only one /* */ style comment - _ws1 = r':\s*/[*].*?[*]/\s*' + flags = re.IGNORECASE tokens = { - 'whitespace': [ - # preprocessor directives: without whitespace - ('^#if\s+0', Comment.Preproc, 'if0'), - ('^#', Comment.Preproc, 'macro'), - # or with whitespace - ('^(' + _ws1 + r')(#if\s+0)', - bygroups(using(this), Comment.Preproc), 'if0'), - ('^(' + _ws1 + ')(#)', - bygroups(using(this), Comment.Preproc), 'macro'), - (r'\n', Text), - (r'\s+', Text), - (r'\\\n', Text), # line continuation - (r'//(\n|(.|\n)*?[^\\]\n)', Comment.Single), - (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline), - ], - 'statements': [ - (r'(L|@)?"', String, 'string'), - (r"(L|@)?'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])'", - String.Char), - (r'(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[lL]?', Number.Float), - (r'(\d+\.\d*|\.\d+|\d+[fF])[fF]?', Number.Float), - (r'0x[0-9a-fA-F]+[Ll]?', Number.Hex), - (r'0[0-7]+[Ll]?', Number.Oct), - (r'\d+[Ll]?', Number.Integer), - (r'[~!%^&*+=|?:<>/-]', Operator), - (r'[()\[\],.]', Punctuation), - (r'(auto|break|case|const|continue|default|do|else|enum|extern|' - r'for|goto|if|register|restricted|return|sizeof|static|struct|' - r'switch|typedef|union|volatile|virtual|while|in|@selector|' - r'@private|@protected|@public|@encode|' - r'@synchronized|@try|@throw|@catch|@finally|@end|@property|' - r'@synthesize|@dynamic|@optional)\b', Keyword), - (r'(int|long|float|short|double|char|unsigned|signed|void|' - r'id|BOOL|IBOutlet|IBAction|SEL)\b', Keyword.Type), - (r'(_{0,2}inline|naked|restrict|thread|typename)\b', - Keyword.Reserved), - (r'__(asm|int8|based|except|int16|stdcall|cdecl|fastcall|int32|' - r'declspec|finally|int64|try|leave)\b', Keyword.Reserved), - (r'(TRUE|FALSE|nil|NULL)\b', Name.Builtin), - ('[a-zA-Z$_][a-zA-Z0-9$_]*:(?!:)', Name.Label), - ('[a-zA-Z$_][a-zA-Z0-9$_]*', Name), - (r'(@interface|@implementation)(\s+)', bygroups(Keyword, Text), - ('#pop', 'classname')), - (r'(@class|@protocol)(\s+)', bygroups(Keyword, Text), - ('#pop', 'forward_classname')), - ], 'root': [ - include('whitespace'), - # functions - (r'((?:[a-zA-Z0-9_*\s])+?(?:\s|[*]))' # return arguments - r'([a-zA-Z$_][a-zA-Z0-9$_]*)' # method name - r'(\s*\([^;]*?\))' # signature - r'(' + _ws + r')?({)', - bygroups(using(this), Name.Function, - using(this), Text, Punctuation), - 'function'), - # methods - (r'^([-+])(\s*)' # method marker - r'(\(.*?\))?(\s*)' # return type - r'([a-zA-Z$_][a-zA-Z0-9$_]*:?)', # begin of method name - bygroups(Keyword, Text, using(this), - Text, Name.Function), - 'method'), - # function declarations - (r'((?:[a-zA-Z0-9_*\s])+?(?:\s|[*]))' # return arguments - r'([a-zA-Z$_][a-zA-Z0-9$_]*)' # method name - r'(\s*\([^;]*?\))' # signature - r'(' + _ws + r')?(;)', - bygroups(using(this), Name.Function, - using(this), Text, Punctuation)), - (r'(@interface|@implementation)(\s+)', bygroups(Keyword, Text), - 'classname'), - (r'(@class|@protocol)(\s+)', bygroups(Keyword, Text), - 'forward_classname'), - (r'(\s*)(@end)(\s*)', bygroups(Text, Keyword, Text)), - ('', Text, 'statement'), - ], - 'classname' : [ - # interface definition that inherits - ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*:\s*)([a-zA-Z$_][a-zA-Z0-9$_]*)?', - bygroups(Name.Class, Text, Name.Class), '#pop'), - # interface definition for a category - ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*)(\([a-zA-Z$_][a-zA-Z0-9$_]*\))', - bygroups(Name.Class, Text, Name.Label), '#pop'), - # simple interface / implementation - ('([a-zA-Z$_][a-zA-Z0-9$_]*)', Name.Class, '#pop') - ], - 'forward_classname' : [ - ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*,\s*)', - bygroups(Name.Class, Text), 'forward_classname'), - ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*;?)', - bygroups(Name.Class, Text), '#pop') - ], - 'statement' : [ - include('whitespace'), - include('statements'), - ('[{}]', Punctuation), - (';', Punctuation, '#pop'), - ], - 'function': [ - include('whitespace'), - include('statements'), - (';', Punctuation), - ('{', Punctuation, '#push'), - ('}', Punctuation, '#pop'), - ], - 'method': [ - include('whitespace'), - (r'(\(.*?\))([a-zA-Z$_][a-zA-Z0-9$_]*)', bygroups(using(this), - Name.Variable)), - (r'[a-zA-Z$_][a-zA-Z0-9$_]*:', Name.Function), - (';', Punctuation, '#pop'), - ('{', Punctuation, 'function'), - ('', Text, '#pop'), - ], - 'string': [ - (r'"', String, '#pop'), - (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})', String.Escape), - (r'[^\\"\n]+', String), # all other characters - (r'\\\n', String), # line continuation - (r'\\', String), # stray backslash - ], - 'macro': [ - (r'[^/\n]+', Comment.Preproc), - (r'/[*](.|\n)*?[*]/', Comment.Multiline), - (r'//.*?\n', Comment.Single, '#pop'), - (r'/', Comment.Preproc), - (r'(?<=\\)\n', Comment.Preproc), - (r'\n', Comment.Preproc, '#pop'), - ], - 'if0': [ - (r'^\s*#if.*?(?<!\\)\n', Comment.Preproc, '#push'), - (r'^\s*#endif.*?(?<!\\)\n', Comment.Preproc, '#pop'), - (r'.*?\n', Comment), + # Whitespace + (r'\s+', Text), + + # single line comment + (r'//.*?\n', Comment.Single), + + # lid header + (r'(.*?)(:)([ \t]*)(.*(?:\n[ \t].+)*)', + bygroups(Name.Attribute, Operator, Text, String)), ] } - def analyse_text(text): - if '@import' in text or '@interface' in text or \ - '@implementation' in text: - return True - elif '@"' in text: # strings - return True - elif re.match(r'\[[a-zA-Z0-9.]:', text): # message - return True - return False + +class DylanConsoleLexer(Lexer): + """ + For Dylan interactive console output like: + + .. sourcecode:: dylan-console + + ? let a = 1; + => 1 + ? a + => 1 + + This is based on a copy of the RubyConsoleLexer. + + *New in Pygments 1.6.* + """ + name = 'Dylan session' + aliases = ['dylan-console', 'dylan-repl'] + filenames = ['*.dylan-console'] + mimetypes = ['text/x-dylan-console'] + + _line_re = re.compile('.*?\n') + _prompt_re = re.compile('\?| ') + + def get_tokens_unprocessed(self, text): + dylexer = DylanLexer(**self.options) + + curcode = '' + insertions = [] + for match in self._line_re.finditer(text): + line = match.group() + m = self._prompt_re.match(line) + if m is not None: + end = m.end() + insertions.append((len(curcode), + [(0, Generic.Prompt, line[:end])])) + curcode += line[end:] + else: + if curcode: + for item in do_insertions(insertions, + dylexer.get_tokens_unprocessed(curcode)): + yield item + curcode = '' + insertions = [] + yield match.start(), Generic.Output, line + if curcode: + for item in do_insertions(insertions, + dylexer.get_tokens_unprocessed(curcode)): + yield item + + +def objective(baselexer): + """ + Generate a subclass of baselexer that accepts the Objective-C syntax + extensions. + """ + + # Have to be careful not to accidentally match JavaDoc/Doxygen syntax here, + # since that's quite common in ordinary C/C++ files. It's OK to match + # JavaDoc/Doxygen keywords that only apply to Objective-C, mind. + # + # The upshot of this is that we CANNOT match @class or @interface + _oc_keywords = re.compile(r'@(?:end|implementation|protocol)') + + # Matches [ <ws>? identifier <ws> ( identifier <ws>? ] | identifier? : ) + # (note the identifier is *optional* when there is a ':'!) + _oc_message = re.compile(r'\[\s*[a-zA-Z_][a-zA-Z0-9_]*\s+' + r'(?:[a-zA-Z_][a-zA-Z0-9_]*\s*\]|' + r'(?:[a-zA-Z_][a-zA-Z0-9_]*)?:)') + + class GeneratedObjectiveCVariant(baselexer): + """ + Implements Objective-C syntax on top of an existing C family lexer. + """ + + tokens = { + 'statements': [ + (r'@"', String, 'string'), + (r"@'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])'", + String.Char), + (r'@(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[lL]?', Number.Float), + (r'@(\d+\.\d*|\.\d+|\d+[fF])[fF]?', Number.Float), + (r'@0x[0-9a-fA-F]+[Ll]?', Number.Hex), + (r'@0[0-7]+[Ll]?', Number.Oct), + (r'@\d+[Ll]?', Number.Integer), + (r'(in|@selector|@private|@protected|@public|@encode|' + r'@synchronized|@try|@throw|@catch|@finally|@end|@property|' + r'@synthesize|@dynamic|@optional)\b', Keyword), + (r'(id|Class|IMP|SEL|BOOL|IBOutlet|IBAction|unichar)\b', + Keyword.Type), + (r'@(true|false|YES|NO)\n', Name.Builtin), + (r'(YES|NO|nil)\b', Name.Builtin), + (r'(@interface|@implementation)(\s+)', bygroups(Keyword, Text), + ('#pop', 'oc_classname')), + (r'(@class|@protocol)(\s+)', bygroups(Keyword, Text), + ('#pop', 'oc_forward_classname')), + inherit, + ], + 'oc_classname' : [ + # interface definition that inherits + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*:\s*)([a-zA-Z$_][a-zA-Z0-9$_]*)?', + bygroups(Name.Class, Text, Name.Class), '#pop'), + # interface definition for a category + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*)(\([a-zA-Z$_][a-zA-Z0-9$_]*\))', + bygroups(Name.Class, Text, Name.Label), '#pop'), + # simple interface / implementation + ('([a-zA-Z$_][a-zA-Z0-9$_]*)', Name.Class, '#pop') + ], + 'oc_forward_classname' : [ + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*,\s*)', + bygroups(Name.Class, Text), 'oc_forward_classname'), + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*;?)', + bygroups(Name.Class, Text), '#pop') + ], + 'root': [ + # methods + (r'^([-+])(\s*)' # method marker + r'(\(.*?\))?(\s*)' # return type + r'([a-zA-Z$_][a-zA-Z0-9$_]*:?)', # begin of method name + bygroups(Keyword, Text, using(this), + Text, Name.Function), + 'method'), + inherit, + ], + 'method': [ + include('whitespace'), + # TODO unsure if ellipses are allowed elsewhere, see + # discussion in Issue 789 + (r',', Punctuation), + (r'\.\.\.', Punctuation), + (r'(\(.*?\))([a-zA-Z$_][a-zA-Z0-9$_]*)', bygroups(using(this), + Name.Variable)), + (r'[a-zA-Z$_][a-zA-Z0-9$_]*:', Name.Function), + (';', Punctuation, '#pop'), + ('{', Punctuation, 'function'), + ('', Text, '#pop'), + ], + } + + def analyse_text(text): + if _oc_keywords.search(text): + return 1.0 + elif '@"' in text: # strings + return 0.8 + elif _oc_message.search(text): + return 0.8 + return 0 + + return GeneratedObjectiveCVariant + + +class ObjectiveCLexer(objective(CLexer)): + """ + For Objective-C source code with preprocessor directives. + """ + + name = 'Objective-C' + aliases = ['objective-c', 'objectivec', 'obj-c', 'objc'] + filenames = ['*.m', '*.h'] + mimetypes = ['text/x-objective-c'] + priority = 0.05 # Lower than C + + +class ObjectiveCppLexer(objective(CppLexer)): + """ + For Objective-C++ source code with preprocessor directives. + """ + + name = 'Objective-C++' + aliases = ['objective-c++', 'objectivec++', 'obj-c++', 'objc++'] + filenames = ['*.mm', '*.hh'] + mimetypes = ['text/x-objective-c++'] + priority = 0.05 # Lower than C++ class FortranLexer(RegexLexer): @@ -1296,16 +1325,16 @@ class FortranLexer(RegexLexer): ], 'core': [ # Statements - (r'\b(ACCEPT|ALLOCATABLE|ALLOCATE|ARRAY|ASSIGN|ASYNCHRONOUS|' - r'BACKSPACE|BIND|BLOCK DATA|BYTE|CALL|CASE|CLOSE|COMMON|CONTAINS|' + (r'\b(ABSTRACT|ACCEPT|ALLOCATABLE|ALLOCATE|ARRAY|ASSIGN|ASYNCHRONOUS|' + r'BACKSPACE|BIND|BLOCK( DATA)?|BYTE|CALL|CASE|CLASS|CLOSE|COMMON|CONTAINS|' r'CONTINUE|CYCLE|DATA|DEALLOCATE|DECODE|DEFERRED|DIMENSION|DO|' - r'ELSE|ENCODE|END FILE|ENDIF|END|ENTRY|ENUMERATOR|EQUIVALENCE|' + r'ELEMENTAL|ELSE|ENCODE|END( FILE)?|ENDIF|ENTRY|ENUMERATOR|EQUIVALENCE|' r'EXIT|EXTERNAL|EXTRINSIC|FINAL|FORALL|FORMAT|FUNCTION|GENERIC|' r'GOTO|IF|IMPLICIT|IMPORT|INCLUDE|INQUIRE|INTENT|INTERFACE|' r'INTRINSIC|MODULE|NAMELIST|NULLIFY|NONE|NON_INTRINSIC|' r'NON_OVERRIDABLE|NOPASS|OPEN|OPTIONAL|OPTIONS|PARAMETER|PASS|' r'PAUSE|POINTER|PRINT|PRIVATE|PROGRAM|PROTECTED|PUBLIC|PURE|READ|' - r'RECURSIVE|RETURN|REWIND|SAVE|SELECT|SEQUENCE|STOP|SUBROUTINE|' + r'RECURSIVE|RESULT|RETURN|REWIND|SAVE|SELECT|SEQUENCE|STOP|SUBROUTINE|' r'TARGET|THEN|TYPE|USE|VALUE|VOLATILE|WHERE|WRITE|WHILE)\s*\b', Keyword), @@ -1450,7 +1479,8 @@ class PrologLexer(RegexLexer): (r"'(?:''|[^'])*'", String.Atom), # quoted atom # Needs to not be followed by an atom. #(r'=(?=\s|[a-zA-Z\[])', Operator), - (r'(is|<|>|=<|>=|==|=:=|=|/|//|\*|\+|-)(?=\s|[a-zA-Z0-9\[])', + (r'is\b', Operator), + (r'(<|>|=<|>=|==|=:=|=|/|//|\*|\+|-)(?=\s|[a-zA-Z0-9\[])', Operator), (r'(mod|div|not)\b', Operator), (r'_', Keyword), # The don't-care variable @@ -1825,8 +1855,7 @@ class GoLexer(RegexLexer): (r'(var|func|struct|map|chan|type|interface|const)\b', Keyword.Declaration), (r'(break|default|select|case|defer|go' r'|else|goto|switch|fallthrough|if|range' - r'|continue|for|return)\b', Keyword - ), + r'|continue|for|return)\b', Keyword), (r'(true|false|iota|nil)\b', Keyword.Constant), # It seems the builtin types aren't actually keywords, but # can be used as functions. So we need two declarations. @@ -1836,14 +1865,13 @@ class GoLexer(RegexLexer): r'|complex64|complex128|byte|rune' r'|string|bool|error|uintptr' r'|print|println|panic|recover|close|complex|real|imag' - r'|len|cap|append|copy|delete|new|make)\b(\()', bygroups(Name.Builtin, Punctuation) - ), + r'|len|cap|append|copy|delete|new|make)\b(\()', + bygroups(Name.Builtin, Punctuation)), (r'(uint|uint8|uint16|uint32|uint64' r'|int|int8|int16|int32|int64' r'|float|float32|float64' r'|complex64|complex128|byte|rune' - r'|string|bool|error|uintptr)\b', Keyword.Type - ), + r'|string|bool|error|uintptr)\b', Keyword.Type), # imaginary_lit (r'\d+i', Number), (r'\d+\.\d*([Ee][-+]\d+)?i', Number), @@ -2162,10 +2190,12 @@ class AdaLexer(RegexLexer): (r'(pragma)(\s+)([a-zA-Z0-9_]+)', bygroups(Keyword.Reserved, Text, Comment.Preproc)), (r'(true|false|null)\b', Keyword.Constant), - (r'(Byte|Character|Float|Integer|Long_Float|Long_Integer|' - r'Long_Long_Float|Long_Long_Integer|Natural|Positive|Short_Float|' - r'Short_Integer|Short_Short_Float|Short_Short_Integer|String|' - r'Wide_String|Duration)\b', Keyword.Type), + (r'(Address|Byte|Boolean|Character|Controlled|Count|Cursor|' + r'Duration|File_Mode|File_Type|Float|Generator|Integer|Long_Float|' + r'Long_Integer|Long_Long_Float|Long_Long_Integer|Natural|Positive|' + r'Reference_Type|Short_Float|Short_Integer|Short_Short_Float|' + r'Short_Short_Integer|String|Wide_Character|Wide_String)\b', + Keyword.Type), (r'(and(\s+then)?|in|mod|not|or(\s+else)|rem)\b', Operator.Word), (r'generic|private', Keyword.Declaration), (r'package', Keyword.Declaration, 'package'), @@ -2220,7 +2250,8 @@ class AdaLexer(RegexLexer): (r'\(', Punctuation, 'formal_part'), (r'with|and|use', Keyword.Reserved), (r'array\b', Keyword.Reserved, ('#pop', 'array_def')), - (r'record\b', Keyword.Reserved, ('formal_part')), + (r'record\b', Keyword.Reserved, ('record_def')), + (r'(null record)(;)', bygroups(Keyword.Reserved, Punctuation), '#pop'), include('root'), ], 'array_def' : [ @@ -2229,6 +2260,10 @@ class AdaLexer(RegexLexer): Keyword.Reserved)), include('root'), ], + 'record_def' : [ + (r'end record', Keyword.Reserved, '#pop'), + include('root'), + ], 'import': [ (r'[a-z0-9_.]+', Name.Namespace, '#pop'), (r'', Text, '#pop'), @@ -2468,7 +2503,9 @@ class BlitzMaxLexer(RegexLexer): bmax_sktypes = r'@{1,2}|[!#$%]' bmax_lktypes = r'\b(Int|Byte|Short|Float|Double|Long)\b' bmax_name = r'[a-z_][a-z0-9_]*' - bmax_var = r'(%s)(?:(?:([ \t]*)(%s)|([ \t]*:[ \t]*\b(?:Shl|Shr|Sar|Mod)\b)|([ \t]*)([:])([ \t]*)(?:%s|(%s)))(?:([ \t]*)(Ptr))?)' % (bmax_name, bmax_sktypes, bmax_lktypes, bmax_name) + bmax_var = (r'(%s)(?:(?:([ \t]*)(%s)|([ \t]*:[ \t]*\b(?:Shl|Shr|Sar|Mod)\b)' + r'|([ \t]*)([:])([ \t]*)(?:%s|(%s)))(?:([ \t]*)(Ptr))?)') % \ + (bmax_name, bmax_sktypes, bmax_lktypes, bmax_name) bmax_func = bmax_var + r'?((?:[ \t]|\.\.\n)*)([(])' flags = re.MULTILINE | re.IGNORECASE @@ -2937,13 +2974,13 @@ class RustLexer(RegexLexer): (r'/[*](.|\n)*?[*]/', Comment.Multiline), # Keywords - (r'(alt|as|assert|be|break|check|claim|class|const' - r'|cont|copy|crust|do|else|enum|export|fail' - r'|false|fn|for|if|iface|impl|import|let|log' - r'|loop|mod|mut|native|pure|resource|ret|true' - r'|type|unsafe|use|white|note|bind|prove|unchecked' - r'|with|syntax|u8|u16|u32|u64|i8|i16|i32|i64|uint' - r'|int|f32|f64)\b', Keyword), + (r'(as|assert|break|const' + r'|copy|do|else|enum|extern|fail' + r'|false|fn|for|if|impl|let|log' + r'|loop|match|mod|move|mut|once|priv|pub|pure' + r'|ref|return|static|struct|trait|true|type|unsafe|use|while' + r'|u8|u16|u32|u64|i8|i16|i32|i64|uint' + r'|int|float|f32|f64|str)\b', Keyword), # Character Literal (r"""'(\\['"\\nrt]|\\x[0-9a-fA-F]{2}|\\[0-7]{1,3}""" @@ -2972,8 +3009,8 @@ class RustLexer(RegexLexer): (r'#\[', Comment.Preproc, 'attribute['), (r'#\(', Comment.Preproc, 'attribute('), # Macros - (r'#[A-Za-z_][A-Za-z0-9_]*\[', Comment.Preproc, 'attribute['), - (r'#[A-Za-z_][A-Za-z0-9_]*\(', Comment.Preproc, 'attribute('), + (r'[A-Za-z_][A-Za-z0-9_]*!\[', Comment.Preproc, 'attribute['), + (r'[A-Za-z_][A-Za-z0-9_]*!\(', Comment.Preproc, 'attribute('), ], 'number_lit': [ (r'(([ui](8|16|32|64)?)|(f(32|64)?))?', Keyword, '#pop'), @@ -3003,7 +3040,7 @@ class RustLexer(RegexLexer): } -class CUDALexer(CLexer): +class CudaLexer(CLexer): """ For NVIDIA `CUDA™ <http://developer.nvidia.com/category/zone/cuda-zone>`_ source. @@ -3050,3 +3087,410 @@ class CUDALexer(CLexer): elif value in self.functions: token = Name.Function yield index, token, value + + +class MonkeyLexer(RegexLexer): + """ + For + `Monkey <https://en.wikipedia.org/wiki/Monkey_(programming_language)>`_ + source code. + + *New in Pygments 1.6.* + """ + + name = 'Monkey' + aliases = ['monkey'] + filenames = ['*.monkey'] + mimetypes = ['text/x-monkey'] + + name_variable = r'[a-z_][a-zA-Z0-9_]*' + name_function = r'[A-Z][a-zA-Z0-9_]*' + name_constant = r'[A-Z_][A-Z0-9_]*' + name_class = r'[A-Z][a-zA-Z0-9_]*' + name_module = r'[a-z0-9_]*' + + keyword_type = r'(?:Int|Float|String|Bool|Object|Array|Void)' + # ? == Bool // % == Int // # == Float // $ == String + keyword_type_special = r'[?%#$]' + + flags = re.MULTILINE + + tokens = { + 'root': [ + #Text + (r'\s+', Text), + # Comments + (r"'.*", Comment), + (r'(?i)^#rem\b', Comment.Multiline, 'comment'), + # preprocessor directives + (r'(?i)^(?:#If|#ElseIf|#Else|#EndIf|#End|#Print|#Error)\b', Comment.Preproc), + # preprocessor variable (any line starting with '#' that is not a directive) + (r'^#', Comment.Preproc, 'variables'), + # String + ('"', String.Double, 'string'), + # Numbers + (r'[0-9]+\.[0-9]*(?!\.)', Number.Float), + (r'\.[0-9]+(?!\.)', Number.Float), + (r'[0-9]+', Number.Integer), + (r'\$[0-9a-fA-Z]+', Number.Hex), + (r'\%[10]+', Number), # Binary + # Native data types + (r'\b%s\b' % keyword_type, Keyword.Type), + # Exception handling + (r'(?i)\b(?:Try|Catch|Throw)\b', Keyword.Reserved), + (r'Throwable', Name.Exception), + # Builtins + (r'(?i)\b(?:Null|True|False)\b', Name.Builtin), + (r'(?i)\b(?:Self|Super)\b', Name.Builtin.Pseudo), + (r'\b(?:HOST|LANG|TARGET|CONFIG)\b', Name.Constant), + # Keywords + (r'(?i)^(Import)(\s+)(.*)(\n)', + bygroups(Keyword.Namespace, Text, Name.Namespace, Text)), + (r'(?i)^Strict\b.*\n', Keyword.Reserved), + (r'(?i)(Const|Local|Global|Field)(\s+)', + bygroups(Keyword.Declaration, Text), 'variables'), + (r'(?i)(New|Class|Interface|Extends|Implements)(\s+)', + bygroups(Keyword.Reserved, Text), 'classname'), + (r'(?i)(Function|Method)(\s+)', + bygroups(Keyword.Reserved, Text), 'funcname'), + (r'(?i)(?:End|Return|Public|Private|Extern|Property|' + r'Final|Abstract)\b', Keyword.Reserved), + # Flow Control stuff + (r'(?i)(?:If|Then|Else|ElseIf|EndIf|' + r'Select|Case|Default|' + r'While|Wend|' + r'Repeat|Until|Forever|' + r'For|To|Until|Step|EachIn|Next|' + r'Exit|Continue)\s+', Keyword.Reserved), + # not used yet + (r'(?i)\b(?:Module|Inline)\b', Keyword.Reserved), + # Array + (r'[\[\]]', Punctuation), + # Other + (r'<=|>=|<>|\*=|/=|\+=|-=|&=|~=|\|=|[-&*/^+=<>|~]', Operator), + (r'(?i)(?:Not|Mod|Shl|Shr|And|Or)', Operator.Word), + (r'[\(\){}!#,.:]', Punctuation), + # catch the rest + (r'%s\b' % name_constant, Name.Constant), + (r'%s\b' % name_function, Name.Function), + (r'%s\b' % name_variable, Name.Variable), + ], + 'funcname': [ + (r'(?i)%s\b' % name_function, Name.Function), + (r':', Punctuation, 'classname'), + (r'\s+', Text), + (r'\(', Punctuation, 'variables'), + (r'\)', Punctuation, '#pop') + ], + 'classname': [ + (r'%s\.' % name_module, Name.Namespace), + (r'%s\b' % keyword_type, Keyword.Type), + (r'%s\b' % name_class, Name.Class), + # array (of given size) + (r'(\[)(\s*)(\d*)(\s*)(\])', + bygroups(Punctuation, Text, Number.Integer, Text, Punctuation)), + # generics + (r'\s+(?!<)', Text, '#pop'), + (r'<', Punctuation, '#push'), + (r'>', Punctuation, '#pop'), + (r'\n', Text, '#pop'), + (r'', Text, '#pop') + ], + 'variables': [ + (r'%s\b' % name_constant, Name.Constant), + (r'%s\b' % name_variable, Name.Variable), + (r'%s' % keyword_type_special, Keyword.Type), + (r'\s+', Text), + (r':', Punctuation, 'classname'), + (r',', Punctuation, '#push'), + (r'', Text, '#pop') + ], + 'string': [ + (r'[^"~]+', String.Double), + (r'~q|~n|~r|~t|~z|~~', String.Escape), + (r'"', String.Double, '#pop'), + ], + 'comment' : [ + (r'(?i)^#rem.*?', Comment.Multiline, "#push"), + (r'(?i)^#end.*?', Comment.Multiline, "#pop"), + (r'\n', Comment.Multiline), + (r'.+', Comment.Multiline), + ], + } + + +class CobolLexer(RegexLexer): + """ + Lexer for OpenCOBOL code. + + *New in Pygments 1.6.* + """ + name = 'COBOL' + aliases = ['cobol'] + filenames = ['*.cob', '*.COB', '*.cpy', '*.CPY'] + mimetypes = ['text/x-cobol'] + flags = re.IGNORECASE | re.MULTILINE + + # Data Types: by PICTURE and USAGE + # Operators: **, *, +, -, /, <, >, <=, >=, =, <> + # Logical (?): NOT, AND, OR + + # Reserved words: + # http://opencobol.add1tocobol.com/#reserved-words + # Intrinsics: + # http://opencobol.add1tocobol.com/#does-opencobol-implement-any-intrinsic-functions + + tokens = { + 'root': [ + include('comment'), + include('strings'), + include('core'), + include('nums'), + (r'[a-z0-9]([_a-z0-9\-]*[a-z0-9]+)?', Name.Variable), + # (r'[\s]+', Text), + (r'[ \t]+', Text), + ], + 'comment': [ + (r'(^.{6}[*/].*\n|^.{6}|\*>.*\n)', Comment), + ], + 'core': [ + # Figurative constants + (r'(^|(?<=[^0-9a-z_\-]))(ALL\s+)?' + r'((ZEROES)|(HIGH-VALUE|LOW-VALUE|QUOTE|SPACE|ZERO)(S)?)' + r'\s*($|(?=[^0-9a-z_\-]))', + Name.Constant), + + # Reserved words STATEMENTS and other bolds + (r'(^|(?<=[^0-9a-z_\-]))' + r'(ACCEPT|ADD|ALLOCATE|CALL|CANCEL|CLOSE|COMPUTE|' + r'CONFIGURATION|CONTINUE|' + r'DATA|DELETE|DISPLAY|DIVIDE|DIVISION|ELSE|END|END-ACCEPT|' + r'END-ADD|END-CALL|END-COMPUTE|END-DELETE|END-DISPLAY|' + r'END-DIVIDE|END-EVALUATE|END-IF|END-MULTIPLY|END-OF-PAGE|' + r'END-PERFORM|END-READ|END-RETURN|END-REWRITE|END-SEARCH|' + r'END-START|END-STRING|END-SUBTRACT|END-UNSTRING|END-WRITE|' + r'ENVIRONMENT|EVALUATE|EXIT|FD|FILE|FILE-CONTROL|FOREVER|' + r'FREE|GENERATE|GO|GOBACK|' + r'IDENTIFICATION|IF|INITIALIZE|' + r'INITIATE|INPUT-OUTPUT|INSPECT|INVOKE|I-O-CONTROL|LINKAGE|' + r'LOCAL-STORAGE|MERGE|MOVE|MULTIPLY|OPEN|' + r'PERFORM|PROCEDURE|PROGRAM-ID|RAISE|READ|RELEASE|RESUME|' + r'RETURN|REWRITE|SCREEN|' + r'SD|SEARCH|SECTION|SET|SORT|START|STOP|STRING|SUBTRACT|' + r'SUPPRESS|TERMINATE|THEN|UNLOCK|UNSTRING|USE|VALIDATE|' + r'WORKING-STORAGE|WRITE)' + r'\s*($|(?=[^0-9a-z_\-]))', Keyword.Reserved), + + # Reserved words + (r'(^|(?<=[^0-9a-z_\-]))' + r'(ACCESS|ADDRESS|ADVANCING|AFTER|ALL|' + r'ALPHABET|ALPHABETIC|ALPHABETIC-LOWER|ALPHABETIC-UPPER|' + r'ALPHANUMERIC|ALPHANUMERIC-EDITED|ALSO|ALTER|ALTERNATE' + r'ANY|ARE|AREA|AREAS|ARGUMENT-NUMBER|ARGUMENT-VALUE|AS|' + r'ASCENDING|ASSIGN|AT|AUTO|AUTO-SKIP|AUTOMATIC|AUTOTERMINATE|' + r'BACKGROUND-COLOR|BASED|BEEP|BEFORE|BELL|' + r'BLANK|' + r'BLINK|BLOCK|BOTTOM|BY|BYTE-LENGTH|CHAINING|' + r'CHARACTER|CHARACTERS|CLASS|CODE|CODE-SET|COL|COLLATING|' + r'COLS|COLUMN|COLUMNS|COMMA|COMMAND-LINE|COMMIT|COMMON|' + r'CONSTANT|CONTAINS|CONTENT|CONTROL|' + r'CONTROLS|CONVERTING|COPY|CORR|CORRESPONDING|COUNT|CRT|' + r'CURRENCY|CURSOR|CYCLE|DATE|DAY|DAY-OF-WEEK|DE|DEBUGGING|' + r'DECIMAL-POINT|DECLARATIVES|DEFAULT|DELIMITED|' + r'DELIMITER|DEPENDING|DESCENDING|DETAIL|DISK|' + r'DOWN|DUPLICATES|DYNAMIC|EBCDIC|' + r'ENTRY|ENVIRONMENT-NAME|ENVIRONMENT-VALUE|EOL|EOP|' + r'EOS|ERASE|ERROR|ESCAPE|EXCEPTION|' + r'EXCLUSIVE|EXTEND|EXTERNAL|' + r'FILE-ID|FILLER|FINAL|FIRST|FIXED|FLOAT-LONG|FLOAT-SHORT|' + r'FOOTING|FOR|FOREGROUND-COLOR|FORMAT|FROM|FULL|FUNCTION|' + r'FUNCTION-ID|GIVING|GLOBAL|GROUP|' + r'HEADING|HIGHLIGHT|I-O|ID|' + r'IGNORE|IGNORING|IN|INDEX|INDEXED|INDICATE|' + r'INITIAL|INITIALIZED|INPUT|' + r'INTO|INTRINSIC|INVALID|IS|JUST|JUSTIFIED|KEY|LABEL|' + r'LAST|LEADING|LEFT|LENGTH|LIMIT|LIMITS|LINAGE|' + r'LINAGE-COUNTER|LINE|LINES|LOCALE|LOCK|' + r'LOWLIGHT|MANUAL|MEMORY|MINUS|MODE|' + r'MULTIPLE|NATIONAL|NATIONAL-EDITED|NATIVE|' + r'NEGATIVE|NEXT|NO|NULL|NULLS|NUMBER|NUMBERS|NUMERIC|' + r'NUMERIC-EDITED|OBJECT-COMPUTER|OCCURS|OF|OFF|OMITTED|ON|ONLY|' + r'OPTIONAL|ORDER|ORGANIZATION|OTHER|OUTPUT|OVERFLOW|' + r'OVERLINE|PACKED-DECIMAL|PADDING|PAGE|PARAGRAPH|' + r'PLUS|POINTER|POSITION|POSITIVE|PRESENT|PREVIOUS|' + r'PRINTER|PRINTING|PROCEDURE-POINTER|PROCEDURES|' + r'PROCEED|PROGRAM|PROGRAM-POINTER|PROMPT|QUOTE|' + r'QUOTES|RANDOM|RD|RECORD|RECORDING|RECORDS|RECURSIVE|' + r'REDEFINES|REEL|REFERENCE|RELATIVE|REMAINDER|REMOVAL|' + r'RENAMES|REPLACING|REPORT|REPORTING|REPORTS|REPOSITORY|' + r'REQUIRED|RESERVE|RETURNING|REVERSE-VIDEO|REWIND|' + r'RIGHT|ROLLBACK|ROUNDED|RUN|SAME|SCROLL|' + r'SECURE|SEGMENT-LIMIT|SELECT|SENTENCE|SEPARATE|' + r'SEQUENCE|SEQUENTIAL|SHARING|SIGN|SIGNED|SIGNED-INT|' + r'SIGNED-LONG|SIGNED-SHORT|SIZE|SORT-MERGE|SOURCE|' + r'SOURCE-COMPUTER|SPECIAL-NAMES|STANDARD|' + r'STANDARD-1|STANDARD-2|STATUS|SUM|' + r'SYMBOLIC|SYNC|SYNCHRONIZED|TALLYING|TAPE|' + r'TEST|THROUGH|THRU|TIME|TIMES|TO|TOP|TRAILING|' + r'TRANSFORM|TYPE|UNDERLINE|UNIT|UNSIGNED|' + r'UNSIGNED-INT|UNSIGNED-LONG|UNSIGNED-SHORT|UNTIL|UP|' + r'UPDATE|UPON|USAGE|USING|VALUE|VALUES|VARYING|WAIT|WHEN|' + r'WITH|WORDS|YYYYDDD|YYYYMMDD)' + r'\s*($|(?=[^0-9a-z_\-]))', Keyword.Pseudo), + + # inactive reserved words + (r'(^|(?<=[^0-9a-z_\-]))' + r'(ACTIVE-CLASS|ALIGNED|ANYCASE|ARITHMETIC|ATTRIBUTE|B-AND|' + r'B-NOT|B-OR|B-XOR|BIT|BOOLEAN|CD|CENTER|CF|CH|CHAIN|CLASS-ID|' + r'CLASSIFICATION|COMMUNICATION|CONDITION|DATA-POINTER|' + r'DESTINATION|DISABLE|EC|EGI|EMI|ENABLE|END-RECEIVE|' + r'ENTRY-CONVENTION|EO|ESI|EXCEPTION-OBJECT|EXPANDS|FACTORY|' + r'FLOAT-BINARY-16|FLOAT-BINARY-34|FLOAT-BINARY-7|' + r'FLOAT-DECIMAL-16|FLOAT-DECIMAL-34|FLOAT-EXTENDED|FORMAT|' + r'FUNCTION-POINTER|GET|GROUP-USAGE|IMPLEMENTS|INFINITY|' + r'INHERITS|INTERFACE|INTERFACE-ID|INVOKE|LC_ALL|LC_COLLATE|' + r'LC_CTYPE|LC_MESSAGES|LC_MONETARY|LC_NUMERIC|LC_TIME|' + r'LINE-COUNTER|MESSAGE|METHOD|METHOD-ID|NESTED|NONE|NORMAL|' + r'OBJECT|OBJECT-REFERENCE|OPTIONS|OVERRIDE|PAGE-COUNTER|PF|PH|' + r'PROPERTY|PROTOTYPE|PURGE|QUEUE|RAISE|RAISING|RECEIVE|' + r'RELATION|REPLACE|REPRESENTS-NOT-A-NUMBER|RESET|RESUME|RETRY|' + r'RF|RH|SECONDS|SEGMENT|SELF|SEND|SOURCES|STATEMENT|STEP|' + r'STRONG|SUB-QUEUE-1|SUB-QUEUE-2|SUB-QUEUE-3|SUPER|SYMBOL|' + r'SYSTEM-DEFAULT|TABLE|TERMINAL|TEXT|TYPEDEF|UCS-4|UNIVERSAL|' + r'USER-DEFAULT|UTF-16|UTF-8|VAL-STATUS|VALID|VALIDATE|' + r'VALIDATE-STATUS)\s*($|(?=[^0-9a-z_\-]))', Error), + + # Data Types + (r'(^|(?<=[^0-9a-z_\-]))' + r'(PIC\s+.+?(?=(\s|\.\s))|PICTURE\s+.+?(?=(\s|\.\s))|' + r'(COMPUTATIONAL)(-[1-5X])?|(COMP)(-[1-5X])?|' + r'BINARY-C-LONG|' + r'BINARY-CHAR|BINARY-DOUBLE|BINARY-LONG|BINARY-SHORT|' + r'BINARY)\s*($|(?=[^0-9a-z_\-]))', Keyword.Type), + + # Operators + (r'(\*\*|\*|\+|-|/|<=|>=|<|>|==|/=|=)', Operator), + + # (r'(::)', Keyword.Declaration), + + (r'([(),;:&%.])', Punctuation), + + # Intrinsics + (r'(^|(?<=[^0-9a-z_\-]))(ABS|ACOS|ANNUITY|ASIN|ATAN|BYTE-LENGTH|' + r'CHAR|COMBINED-DATETIME|CONCATENATE|COS|CURRENT-DATE|' + r'DATE-OF-INTEGER|DATE-TO-YYYYMMDD|DAY-OF-INTEGER|DAY-TO-YYYYDDD|' + r'EXCEPTION-(?:FILE|LOCATION|STATEMENT|STATUS)|EXP10|EXP|E|' + r'FACTORIAL|FRACTION-PART|INTEGER-OF-(?:DATE|DAY|PART)|INTEGER|' + r'LENGTH|LOCALE-(?:DATE|TIME(?:-FROM-SECONDS)?)|LOG10|LOG|' + r'LOWER-CASE|MAX|MEAN|MEDIAN|MIDRANGE|MIN|MOD|NUMVAL(?:-C)?|' + r'ORD(?:-MAX|-MIN)?|PI|PRESENT-VALUE|RANDOM|RANGE|REM|REVERSE|' + r'SECONDS-FROM-FORMATTED-TIME|SECONDS-PAST-MIDNIGHT|SIGN|SIN|SQRT|' + r'STANDARD-DEVIATION|STORED-CHAR-LENGTH|SUBSTITUTE(?:-CASE)?|' + r'SUM|TAN|TEST-DATE-YYYYMMDD|TEST-DAY-YYYYDDD|TRIM|' + r'UPPER-CASE|VARIANCE|WHEN-COMPILED|YEAR-TO-YYYY)\s*' + r'($|(?=[^0-9a-z_\-]))', Name.Function), + + # Booleans + (r'(^|(?<=[^0-9a-z_\-]))(true|false)\s*($|(?=[^0-9a-z_\-]))', Name.Builtin), + # Comparing Operators + (r'(^|(?<=[^0-9a-z_\-]))(equal|equals|ne|lt|le|gt|ge|' + r'greater|less|than|not|and|or)\s*($|(?=[^0-9a-z_\-]))', Operator.Word), + ], + + # \"[^\"\n]*\"|\'[^\'\n]*\' + 'strings': [ + # apparently strings can be delimited by EOL if they are continued + # in the next line + (r'"[^"\n]*("|\n)', String.Double), + (r"'[^'\n]*('|\n)", String.Single), + ], + + 'nums': [ + (r'\d+(\s*|\.$|$)', Number.Integer), + (r'[+-]?\d*\.\d+([eE][-+]?\d+)?', Number.Float), + (r'[+-]?\d+\.\d*([eE][-+]?\d+)?', Number.Float), + ], + } + + +class CobolFreeformatLexer(CobolLexer): + """ + Lexer for Free format OpenCOBOL code. + + *New in Pygments 1.6.* + """ + name = 'COBOLFree' + aliases = ['cobolfree'] + filenames = ['*.cbl', '*.CBL'] + mimetypes = [] + flags = re.IGNORECASE | re.MULTILINE + + tokens = { + 'comment': [ + (r'(\*>.*\n|^\w*\*.*$)', Comment), + ], + } + + +class LogosLexer(ObjectiveCppLexer): + """ + For Logos + Objective-C source code with preprocessor directives. + + *New in Pygments 1.6.* + """ + + name = 'Logos' + aliases = ['logos'] + filenames = ['*.x', '*.xi', '*.xm', '*.xmi'] + mimetypes = ['text/x-logos'] + priority = 0.25 + + tokens = { + 'statements': [ + (r'(%orig|%log)\b', Keyword), + (r'(%c)\b(\()(\s*)([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*)(\))', + bygroups(Keyword, Punctuation, Text, Name.Class, Text, Punctuation)), + (r'(%init)\b(\()', + bygroups(Keyword, Punctuation), 'logos_init_directive'), + (r'(%init)(?=\s*;)', bygroups(Keyword)), + (r'(%hook|%group)(\s+)([a-zA-Z$_][a-zA-Z0-9$_]+)', + bygroups(Keyword, Text, Name.Class), '#pop'), + (r'(%subclass)(\s+)', bygroups(Keyword, Text), + ('#pop', 'logos_classname')), + inherit, + ], + 'logos_init_directive' : [ + ('\s+', Text), + (',', Punctuation, ('logos_init_directive', '#pop')), + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*)(=)(\s*)([^);]*)', + bygroups(Name.Class, Text, Punctuation, Text, Text)), + ('([a-zA-Z$_][a-zA-Z0-9$_]*)', Name.Class), + ('\)', Punctuation, '#pop'), + ], + 'logos_classname' : [ + ('([a-zA-Z$_][a-zA-Z0-9$_]*)(\s*:\s*)([a-zA-Z$_][a-zA-Z0-9$_]*)?', + bygroups(Name.Class, Text, Name.Class), '#pop'), + ('([a-zA-Z$_][a-zA-Z0-9$_]*)', Name.Class, '#pop') + ], + 'root': [ + (r'(%subclass)(\s+)', bygroups(Keyword, Text), + 'logos_classname'), + (r'(%hook|%group)(\s+)([a-zA-Z$_][a-zA-Z0-9$_]+)', + bygroups(Keyword, Text, Name.Class)), + (r'(%config)(\s*\(\s*)(\w+)(\s*=\s*)(.*?)(\s*\)\s*)', + bygroups(Keyword, Text, Name.Variable, Text, String, Text)), + (r'(%ctor)(\s*)({)', bygroups(Keyword, Text, Punctuation), + 'function'), + (r'(%new)(\s*)(\()(\s*.*?\s*)(\))', + bygroups(Keyword, Text, Keyword, String, Keyword)), + (r'(\s*)(%end)(\s*)', bygroups(Text, Keyword, Text)), + inherit, + ], + } + + _logos_keywords = re.compile(r'%(?:hook|ctor|init|c\()') + + def analyse_text(text): + if LogosLexer._logos_keywords.search(text): + return 1.0 + return 0 diff --git a/pygments/lexers/dalvik.py b/pygments/lexers/dalvik.py new file mode 100644 index 00000000..de9b11fa --- /dev/null +++ b/pygments/lexers/dalvik.py @@ -0,0 +1,104 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.dalvik + ~~~~~~~~~~~~~~~~~~~~~~ + + Pygments lexers for Dalvik VM-related languages. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from pygments.lexer import RegexLexer, include, bygroups +from pygments.token import Keyword, Text, Comment, Name, String, Number, \ + Punctuation + +__all__ = ['SmaliLexer'] + + +class SmaliLexer(RegexLexer): + """ + For `Smali <http://code.google.com/p/smali/>`_ (Android/Dalvik) assembly + code. + + *New in Pygments 1.6.* + """ + name = 'Smali' + aliases = ['smali'] + filenames = ['*.smali'] + mimetypes = ['text/smali'] + + tokens = { + 'root': [ + include('comment'), + include('label'), + include('field'), + include('method'), + include('class'), + include('directive'), + include('access-modifier'), + include('instruction'), + include('literal'), + include('punctuation'), + include('type'), + include('whitespace') + ], + 'directive': [ + (r'^[ \t]*\.(class|super|implements|field|subannotation|annotation|' + r'enum|method|registers|locals|array-data|packed-switch|' + r'sparse-switch|catchall|catch|line|parameter|local|prologue|' + r'epilogue|source)', Keyword), + (r'^[ \t]*\.end (field|subannotation|annotation|method|array-data|' + 'packed-switch|sparse-switch|parameter|local)', Keyword), + (r'^[ \t]*\.restart local', Keyword), + ], + 'access-modifier': [ + (r'(public|private|protected|static|final|synchronized|bridge|' + r'varargs|native|abstract|strictfp|synthetic|constructor|' + r'declared-synchronized|interface|enum|annotation|volatile|' + r'transient)', Keyword), + ], + 'whitespace': [ + (r'\n', Text), + (r'\s+', Text), + ], + 'instruction': [ + (r'\b[vp]\d+\b', Name.Builtin), # registers + (r'\b[a-z][A-Za-z0-9/-]+\s+', Text), # instructions + ], + 'literal': [ + (r'".*"', String), + (r'0x[0-9A-Fa-f]+t?', Number.Hex), + (r'[0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), + (r'[0-9]+L?', Number.Integer), + ], + 'field': [ + (r'(\$?\b)([A-Za-z0-9_$]*)(:)', + bygroups(Punctuation, Name.Variable, Punctuation)), + ], + 'method': [ + (r'<(?:cl)?init>', Name.Function), # constructor + (r'(\$?\b)([A-Za-z0-9_$]*)(\()', + bygroups(Punctuation, Name.Function, Punctuation)), + ], + 'label': [ + (r':[A-Za-z0-9_]+', Name.Label), + ], + 'class': [ + # class names in the form Lcom/namespace/ClassName; + # I only want to color the ClassName part, so the namespace part is + # treated as 'Text' + (r'(L)((?:[A-Za-z0-9_$]+/)*)([A-Za-z0-9_$]+)(;)', + bygroups(Keyword.Type, Text, Name.Class, Text)), + ], + 'punctuation': [ + (r'->', Punctuation), + (r'[{},\(\):=\.-]', Punctuation), + ], + 'type': [ + (r'[ZBSCIJFDV\[]+', Keyword.Type), + ], + 'comment': [ + (r'#.*?\n', Comment), + ], + } diff --git a/pygments/lexers/dotnet.py b/pygments/lexers/dotnet.py index 17edddc0..bdd9edc1 100644 --- a/pygments/lexers/dotnet.py +++ b/pygments/lexers/dotnet.py @@ -5,7 +5,7 @@ Lexers for .net languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import re diff --git a/pygments/lexers/foxpro.py b/pygments/lexers/foxpro.py new file mode 100644 index 00000000..51cd499b --- /dev/null +++ b/pygments/lexers/foxpro.py @@ -0,0 +1,428 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.foxpro + ~~~~~~~~~~~~~~~~~~~~~~ + + Simple lexer for Microsoft Visual FoxPro source code. + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import RegexLexer +from pygments.token import Punctuation, Text, Comment, Operator, Keyword, \ + Name, String + +__all__ = ['FoxProLexer'] + + +class FoxProLexer(RegexLexer): + """Lexer for Microsoft Visual FoxPro language. + + FoxPro syntax allows to shorten all keywords and function names + to 4 characters. Shortened forms are not recognized by this lexer. + + *New in Pygments 1.6.* + """ + + name = 'FoxPro' + aliases = ['Clipper', 'XBase'] + filenames = ['*.PRG', '*.prg'] + mimetype = [] + + flags = re.IGNORECASE | re.MULTILINE + + tokens = { + 'root': [ + (r';\s*\n', Punctuation), # consume newline + (r'(^|\n)\s*', Text, 'newline'), + + # Square brackets may be used for array indices + # and for string literal. Look for arrays + # before matching string literals. + (r'(?<=\w)\[[0-9, ]+\]', Text), + (r'\'[^\'\n]*\'|"[^"\n]*"|\[[^]*]\]', String), + (r'(^\s*\*|&&|&&).*?\n', Comment.Single), + + (r'(ABS|ACLASS|ACOPY|ACOS|ADATABASES|ADBOBJECTS|ADDBS|' + r'ADDPROPERTY|ADEL|ADIR|ADLLS|ADOCKSTATE|AELEMENT|AERROR|' + r'AEVENTS|AFIELDS|AFONT|AGETCLASS|AGETFILEVERSION|AINS|' + r'AINSTANCE|ALANGUAGE|ALEN|ALIAS|ALINES|ALLTRIM|' + r'AMEMBERS|AMOUSEOBJ|ANETRESOURCES|APRINTERS|APROCINFO|' + r'ASC|ASCAN|ASELOBJ|ASESSIONS|ASIN|ASORT|ASQLHANDLES|' + r'ASTACKINFO|ASUBSCRIPT|AT|AT_C|ATAGINFO|ATAN|ATC|ATCC|' + r'ATCLINE|ATLINE|ATN2|AUSED|AVCXCLASSES|BAR|BARCOUNT|' + r'BARPROMPT|BETWEEN|BINDEVENT|BINTOC|BITAND|BITCLEAR|' + r'BITLSHIFT|BITNOT|BITOR|BITRSHIFT|BITSET|BITTEST|BITXOR|' + r'BOF|CANDIDATE|CAPSLOCK|CAST|CDOW|CDX|CEILING|CHR|CHRSAW|' + r'CHRTRAN|CHRTRANC|CLEARRESULTSET|CMONTH|CNTBAR|CNTPAD|COL|' + r'COM|Functions|COMARRAY|COMCLASSINFO|COMPOBJ|COMPROP|' + r'COMRETURNERROR|COS|CPCONVERT|CPCURRENT|CPDBF|CREATEBINARY|' + r'CREATEOBJECT|CREATEOBJECTEX|CREATEOFFLINE|CTOBIN|CTOD|' + r'CTOT|CURDIR|CURSORGETPROP|CURSORSETPROP|CURSORTOXML|' + r'CURVAL|DATE|DATETIME|DAY|DBC|DBF|DBGETPROP|DBSETPROP|' + r'DBUSED|DDEAbortTrans|DDEAdvise|DDEEnabled|DDEExecute|' + r'DDEInitiate|DDELastError|DDEPoke|DDERequest|DDESetOption|' + r'DDESetService|DDESetTopic|DDETerminate|DEFAULTEXT|' + r'DELETED|DESCENDING|DIFFERENCE|DIRECTORY|DISKSPACE|' + r'DisplayPath|DMY|DODEFAULT|DOW|DRIVETYPE|DROPOFFLINE|' + r'DTOC|DTOR|DTOS|DTOT|EDITSOURCE|EMPTY|EOF|ERROR|EVAL(UATE)?|' + r'EVENTHANDLER|EVL|EXECSCRIPT|EXP|FCHSIZE|FCLOSE|FCOUNT|' + r'FCREATE|FDATE|FEOF|FERROR|FFLUSH|FGETS|FIELD|FILE|' + r'FILETOSTR|FILTER|FKLABEL|FKMAX|FLDLIST|FLOCK|FLOOR|' + r'FONTMETRIC|FOPEN|FOR|FORCEEXT|FORCEPATH|FOUND|FPUTS|' + r'FREAD|FSEEK|FSIZE|FTIME|FULLPATH|FV|FWRITE|' + r'GETAUTOINCVALUE|GETBAR|GETCOLOR|GETCP|GETDIR|GETENV|' + r'GETFILE|GETFLDSTATE|GETFONT|GETINTERFACE|' + r'GETNEXTMODIFIED|GETOBJECT|GETPAD|GETPEM|GETPICT|' + r'GETPRINTER|GETRESULTSET|GETWORDCOUNT|GETWORDNUM|' + r'GETCURSORADAPTER|GOMONTH|HEADER|HOME|HOUR|ICASE|' + r'IDXCOLLATE|IIF|IMESTATUS|INDBC|INDEXSEEK|INKEY|INLIST|' + r'INPUTBOX|INSMODE|INT|ISALPHA|ISBLANK|ISCOLOR|ISDIGIT|' + r'ISEXCLUSIVE|ISFLOCKED|ISLEADBYTE|ISLOWER|ISMEMOFETCHED|' + r'ISMOUSE|ISNULL|ISPEN|ISREADONLY|ISRLOCKED|' + r'ISTRANSACTABLE|ISUPPER|JUSTDRIVE|JUSTEXT|JUSTFNAME|' + r'JUSTPATH|JUSTSTEM|KEY|KEYMATCH|LASTKEY|LEFT|LEFTC|LEN|' + r'LENC|LIKE|LIKEC|LINENO|LOADPICTURE|LOCFILE|LOCK|LOG|' + r'LOG10|LOOKUP|LOWER|LTRIM|LUPDATE|MAKETRANSACTABLE|MAX|' + r'MCOL|MDOWN|MDX|MDY|MEMLINES|MEMORY|MENU|MESSAGE|' + r'MESSAGEBOX|MIN|MINUTE|MLINE|MOD|MONTH|MRKBAR|MRKPAD|' + r'MROW|MTON|MWINDOW|NDX|NEWOBJECT|NORMALIZE|NTOM|NUMLOCK|' + r'NVL|OBJNUM|OBJTOCLIENT|OBJVAR|OCCURS|OEMTOANSI|OLDVAL|' + r'ON|ORDER|OS|PAD|PADL|PARAMETERS|PAYMENT|PCOL|PCOUNT|' + r'PEMSTATUS|PI|POPUP|PRIMARY|PRINTSTATUS|PRMBAR|PRMPAD|' + r'PROGRAM|PROMPT|PROPER|PROW|PRTINFO|PUTFILE|PV|QUARTER|' + r'RAISEEVENT|RAND|RAT|RATC|RATLINE|RDLEVEL|READKEY|RECCOUNT|' + r'RECNO|RECSIZE|REFRESH|RELATION|REPLICATE|REQUERY|RGB|' + r'RGBSCHEME|RIGHT|RIGHTC|RLOCK|ROUND|ROW|RTOD|RTRIM|' + r'SAVEPICTURE|SCHEME|SCOLS|SEC|SECONDS|SEEK|SELECT|SET|' + r'SETFLDSTATE|SETRESULTSET|SIGN|SIN|SKPBAR|SKPPAD|SOUNDEX|' + r'SPACE|SQLCANCEL|SQLCOLUMNS|SQLCOMMIT|SQLCONNECT|' + r'SQLDISCONNECT|SQLEXEC|SQLGETPROP|SQLIDLEDISCONNECT|' + r'SQLMORERESULTS|SQLPREPARE|SQLROLLBACK|SQLSETPROP|' + r'SQLSTRINGCONNECT|SQLTABLES|SQRT|SROWS|STR|STRCONV|' + r'STREXTRACT|STRTOFILE|STRTRAN|STUFF|STUFFC|SUBSTR|' + r'SUBSTRC|SYS|SYSMETRIC|TABLEREVERT|TABLEUPDATE|TAG|' + r'TAGCOUNT|TAGNO|TAN|TARGET|TEXTMERGE|TIME|TRANSFORM|' + r'TRIM|TTOC|TTOD|TXNLEVEL|TXTWIDTH|TYPE|UNBINDEVENTS|' + r'UNIQUE|UPDATED|UPPER|USED|VAL|VARREAD|VARTYPE|VERSION|' + r'WBORDER|WCHILD|WCOLS|WDOCKABLE|WEEK|WEXIST|WFONT|WLAST|' + r'WLCOL|WLROW|WMAXIMUM|WMINIMUM|WONTOP|WOUTPUT|WPARENT|' + r'WREAD|WROWS|WTITLE|WVISIBLE|XMLTOCURSOR|XMLUPDATEGRAM|' + r'YEAR)(?=\s*\()', Name.Function), + + (r'_ALIGNMENT|_ASCIICOLS|_ASCIIROWS|_ASSIST|_BEAUTIFY|_BOX|' + r'_BROWSER|_BUILDER|_CALCMEM|_CALCVALUE|_CLIPTEXT|_CONVERTER|' + r'_COVERAGE|_CUROBJ|_DBLCLICK|_DIARYDATE|_DOS|_FOXDOC|_FOXREF|' + r'_GALLERY|_GENGRAPH|_GENHTML|_GENMENU|_GENPD|_GENSCRN|' + r'_GENXTAB|_GETEXPR|_INCLUDE|_INCSEEK|_INDENT|_LMARGIN|_MAC|' + r'_MENUDESIGNER|_MLINE|_PADVANCE|_PAGENO|_PAGETOTAL|_PBPAGE|' + r'_PCOLNO|_PCOPIES|_PDRIVER|_PDSETUP|_PECODE|_PEJECT|_PEPAGE|' + r'_PLENGTH|_PLINENO|_PLOFFSET|_PPITCH|_PQUALITY|_PRETEXT|' + r'_PSCODE|_PSPACING|_PWAIT|_RMARGIN|_REPORTBUILDER|' + r'_REPORTOUTPUT|_REPORTPREVIEW|_SAMPLES|_SCCTEXT|_SCREEN|' + r'_SHELL|_SPELLCHK|_STARTUP|_TABS|_TALLY|_TASKPANE|_TEXT|' + r'_THROTTLE|_TOOLBOX|_TOOLTIPTIMEOUT|_TRANSPORT|_TRIGGERLEVEL|' + r'_UNIX|_VFP|_WINDOWS|_WIZARD|_WRAP', Keyword.Pseudo), + + (r'THISFORMSET|THISFORM|THIS', Name.Builtin), + + (r'Application|CheckBox|Collection|Column|ComboBox|' + r'CommandButton|CommandGroup|Container|Control|CursorAdapter|' + r'Cursor|Custom|DataEnvironment|DataObject|EditBox|' + r'Empty|Exception|Fields|Files|File|FormSet|Form|FoxCode|' + r'Grid|Header|Hyperlink|Image|Label|Line|ListBox|Objects|' + r'OptionButton|OptionGroup|PageFrame|Page|ProjectHook|Projects|' + r'Project|Relation|ReportListener|Separator|Servers|Server|' + r'Session|Shape|Spinner|Tables|TextBox|Timer|ToolBar|' + r'XMLAdapter|XMLField|XMLTable', Name.Class), + + (r'm\.[a-z_]\w*', Name.Variable), + (r'\.(F|T|AND|OR|NOT|NULL)\.|\b(AND|OR|NOT|NULL)\b', Operator.Word), + + (r'\.(ActiveColumn|ActiveControl|ActiveForm|ActivePage|' + r'ActiveProject|ActiveRow|AddLineFeeds|ADOCodePage|Alias|' + r'Alignment|Align|AllowAddNew|AllowAutoColumnFit|' + r'AllowCellSelection|AllowDelete|AllowHeaderSizing|' + r'AllowInsert|AllowModalMessages|AllowOutput|AllowRowSizing|' + r'AllowSimultaneousFetch|AllowTabs|AllowUpdate|' + r'AlwaysOnBottom|AlwaysOnTop|Anchor|Application|' + r'AutoActivate|AutoCenter|AutoCloseTables|AutoComplete|' + r'AutoCompSource|AutoCompTable|AutoHideScrollBar|' + r'AutoIncrement|AutoOpenTables|AutoRelease|AutoSize|' + r'AutoVerbMenu|AutoYield|BackColor|ForeColor|BackStyle|' + r'BaseClass|BatchUpdateCount|BindControls|BorderColor|' + r'BorderStyle|BorderWidth|BoundColumn|BoundTo|Bound|' + r'BreakOnError|BufferModeOverride|BufferMode|' + r'BuildDateTime|ButtonCount|Buttons|Cancel|Caption|' + r'Centered|Century|ChildAlias|ChildOrder|ChildTable|' + r'ClassLibrary|Class|ClipControls|Closable|CLSID|CodePage|' + r'ColorScheme|ColorSource|ColumnCount|ColumnLines|' + r'ColumnOrder|Columns|ColumnWidths|CommandClauses|' + r'Comment|CompareMemo|ConflictCheckCmd|ConflictCheckType|' + r'ContinuousScroll|ControlBox|ControlCount|Controls|' + r'ControlSource|ConversionFunc|Count|CurrentControl|' + r'CurrentDataSession|CurrentPass|CurrentX|CurrentY|' + r'CursorSchema|CursorSource|CursorStatus|Curvature|' + r'Database|DataSessionID|DataSession|DataSourceType|' + r'DataSource|DataType|DateFormat|DateMark|Debug|' + r'DeclareXMLPrefix|DEClassLibrary|DEClass|DefaultFilePath|' + r'Default|DefOLELCID|DeleteCmdDataSourceType|DeleteCmdDataSource|' + r'DeleteCmd|DeleteMark|Description|Desktop|' + r'Details|DisabledBackColor|DisabledForeColor|' + r'DisabledItemBackColor|DisabledItemForeColor|' + r'DisabledPicture|DisableEncode|DisplayCount|' + r'DisplayValue|Dockable|Docked|DockPosition|' + r'DocumentFile|DownPicture|DragIcon|DragMode|DrawMode|' + r'DrawStyle|DrawWidth|DynamicAlignment|DynamicBackColor|' + r'DynamicForeColor|DynamicCurrentControl|DynamicFontBold|' + r'DynamicFontItalic|DynamicFontStrikethru|' + r'DynamicFontUnderline|DynamicFontName|DynamicFontOutline|' + r'DynamicFontShadow|DynamicFontSize|DynamicInputMask|' + r'DynamicLineHeight|EditorOptions|Enabled|' + r'EnableHyperlinks|Encrypted|ErrorNo|Exclude|Exclusive|' + r'FetchAsNeeded|FetchMemoCmdList|FetchMemoDataSourceType|' + r'FetchMemoDataSource|FetchMemo|FetchSize|' + r'FileClassLibrary|FileClass|FillColor|FillStyle|Filter|' + r'FirstElement|FirstNestedTable|Flags|FontBold|FontItalic|' + r'FontStrikethru|FontUnderline|FontCharSet|FontCondense|' + r'FontExtend|FontName|FontOutline|FontShadow|FontSize|' + r'ForceCloseTag|Format|FormCount|FormattedOutput|Forms|' + r'FractionDigits|FRXDataSession|FullName|GDIPlusGraphics|' + r'GridLineColor|GridLines|GridLineWidth|HalfHeightCaption|' + r'HeaderClassLibrary|HeaderClass|HeaderHeight|Height|' + r'HelpContextID|HideSelection|HighlightBackColor|' + r'HighlightForeColor|HighlightStyle|HighlightRowLineWidth|' + r'HighlightRow|Highlight|HomeDir|Hours|HostName|' + r'HScrollSmallChange|hWnd|Icon|IncrementalSearch|Increment|' + r'InitialSelectedAlias|InputMask|InsertCmdDataSourceType|' + r'InsertCmdDataSource|InsertCmdRefreshCmd|' + r'InsertCmdRefreshFieldList|InsertCmdRefreshKeyFieldList|' + r'InsertCmd|Instancing|IntegralHeight|' + r'Interval|IMEMode|IsAttribute|IsBase64|IsBinary|IsNull|' + r'IsDiffGram|IsLoaded|ItemBackColor,|ItemData|ItemIDData|' + r'ItemTips|IXMLDOMElement|KeyboardHighValue|KeyboardLowValue|' + r'Keyfield|KeyFieldList|KeyPreview|KeySort|LanguageOptions|' + r'LeftColumn|Left|LineContents|LineNo|LineSlant|LinkMaster|' + r'ListCount|ListenerType|ListIndex|ListItemID|ListItem|' + r'List|LockColumnsLeft|LockColumns|LockScreen|MacDesktop|' + r'MainFile|MapN19_4ToCurrency|MapBinary|MapVarchar|Margin|' + r'MaxButton|MaxHeight|MaxLeft|MaxLength|MaxRecords|MaxTop|' + r'MaxWidth|MDIForm|MemberClassLibrary|MemberClass|' + r'MemoWindow|Message|MinButton|MinHeight|MinWidth|' + r'MouseIcon|MousePointer|Movable|MoverBars|MultiSelect|' + r'Name|NestedInto|NewIndex|NewItemID|NextSiblingTable|' + r'NoCpTrans|NoDataOnLoad|NoData|NullDisplay|' + r'NumberOfElements|Object|OLEClass|OLEDragMode|' + r'OLEDragPicture|OLEDropEffects|OLEDropHasData|' + r'OLEDropMode|OLEDropTextInsertion|OLELCID|' + r'OLERequestPendingTimeout|OLEServerBusyRaiseError|' + r'OLEServerBusyTimeout|OLETypeAllowed|OneToMany|' + r'OpenViews|OpenWindow|Optimize|OrderDirection|Order|' + r'OutputPageCount|OutputType|PageCount|PageHeight|' + r'PageNo|PageOrder|Pages|PageTotal|PageWidth|' + r'PanelLink|Panel|ParentAlias|ParentClass|ParentTable|' + r'Parent|Partition|PasswordChar|PictureMargin|' + r'PicturePosition|PictureSpacing|PictureSelectionDisplay|' + r'PictureVal|Picture|Prepared|' + r'PolyPoints|PreserveWhiteSpace|PreviewContainer|' + r'PrintJobName|Procedure|PROCESSID|ProgID|ProjectHookClass|' + r'ProjectHookLibrary|ProjectHook|QuietMode|' + r'ReadCycle|ReadLock|ReadMouse|ReadObject|ReadOnly|' + r'ReadSave|ReadTimeout|RecordMark|RecordSourceType|' + r'RecordSource|RefreshAlias|' + r'RefreshCmdDataSourceType|RefreshCmdDataSource|RefreshCmd|' + r'RefreshIgnoreFieldList|RefreshTimeStamp|RelationalExpr|' + r'RelativeColumn|RelativeRow|ReleaseType|Resizable|' + r'RespectCursorCP|RespectNesting|RightToLeft|RotateFlip|' + r'Rotation|RowColChange|RowHeight|RowSourceType|' + r'RowSource|ScaleMode|SCCProvider|SCCStatus|ScrollBars|' + r'Seconds|SelectCmd|SelectedID|' + r'SelectedItemBackColor|SelectedItemForeColor|Selected|' + r'SelectionNamespaces|SelectOnEntry|SelLength|SelStart|' + r'SelText|SendGDIPlusImage|SendUpdates|ServerClassLibrary|' + r'ServerClass|ServerHelpFile|ServerName|' + r'ServerProject|ShowTips|ShowInTaskbar|ShowWindow|' + r'Sizable|SizeBox|SOM|Sorted|Sparse|SpecialEffect|' + r'SpinnerHighValue|SpinnerLowValue|SplitBar|StackLevel|' + r'StartMode|StatusBarText|StatusBar|Stretch|StrictDateEntry|' + r'Style|TabIndex|Tables|TabOrientation|Tabs|TabStop|' + r'TabStretch|TabStyle|Tag|TerminateRead|Text|Themes|' + r'ThreadID|TimestampFieldList|TitleBar|ToolTipText|' + r'TopIndex|TopItemID|Top|TwoPassProcess|TypeLibCLSID|' + r'TypeLibDesc|TypeLibName|Type|Unicode|UpdatableFieldList|' + r'UpdateCmdDataSourceType|UpdateCmdDataSource|' + r'UpdateCmdRefreshCmd|UpdateCmdRefreshFieldList|' + r'UpdateCmdRefreshKeyFieldList|UpdateCmd|' + r'UpdateGramSchemaLocation|UpdateGram|UpdateNameList|UpdateType|' + r'UseCodePage|UseCursorSchema|UseDeDataSource|UseMemoSize|' + r'UserValue|UseTransactions|UTF8Encoded|Value|VersionComments|' + r'VersionCompany|VersionCopyright|VersionDescription|' + r'VersionNumber|VersionProduct|VersionTrademarks|Version|' + r'VFPXMLProgID|ViewPortHeight|ViewPortLeft|' + r'ViewPortTop|ViewPortWidth|VScrollSmallChange|View|Visible|' + r'VisualEffect|WhatsThisButton|WhatsThisHelpID|WhatsThisHelp|' + r'WhereType|Width|WindowList|WindowState|WindowType|WordWrap|' + r'WrapCharInCDATA|WrapInCDATA|WrapMemoInCDATA|XMLAdapter|' + r'XMLConstraints|XMLNameIsXPath|XMLNamespace|XMLName|' + r'XMLPrefix|XMLSchemaLocation|XMLTable|XMLType|' + r'XSDfractionDigits|XSDmaxLength|XSDtotalDigits|' + r'XSDtype|ZoomBox)', Name.Attribute), + + (r'\.(ActivateCell|AddColumn|AddItem|AddListItem|AddObject|' + r'AddProperty|AddTableSchema|AddToSCC|Add|' + r'ApplyDiffgram|Attach|AutoFit|AutoOpen|Box|Build|' + r'CancelReport|ChangesToCursor|CheckIn|CheckOut|Circle|' + r'CleanUp|ClearData|ClearStatus|Clear|CloneObject|CloseTables|' + r'Close|Cls|CursorAttach|CursorDetach|CursorFill|' + r'CursorRefresh|DataToClip|DelayedMemoFetch|DeleteColumn|' + r'Dock|DoMessage|DoScroll|DoStatus|DoVerb|Drag|Draw|Eval|' + r'GetData|GetDockState|GetFormat|GetKey|GetLatestVersion|' + r'GetPageHeight|GetPageWidth|Help|Hide|IncludePageInOutput|' + r'IndexToItemID|ItemIDToIndex|Item|LoadXML|Line|Modify|' + r'MoveItem|Move|Nest|OLEDrag|OnPreviewClose|OutputPage|' + r'Point|Print|PSet|Quit|ReadExpression|ReadMethod|' + r'RecordRefresh|Refresh|ReleaseXML|Release|RemoveFromSCC|' + r'RemoveItem|RemoveListItem|RemoveObject|Remove|' + r'Render|Requery|RequestData|ResetToDefault|Reset|Run|' + r'SaveAsClass|SaveAs|SetAll|SetData|SetFocus|SetFormat|' + r'SetMain|SetVar|SetViewPort|ShowWhatsThis|Show|' + r'SupportsListenerType|TextHeight|TextWidth|ToCursor|' + r'ToXML|UndoCheckOut|Unnest|UpdateStatus|WhatsThisMode|' + r'WriteExpression|WriteMethod|ZOrder)', Name.Function), + + (r'\.(Activate|AdjustObjectSize|AfterBand|AfterBuild|' + r'AfterCloseTables|AfterCursorAttach|AfterCursorClose|' + r'AfterCursorDetach|AfterCursorFill|AfterCursorRefresh|' + r'AfterCursorUpdate|AfterDelete|AfterInsert|' + r'AfterRecordRefresh|AfterUpdate|AfterDock|AfterReport|' + r'AfterRowColChange|BeforeBand|BeforeCursorAttach|' + r'BeforeCursorClose|BeforeCursorDetach|BeforeCursorFill|' + r'BeforeCursorRefresh|BeforeCursorUpdate|BeforeDelete|' + r'BeforeInsert|BeforeDock|BeforeOpenTables|' + r'BeforeRecordRefresh|BeforeReport|BeforeRowColChange|' + r'BeforeUpdate|Click|dbc_Activate|dbc_AfterAddTable|' + r'dbc_AfterAppendProc|dbc_AfterCloseTable|dbc_AfterCopyProc|' + r'dbc_AfterCreateConnection|dbc_AfterCreateOffline|' + r'dbc_AfterCreateTable|dbc_AfterCreateView|dbc_AfterDBGetProp|' + r'dbc_AfterDBSetProp|dbc_AfterDeleteConnection|' + r'dbc_AfterDropOffline|dbc_AfterDropTable|' + r'dbc_AfterModifyConnection|dbc_AfterModifyProc|' + r'dbc_AfterModifyTable|dbc_AfterModifyView|dbc_AfterOpenTable|' + r'dbc_AfterRemoveTable|dbc_AfterRenameConnection|' + r'dbc_AfterRenameTable|dbc_AfterRenameView|' + r'dbc_AfterValidateData|dbc_BeforeAddTable|' + r'dbc_BeforeAppendProc|dbc_BeforeCloseTable|' + r'dbc_BeforeCopyProc|dbc_BeforeCreateConnection|' + r'dbc_BeforeCreateOffline|dbc_BeforeCreateTable|' + r'dbc_BeforeCreateView|dbc_BeforeDBGetProp|' + r'dbc_BeforeDBSetProp|dbc_BeforeDeleteConnection|' + r'dbc_BeforeDropOffline|dbc_BeforeDropTable|' + r'dbc_BeforeModifyConnection|dbc_BeforeModifyProc|' + r'dbc_BeforeModifyTable|dbc_BeforeModifyView|' + r'dbc_BeforeOpenTable|dbc_BeforeRemoveTable|' + r'dbc_BeforeRenameConnection|dbc_BeforeRenameTable|' + r'dbc_BeforeRenameView|dbc_BeforeValidateData|' + r'dbc_CloseData|dbc_Deactivate|dbc_ModifyData|dbc_OpenData|' + r'dbc_PackData|DblClick|Deactivate|Deleted|Destroy|DoCmd|' + r'DownClick|DragDrop|DragOver|DropDown|ErrorMessage|Error|' + r'EvaluateContents|GotFocus|Init|InteractiveChange|KeyPress|' + r'LoadReport|Load|LostFocus|Message|MiddleClick|MouseDown|' + r'MouseEnter|MouseLeave|MouseMove|MouseUp|MouseWheel|Moved|' + r'OLECompleteDrag|OLEDragOver|OLEGiveFeedback|OLESetData|' + r'OLEStartDrag|OnMoveItem|Paint|ProgrammaticChange|' + r'QueryAddFile|QueryModifyFile|QueryNewFile|QueryRemoveFile|' + r'QueryRunFile|QueryUnload|RangeHigh|RangeLow|ReadActivate|' + r'ReadDeactivate|ReadShow|ReadValid|ReadWhen|Resize|' + r'RightClick|SCCInit|SCCDestroy|Scrolled|Timer|UIEnable|' + r'UnDock|UnloadReport|Unload|UpClick|Valid|When)', Name.Function), + + (r'\s+', Text), + # everything else is not colored + (r'.', Text), + ], + 'newline': [ + (r'\*.*?$', Comment.Single, '#pop'), + (r'(ACCEPT|ACTIVATE\s*MENU|ACTIVATE\s*POPUP|ACTIVATE\s*SCREEN|' + r'ACTIVATE\s*WINDOW|APPEND|APPEND\s*FROM|APPEND\s*FROM\s*ARRAY|' + r'APPEND\s*GENERAL|APPEND\s*MEMO|ASSIST|AVERAGE|BLANK|BROWSE|' + r'BUILD\s*APP|BUILD\s*EXE|BUILD\s*PROJECT|CALCULATE|CALL|' + r'CANCEL|CHANGE|CLEAR|CLOSE|CLOSE\s*MEMO|COMPILE|CONTINUE|' + r'COPY\s*FILE|COPY\s*INDEXES|COPY\s*MEMO|COPY\s*STRUCTURE|' + r'COPY\s*STRUCTURE\s*EXTENDED|COPY\s*TAG|COPY\s*TO|' + r'COPY\s*TO\s*ARRAY|COUNT|CREATE|CREATE\s*COLOR\s*SET|' + r'CREATE\s*CURSOR|CREATE\s*FROM|CREATE\s*LABEL|CREATE\s*MENU|' + r'CREATE\s*PROJECT|CREATE\s*QUERY|CREATE\s*REPORT|' + r'CREATE\s*SCREEN|CREATE\s*TABLE|CREATE\s*VIEW|DDE|' + r'DEACTIVATE\s*MENU|DEACTIVATE\s*POPUP|DEACTIVATE\s*WINDOW|' + r'DECLARE|DEFINE\s*BAR|DEFINE\s*BOX|DEFINE\s*MENU|' + r'DEFINE\s*PAD|DEFINE\s*POPUP|DEFINE\s*WINDOW|DELETE|' + r'DELETE\s*FILE|DELETE\s*TAG|DIMENSION|DIRECTORY|DISPLAY|' + r'DISPLAY\s*FILES|DISPLAY\s*MEMORY|DISPLAY\s*STATUS|' + r'DISPLAY\s*STRUCTURE|DO|EDIT|EJECT|EJECT\s*PAGE|ERASE|' + r'EXIT|EXPORT|EXTERNAL|FILER|FIND|FLUSH|FUNCTION|GATHER|' + r'GETEXPR|GO|GOTO|HELP|HIDE\s*MENU|HIDE\s*POPUP|' + r'HIDE\s*WINDOW|IMPORT|INDEX|INPUT|INSERT|JOIN|KEYBOARD|' + r'LABEL|LIST|LOAD|LOCATE|LOOP|MENU|MENU\s*TO|MODIFY\s*COMMAND|' + r'MODIFY\s*FILE|MODIFY\s*GENERAL|MODIFY\s*LABEL|MODIFY\s*MEMO|' + r'MODIFY\s*MENU|MODIFY\s*PROJECT|MODIFY\s*QUERY|' + r'MODIFY\s*REPORT|MODIFY\s*SCREEN|MODIFY\s*STRUCTURE|' + r'MODIFY\s*WINDOW|MOVE\s*POPUP|MOVE\s*WINDOW|NOTE|' + r'ON\s*APLABOUT|ON\s*BAR|ON\s*ERROR|ON\s*ESCAPE|' + r'ON\s*EXIT\s*BAR|ON\s*EXIT\s*MENU|ON\s*EXIT\s*PAD|' + r'ON\s*EXIT\s*POPUP|ON\s*KEY|ON\s*KEY\s*=|ON\s*KEY\s*LABEL|' + r'ON\s*MACHELP|ON\s*PAD|ON\s*PAGE|ON\s*READERROR|' + r'ON\s*SELECTION\s*BAR|ON\s*SELECTION\s*MENU|' + r'ON\s*SELECTION\s*PAD|ON\s*SELECTION\s*POPUP|ON\s*SHUTDOWN|' + r'PACK|PARAMETERS|PLAY\s*MACRO|POP\s*KEY|POP\s*MENU|' + r'POP\s*POPUP|PRIVATE|PROCEDURE|PUBLIC|PUSH\s*KEY|' + r'PUSH\s*MENU|PUSH\s*POPUP|QUIT|READ|READ\s*MENU|RECALL|' + r'REINDEX|RELEASE|RELEASE\s*MODULE|RENAME|REPLACE|' + r'REPLACE\s*FROM\s*ARRAY|REPORT|RESTORE\s*FROM|' + r'RESTORE\s*MACROS|RESTORE\s*SCREEN|RESTORE\s*WINDOW|' + r'RESUME|RETRY|RETURN|RUN|RUN\s*\/N"|RUNSCRIPT|' + r'SAVE\s*MACROS|SAVE\s*SCREEN|SAVE\s*TO|SAVE\s*WINDOWS|' + r'SCATTER|SCROLL|SEEK|SELECT|SET|SET\s*ALTERNATE|' + r'SET\s*ANSI|SET\s*APLABOUT|SET\s*AUTOSAVE|SET\s*BELL|' + r'SET\s*BLINK|SET\s*BLOCKSIZE|SET\s*BORDER|SET\s*BRSTATUS|' + r'SET\s*CARRY|SET\s*CENTURY|SET\s*CLEAR|SET\s*CLOCK|' + r'SET\s*COLLATE|SET\s*COLOR\s*OF|SET\s*COLOR\s*OF\s*SCHEME|' + r'SET\s*COLOR\s*SET|SET\s*COLOR\s*TO|SET\s*COMPATIBLE|' + r'SET\s*CONFIRM|SET\s*CONSOLE|SET\s*CURRENCY|SET\s*CURSOR|' + r'SET\s*DATE|SET\s*DEBUG|SET\s*DECIMALS|SET\s*DEFAULT|' + r'SET\s*DELETED|SET\s*DELIMITERS|SET\s*DEVELOPMENT|' + r'SET\s*DEVICE|SET\s*DISPLAY|SET\s*DOHISTORY|SET\s*ECHO|' + r'SET\s*ESCAPE|SET\s*EXACT|SET\s*EXCLUSIVE|SET\s*FIELDS|' + r'SET\s*FILTER|SET\s*FIXED|SET\s*FORMAT|SET\s*FULLPATH|' + r'SET\s*FUNCTION|SET\s*HEADINGS|SET\s*HELP|SET\s*HELPFILTER|' + r'SET\s*HOURS|SET\s*INDEX|SET\s*INTENSITY|SET\s*KEY|' + r'SET\s*KEYCOMP|SET\s*LIBRARY|SET\s*LOCK|SET\s*LOGERRORS|' + r'SET\s*MACDESKTOP|SET\s*MACHELP|SET\s*MACKEY|SET\s*MARGIN|' + r'SET\s*MARK\s*OF|SET\s*MARK\s*TO|SET\s*MEMOWIDTH|' + r'SET\s*MESSAGE|SET\s*MOUSE|SET\s*MULTILOCKS|SET\s*NEAR|' + r'SET\s*NOCPTRANS|SET\s*NOTIFY|SET\s*ODOMETER|SET\s*OPTIMIZE|' + r'SET\s*ORDER|SET\s*PALETTE|SET\s*PATH|SET\s*PDSETUP|' + r'SET\s*POINT|SET\s*PRINTER|SET\s*PROCEDURE|SET\s*READBORDER|' + r'SET\s*REFRESH|SET\s*RELATION|SET\s*RELATION\s*OFF|' + r'SET\s*REPROCESS|SET\s*RESOURCE|SET\s*SAFETY|SET\s*SCOREBOARD|' + r'SET\s*SEPARATOR|SET\s*SHADOWS|SET\s*SKIP|SET\s*SKIP\s*OF|' + r'SET\s*SPACE|SET\s*STATUS|SET\s*STATUS\s*BAR|SET\s*STEP|' + r'SET\s*STICKY|SET\s*SYSMENU|SET\s*TALK|SET\s*TEXTMERGE|' + r'SET\s*TEXTMERGE\s*DELIMITERS|SET\s*TOPIC|SET\s*TRBETWEEN|' + r'SET\s*TYPEAHEAD|SET\s*UDFPARMS|SET\s*UNIQUE|SET\s*VIEW|' + r'SET\s*VOLUME|SET\s*WINDOW\s*OF\s*MEMO|SET\s*XCMDFILE|' + r'SHOW\s*GET|SHOW\s*GETS|SHOW\s*MENU|SHOW\s*OBJECT|' + r'SHOW\s*POPUP|SHOW\s*WINDOW|SIZE\s*POPUP|SKIP|SORT|' + r'STORE|SUM|SUSPEND|TOTAL|TYPE|UNLOCK|UPDATE|USE|WAIT|' + r'ZAP|ZOOM\s*WINDOW|DO\s*CASE|CASE|OTHERWISE|ENDCASE|' + r'DO\s*WHILE|ENDDO|FOR|ENDFOR|NEXT|IF|ELSE|ENDIF|PRINTJOB|' + r'ENDPRINTJOB|SCAN|ENDSCAN|TEXT|ENDTEXT|=)', + Keyword.Reserved, '#pop'), + (r'#\s*(IF|ELIF|ELSE|ENDIF|DEFINE|IFDEF|IFNDEF|INCLUDE)', + Comment.Preproc, '#pop'), + (r'(m\.)?[a-z_]\w*', Name.Variable, '#pop'), + (r'.', Text, '#pop'), + ], + } diff --git a/pygments/lexers/functional.py b/pygments/lexers/functional.py index 39cf77f1..a082811b 100644 --- a/pygments/lexers/functional.py +++ b/pygments/lexers/functional.py @@ -5,7 +5,7 @@ Lexers for functional languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -23,7 +23,8 @@ __all__ = ['RacketLexer', 'SchemeLexer', 'CommonLispLexer', 'HaskellLexer', class RacketLexer(RegexLexer): """ - Lexer for Racket source code. + Lexer for `Racket <http://racket-lang.org/>`_ source code (formerly known as + PLT Scheme). *New in Pygments 1.6.* """ @@ -1664,7 +1665,7 @@ class OpaLexer(RegexLexer): keywords = [ 'and', 'as', 'begin', 'css', 'database', 'db', 'do', 'else', 'end', 'external', 'forall', 'if', 'import', 'match', 'package', 'parser', - 'rec', 'server', 'then', 'type', 'val', 'with', 'xml_parser' + 'rec', 'server', 'then', 'type', 'val', 'with', 'xml_parser', ] # matches both stuff and `stuff` @@ -1901,7 +1902,7 @@ class OpaLexer(RegexLexer): (r'[/*]', Comment), ], - # the coy pasting between string and single-string + # the copy pasting between string and single-string # is kinda sad. Is there a way to avoid that?? 'string': [ (r'[^\\"{]+', String.Double), @@ -1948,6 +1949,7 @@ class OpaLexer(RegexLexer): (r'"', String.Single, ('#pop', 'string')), (r'#'+ident_re, String.Single, '#pop'), (r'#(?={)', String.Single, ('#pop', 'root')), + (r'[^"\'{`=<>]+', String.Single, '#pop'), (r'{', Operator, ('#pop', 'root')), # this is a tail call! ], @@ -1957,7 +1959,7 @@ class OpaLexer(RegexLexer): (r'</', String.Single, ('#pop', 'html-end-tag')), (r'<', String.Single, 'html-open-tag'), (r'{', Operator, 'root'), - (r'.|\s+', String.Single), + (r'[^<{]+', String.Single), ], 'html-comment': [ @@ -2436,7 +2438,7 @@ class KokaLexer(RegexLexer): # builtin names and special names builtin = [ 'for', 'while', 'repeat', - 'foreach', 'foreach-indexed', + 'foreach', 'foreach-indexed', 'error', 'catch', 'finally', 'cs', 'js', 'file', 'ref', 'assigned', ] diff --git a/pygments/lexers/hdl.py b/pygments/lexers/hdl.py index 0db35e74..57ffc349 100644 --- a/pygments/lexers/hdl.py +++ b/pygments/lexers/hdl.py @@ -5,7 +5,7 @@ Lexers for hardware descriptor languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -25,7 +25,7 @@ class VerilogLexer(RegexLexer): *New in Pygments 1.4.* """ name = 'verilog' - aliases = ['v'] + aliases = ['verilog', 'v'] filenames = ['*.v'] mimetypes = ['text/x-verilog'] @@ -137,7 +137,7 @@ class SystemVerilogLexer(RegexLexer): *New in Pygments 1.5.* """ name = 'systemverilog' - aliases = ['sv'] + aliases = ['systemverilog', 'sv'] filenames = ['*.sv', '*.svh'] mimetypes = ['text/x-systemverilog'] diff --git a/pygments/lexers/jvm.py b/pygments/lexers/jvm.py index 161a3382..717621e9 100644 --- a/pygments/lexers/jvm.py +++ b/pygments/lexers/jvm.py @@ -5,7 +5,7 @@ Pygments lexers for JVM languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -60,13 +60,13 @@ class JavaLexer(RegexLexer): (r'(class|interface)(\s+)', bygroups(Keyword.Declaration, Text), 'class'), (r'(import)(\s+)', bygroups(Keyword.Namespace, Text), 'import'), (r'"(\\\\|\\"|[^"])*"', String), - (r"'\\.'|'[^\\]'|'\\u[0-9a-f]{4}'", String.Char), + (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Char), (r'(\.)([a-zA-Z_][a-zA-Z0-9_]*)', bygroups(Operator, Name.Attribute)), (r'[a-zA-Z_][a-zA-Z0-9_]*:', Name.Label), (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), (r'[~\^\*!%&\[\]\(\)\{\}<>\|+=:;,./?-]', Operator), (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+L?', Number.Integer), (r'\n', Text) ], @@ -131,11 +131,117 @@ class ScalaLexer(RegexLexer): flags = re.MULTILINE | re.DOTALL # don't use raw unicode strings! - op = u'[-~\\^\\*!%&\\\\<>\\|+=:/?@\u00a6-\u00a7\u00a9\u00ac\u00ae\u00b0-\u00b1\u00b6\u00d7\u00f7\u03f6\u0482\u0606-\u0608\u060e-\u060f\u06e9\u06fd-\u06fe\u07f6\u09fa\u0b70\u0bf3-\u0bf8\u0bfa\u0c7f\u0cf1-\u0cf2\u0d79\u0f01-\u0f03\u0f13-\u0f17\u0f1a-\u0f1f\u0f34\u0f36\u0f38\u0fbe-\u0fc5\u0fc7-\u0fcf\u109e-\u109f\u1360\u1390-\u1399\u1940\u19e0-\u19ff\u1b61-\u1b6a\u1b74-\u1b7c\u2044\u2052\u207a-\u207c\u208a-\u208c\u2100-\u2101\u2103-\u2106\u2108-\u2109\u2114\u2116-\u2118\u211e-\u2123\u2125\u2127\u2129\u212e\u213a-\u213b\u2140-\u2144\u214a-\u214d\u214f\u2190-\u2328\u232b-\u244a\u249c-\u24e9\u2500-\u2767\u2794-\u27c4\u27c7-\u27e5\u27f0-\u2982\u2999-\u29d7\u29dc-\u29fb\u29fe-\u2b54\u2ce5-\u2cea\u2e80-\u2ffb\u3004\u3012-\u3013\u3020\u3036-\u3037\u303e-\u303f\u3190-\u3191\u3196-\u319f\u31c0-\u31e3\u3200-\u321e\u322a-\u3250\u3260-\u327f\u328a-\u32b0\u32c0-\u33ff\u4dc0-\u4dff\ua490-\ua4c6\ua828-\ua82b\ufb29\ufdfd\ufe62\ufe64-\ufe66\uff0b\uff1c-\uff1e\uff5c\uff5e\uffe2\uffe4\uffe8-\uffee\ufffc-\ufffd]+' - - letter = u'[a-zA-Z\\$_\u00aa\u00b5\u00ba\u00c0-\u00d6\u00d8-\u00f6\u00f8-\u02af\u0370-\u0373\u0376-\u0377\u037b-\u037d\u0386\u0388-\u03f5\u03f7-\u0481\u048a-\u0556\u0561-\u0587\u05d0-\u05f2\u0621-\u063f\u0641-\u064a\u066e-\u066f\u0671-\u06d3\u06d5\u06ee-\u06ef\u06fa-\u06fc\u06ff\u0710\u0712-\u072f\u074d-\u07a5\u07b1\u07ca-\u07ea\u0904-\u0939\u093d\u0950\u0958-\u0961\u0972-\u097f\u0985-\u09b9\u09bd\u09ce\u09dc-\u09e1\u09f0-\u09f1\u0a05-\u0a39\u0a59-\u0a5e\u0a72-\u0a74\u0a85-\u0ab9\u0abd\u0ad0-\u0ae1\u0b05-\u0b39\u0b3d\u0b5c-\u0b61\u0b71\u0b83-\u0bb9\u0bd0\u0c05-\u0c3d\u0c58-\u0c61\u0c85-\u0cb9\u0cbd\u0cde-\u0ce1\u0d05-\u0d3d\u0d60-\u0d61\u0d7a-\u0d7f\u0d85-\u0dc6\u0e01-\u0e30\u0e32-\u0e33\u0e40-\u0e45\u0e81-\u0eb0\u0eb2-\u0eb3\u0ebd-\u0ec4\u0edc-\u0f00\u0f40-\u0f6c\u0f88-\u0f8b\u1000-\u102a\u103f\u1050-\u1055\u105a-\u105d\u1061\u1065-\u1066\u106e-\u1070\u1075-\u1081\u108e\u10a0-\u10fa\u1100-\u135a\u1380-\u138f\u13a0-\u166c\u166f-\u1676\u1681-\u169a\u16a0-\u16ea\u16ee-\u1711\u1720-\u1731\u1740-\u1751\u1760-\u1770\u1780-\u17b3\u17dc\u1820-\u1842\u1844-\u18a8\u18aa-\u191c\u1950-\u19a9\u19c1-\u19c7\u1a00-\u1a16\u1b05-\u1b33\u1b45-\u1b4b\u1b83-\u1ba0\u1bae-\u1baf\u1c00-\u1c23\u1c4d-\u1c4f\u1c5a-\u1c77\u1d00-\u1d2b\u1d62-\u1d77\u1d79-\u1d9a\u1e00-\u1fbc\u1fbe\u1fc2-\u1fcc\u1fd0-\u1fdb\u1fe0-\u1fec\u1ff2-\u1ffc\u2071\u207f\u2102\u2107\u210a-\u2113\u2115\u2119-\u211d\u2124\u2126\u2128\u212a-\u212d\u212f-\u2139\u213c-\u213f\u2145-\u2149\u214e\u2160-\u2188\u2c00-\u2c7c\u2c80-\u2ce4\u2d00-\u2d65\u2d80-\u2dde\u3006-\u3007\u3021-\u3029\u3038-\u303a\u303c\u3041-\u3096\u309f\u30a1-\u30fa\u30ff-\u318e\u31a0-\u31b7\u31f0-\u31ff\u3400-\u4db5\u4e00-\ua014\ua016-\ua48c\ua500-\ua60b\ua610-\ua61f\ua62a-\ua66e\ua680-\ua697\ua722-\ua76f\ua771-\ua787\ua78b-\ua801\ua803-\ua805\ua807-\ua80a\ua80c-\ua822\ua840-\ua873\ua882-\ua8b3\ua90a-\ua925\ua930-\ua946\uaa00-\uaa28\uaa40-\uaa42\uaa44-\uaa4b\uac00-\ud7a3\uf900-\ufb1d\ufb1f-\ufb28\ufb2a-\ufd3d\ufd50-\ufdfb\ufe70-\ufefc\uff21-\uff3a\uff41-\uff5a\uff66-\uff6f\uff71-\uff9d\uffa0-\uffdc]' - - upper = u'[A-Z\\$_\u00c0-\u00d6\u00d8-\u00de\u0100\u0102\u0104\u0106\u0108\u010a\u010c\u010e\u0110\u0112\u0114\u0116\u0118\u011a\u011c\u011e\u0120\u0122\u0124\u0126\u0128\u012a\u012c\u012e\u0130\u0132\u0134\u0136\u0139\u013b\u013d\u013f\u0141\u0143\u0145\u0147\u014a\u014c\u014e\u0150\u0152\u0154\u0156\u0158\u015a\u015c\u015e\u0160\u0162\u0164\u0166\u0168\u016a\u016c\u016e\u0170\u0172\u0174\u0176\u0178-\u0179\u017b\u017d\u0181-\u0182\u0184\u0186-\u0187\u0189-\u018b\u018e-\u0191\u0193-\u0194\u0196-\u0198\u019c-\u019d\u019f-\u01a0\u01a2\u01a4\u01a6-\u01a7\u01a9\u01ac\u01ae-\u01af\u01b1-\u01b3\u01b5\u01b7-\u01b8\u01bc\u01c4\u01c7\u01ca\u01cd\u01cf\u01d1\u01d3\u01d5\u01d7\u01d9\u01db\u01de\u01e0\u01e2\u01e4\u01e6\u01e8\u01ea\u01ec\u01ee\u01f1\u01f4\u01f6-\u01f8\u01fa\u01fc\u01fe\u0200\u0202\u0204\u0206\u0208\u020a\u020c\u020e\u0210\u0212\u0214\u0216\u0218\u021a\u021c\u021e\u0220\u0222\u0224\u0226\u0228\u022a\u022c\u022e\u0230\u0232\u023a-\u023b\u023d-\u023e\u0241\u0243-\u0246\u0248\u024a\u024c\u024e\u0370\u0372\u0376\u0386\u0388-\u038f\u0391-\u03ab\u03cf\u03d2-\u03d4\u03d8\u03da\u03dc\u03de\u03e0\u03e2\u03e4\u03e6\u03e8\u03ea\u03ec\u03ee\u03f4\u03f7\u03f9-\u03fa\u03fd-\u042f\u0460\u0462\u0464\u0466\u0468\u046a\u046c\u046e\u0470\u0472\u0474\u0476\u0478\u047a\u047c\u047e\u0480\u048a\u048c\u048e\u0490\u0492\u0494\u0496\u0498\u049a\u049c\u049e\u04a0\u04a2\u04a4\u04a6\u04a8\u04aa\u04ac\u04ae\u04b0\u04b2\u04b4\u04b6\u04b8\u04ba\u04bc\u04be\u04c0-\u04c1\u04c3\u04c5\u04c7\u04c9\u04cb\u04cd\u04d0\u04d2\u04d4\u04d6\u04d8\u04da\u04dc\u04de\u04e0\u04e2\u04e4\u04e6\u04e8\u04ea\u04ec\u04ee\u04f0\u04f2\u04f4\u04f6\u04f8\u04fa\u04fc\u04fe\u0500\u0502\u0504\u0506\u0508\u050a\u050c\u050e\u0510\u0512\u0514\u0516\u0518\u051a\u051c\u051e\u0520\u0522\u0531-\u0556\u10a0-\u10c5\u1e00\u1e02\u1e04\u1e06\u1e08\u1e0a\u1e0c\u1e0e\u1e10\u1e12\u1e14\u1e16\u1e18\u1e1a\u1e1c\u1e1e\u1e20\u1e22\u1e24\u1e26\u1e28\u1e2a\u1e2c\u1e2e\u1e30\u1e32\u1e34\u1e36\u1e38\u1e3a\u1e3c\u1e3e\u1e40\u1e42\u1e44\u1e46\u1e48\u1e4a\u1e4c\u1e4e\u1e50\u1e52\u1e54\u1e56\u1e58\u1e5a\u1e5c\u1e5e\u1e60\u1e62\u1e64\u1e66\u1e68\u1e6a\u1e6c\u1e6e\u1e70\u1e72\u1e74\u1e76\u1e78\u1e7a\u1e7c\u1e7e\u1e80\u1e82\u1e84\u1e86\u1e88\u1e8a\u1e8c\u1e8e\u1e90\u1e92\u1e94\u1e9e\u1ea0\u1ea2\u1ea4\u1ea6\u1ea8\u1eaa\u1eac\u1eae\u1eb0\u1eb2\u1eb4\u1eb6\u1eb8\u1eba\u1ebc\u1ebe\u1ec0\u1ec2\u1ec4\u1ec6\u1ec8\u1eca\u1ecc\u1ece\u1ed0\u1ed2\u1ed4\u1ed6\u1ed8\u1eda\u1edc\u1ede\u1ee0\u1ee2\u1ee4\u1ee6\u1ee8\u1eea\u1eec\u1eee\u1ef0\u1ef2\u1ef4\u1ef6\u1ef8\u1efa\u1efc\u1efe\u1f08-\u1f0f\u1f18-\u1f1d\u1f28-\u1f2f\u1f38-\u1f3f\u1f48-\u1f4d\u1f59-\u1f5f\u1f68-\u1f6f\u1fb8-\u1fbb\u1fc8-\u1fcb\u1fd8-\u1fdb\u1fe8-\u1fec\u1ff8-\u1ffb\u2102\u2107\u210b-\u210d\u2110-\u2112\u2115\u2119-\u211d\u2124\u2126\u2128\u212a-\u212d\u2130-\u2133\u213e-\u213f\u2145\u2183\u2c00-\u2c2e\u2c60\u2c62-\u2c64\u2c67\u2c69\u2c6b\u2c6d-\u2c6f\u2c72\u2c75\u2c80\u2c82\u2c84\u2c86\u2c88\u2c8a\u2c8c\u2c8e\u2c90\u2c92\u2c94\u2c96\u2c98\u2c9a\u2c9c\u2c9e\u2ca0\u2ca2\u2ca4\u2ca6\u2ca8\u2caa\u2cac\u2cae\u2cb0\u2cb2\u2cb4\u2cb6\u2cb8\u2cba\u2cbc\u2cbe\u2cc0\u2cc2\u2cc4\u2cc6\u2cc8\u2cca\u2ccc\u2cce\u2cd0\u2cd2\u2cd4\u2cd6\u2cd8\u2cda\u2cdc\u2cde\u2ce0\u2ce2\ua640\ua642\ua644\ua646\ua648\ua64a\ua64c\ua64e\ua650\ua652\ua654\ua656\ua658\ua65a\ua65c\ua65e\ua662\ua664\ua666\ua668\ua66a\ua66c\ua680\ua682\ua684\ua686\ua688\ua68a\ua68c\ua68e\ua690\ua692\ua694\ua696\ua722\ua724\ua726\ua728\ua72a\ua72c\ua72e\ua732\ua734\ua736\ua738\ua73a\ua73c\ua73e\ua740\ua742\ua744\ua746\ua748\ua74a\ua74c\ua74e\ua750\ua752\ua754\ua756\ua758\ua75a\ua75c\ua75e\ua760\ua762\ua764\ua766\ua768\ua76a\ua76c\ua76e\ua779\ua77b\ua77d-\ua77e\ua780\ua782\ua784\ua786\ua78b\uff21-\uff3a]' + op = (u'[-~\\^\\*!%&\\\\<>\\|+=:/?@\u00a6-\u00a7\u00a9\u00ac\u00ae\u00b0-\u00b1' + u'\u00b6\u00d7\u00f7\u03f6\u0482\u0606-\u0608\u060e-\u060f\u06e9' + u'\u06fd-\u06fe\u07f6\u09fa\u0b70\u0bf3-\u0bf8\u0bfa\u0c7f\u0cf1-\u0cf2' + u'\u0d79\u0f01-\u0f03\u0f13-\u0f17\u0f1a-\u0f1f\u0f34\u0f36\u0f38' + u'\u0fbe-\u0fc5\u0fc7-\u0fcf\u109e-\u109f\u1360\u1390-\u1399\u1940' + u'\u19e0-\u19ff\u1b61-\u1b6a\u1b74-\u1b7c\u2044\u2052\u207a-\u207c' + u'\u208a-\u208c\u2100-\u2101\u2103-\u2106\u2108-\u2109\u2114\u2116-\u2118' + u'\u211e-\u2123\u2125\u2127\u2129\u212e\u213a-\u213b\u2140-\u2144' + u'\u214a-\u214d\u214f\u2190-\u2328\u232b-\u244a\u249c-\u24e9\u2500-\u2767' + u'\u2794-\u27c4\u27c7-\u27e5\u27f0-\u2982\u2999-\u29d7\u29dc-\u29fb' + u'\u29fe-\u2b54\u2ce5-\u2cea\u2e80-\u2ffb\u3004\u3012-\u3013\u3020' + u'\u3036-\u3037\u303e-\u303f\u3190-\u3191\u3196-\u319f\u31c0-\u31e3' + u'\u3200-\u321e\u322a-\u3250\u3260-\u327f\u328a-\u32b0\u32c0-\u33ff' + u'\u4dc0-\u4dff\ua490-\ua4c6\ua828-\ua82b\ufb29\ufdfd\ufe62\ufe64-\ufe66' + u'\uff0b\uff1c-\uff1e\uff5c\uff5e\uffe2\uffe4\uffe8-\uffee\ufffc-\ufffd]+') + + letter = (u'[a-zA-Z\\$_\u00aa\u00b5\u00ba\u00c0-\u00d6\u00d8-\u00f6' + u'\u00f8-\u02af\u0370-\u0373\u0376-\u0377\u037b-\u037d\u0386' + u'\u0388-\u03f5\u03f7-\u0481\u048a-\u0556\u0561-\u0587\u05d0-\u05f2' + u'\u0621-\u063f\u0641-\u064a\u066e-\u066f\u0671-\u06d3\u06d5' + u'\u06ee-\u06ef\u06fa-\u06fc\u06ff\u0710\u0712-\u072f\u074d-\u07a5' + u'\u07b1\u07ca-\u07ea\u0904-\u0939\u093d\u0950\u0958-\u0961' + u'\u0972-\u097f\u0985-\u09b9\u09bd\u09ce\u09dc-\u09e1\u09f0-\u09f1' + u'\u0a05-\u0a39\u0a59-\u0a5e\u0a72-\u0a74\u0a85-\u0ab9\u0abd' + u'\u0ad0-\u0ae1\u0b05-\u0b39\u0b3d\u0b5c-\u0b61\u0b71\u0b83-\u0bb9' + u'\u0bd0\u0c05-\u0c3d\u0c58-\u0c61\u0c85-\u0cb9\u0cbd\u0cde-\u0ce1' + u'\u0d05-\u0d3d\u0d60-\u0d61\u0d7a-\u0d7f\u0d85-\u0dc6\u0e01-\u0e30' + u'\u0e32-\u0e33\u0e40-\u0e45\u0e81-\u0eb0\u0eb2-\u0eb3\u0ebd-\u0ec4' + u'\u0edc-\u0f00\u0f40-\u0f6c\u0f88-\u0f8b\u1000-\u102a\u103f' + u'\u1050-\u1055\u105a-\u105d\u1061\u1065-\u1066\u106e-\u1070' + u'\u1075-\u1081\u108e\u10a0-\u10fa\u1100-\u135a\u1380-\u138f' + u'\u13a0-\u166c\u166f-\u1676\u1681-\u169a\u16a0-\u16ea\u16ee-\u1711' + u'\u1720-\u1731\u1740-\u1751\u1760-\u1770\u1780-\u17b3\u17dc' + u'\u1820-\u1842\u1844-\u18a8\u18aa-\u191c\u1950-\u19a9\u19c1-\u19c7' + u'\u1a00-\u1a16\u1b05-\u1b33\u1b45-\u1b4b\u1b83-\u1ba0\u1bae-\u1baf' + u'\u1c00-\u1c23\u1c4d-\u1c4f\u1c5a-\u1c77\u1d00-\u1d2b\u1d62-\u1d77' + u'\u1d79-\u1d9a\u1e00-\u1fbc\u1fbe\u1fc2-\u1fcc\u1fd0-\u1fdb' + u'\u1fe0-\u1fec\u1ff2-\u1ffc\u2071\u207f\u2102\u2107\u210a-\u2113' + u'\u2115\u2119-\u211d\u2124\u2126\u2128\u212a-\u212d\u212f-\u2139' + u'\u213c-\u213f\u2145-\u2149\u214e\u2160-\u2188\u2c00-\u2c7c' + u'\u2c80-\u2ce4\u2d00-\u2d65\u2d80-\u2dde\u3006-\u3007\u3021-\u3029' + u'\u3038-\u303a\u303c\u3041-\u3096\u309f\u30a1-\u30fa\u30ff-\u318e' + u'\u31a0-\u31b7\u31f0-\u31ff\u3400-\u4db5\u4e00-\ua014\ua016-\ua48c' + u'\ua500-\ua60b\ua610-\ua61f\ua62a-\ua66e\ua680-\ua697\ua722-\ua76f' + u'\ua771-\ua787\ua78b-\ua801\ua803-\ua805\ua807-\ua80a\ua80c-\ua822' + u'\ua840-\ua873\ua882-\ua8b3\ua90a-\ua925\ua930-\ua946\uaa00-\uaa28' + u'\uaa40-\uaa42\uaa44-\uaa4b\uac00-\ud7a3\uf900-\ufb1d\ufb1f-\ufb28' + u'\ufb2a-\ufd3d\ufd50-\ufdfb\ufe70-\ufefc\uff21-\uff3a\uff41-\uff5a' + u'\uff66-\uff6f\uff71-\uff9d\uffa0-\uffdc]') + + upper = (u'[A-Z\\$_\u00c0-\u00d6\u00d8-\u00de\u0100\u0102\u0104\u0106\u0108' + u'\u010a\u010c\u010e\u0110\u0112\u0114\u0116\u0118\u011a\u011c' + u'\u011e\u0120\u0122\u0124\u0126\u0128\u012a\u012c\u012e\u0130' + u'\u0132\u0134\u0136\u0139\u013b\u013d\u013f\u0141\u0143\u0145' + u'\u0147\u014a\u014c\u014e\u0150\u0152\u0154\u0156\u0158\u015a' + u'\u015c\u015e\u0160\u0162\u0164\u0166\u0168\u016a\u016c\u016e' + u'\u0170\u0172\u0174\u0176\u0178-\u0179\u017b\u017d\u0181-\u0182' + u'\u0184\u0186-\u0187\u0189-\u018b\u018e-\u0191\u0193-\u0194' + u'\u0196-\u0198\u019c-\u019d\u019f-\u01a0\u01a2\u01a4\u01a6-\u01a7' + u'\u01a9\u01ac\u01ae-\u01af\u01b1-\u01b3\u01b5\u01b7-\u01b8\u01bc' + u'\u01c4\u01c7\u01ca\u01cd\u01cf\u01d1\u01d3\u01d5\u01d7\u01d9' + u'\u01db\u01de\u01e0\u01e2\u01e4\u01e6\u01e8\u01ea\u01ec\u01ee' + u'\u01f1\u01f4\u01f6-\u01f8\u01fa\u01fc\u01fe\u0200\u0202\u0204' + u'\u0206\u0208\u020a\u020c\u020e\u0210\u0212\u0214\u0216\u0218' + u'\u021a\u021c\u021e\u0220\u0222\u0224\u0226\u0228\u022a\u022c' + u'\u022e\u0230\u0232\u023a-\u023b\u023d-\u023e\u0241\u0243-\u0246' + u'\u0248\u024a\u024c\u024e\u0370\u0372\u0376\u0386\u0388-\u038f' + u'\u0391-\u03ab\u03cf\u03d2-\u03d4\u03d8\u03da\u03dc\u03de\u03e0' + u'\u03e2\u03e4\u03e6\u03e8\u03ea\u03ec\u03ee\u03f4\u03f7' + u'\u03f9-\u03fa\u03fd-\u042f\u0460\u0462\u0464\u0466\u0468\u046a' + u'\u046c\u046e\u0470\u0472\u0474\u0476\u0478\u047a\u047c\u047e' + u'\u0480\u048a\u048c\u048e\u0490\u0492\u0494\u0496\u0498\u049a' + u'\u049c\u049e\u04a0\u04a2\u04a4\u04a6\u04a8\u04aa\u04ac\u04ae' + u'\u04b0\u04b2\u04b4\u04b6\u04b8\u04ba\u04bc\u04be\u04c0-\u04c1' + u'\u04c3\u04c5\u04c7\u04c9\u04cb\u04cd\u04d0\u04d2\u04d4\u04d6' + u'\u04d8\u04da\u04dc\u04de\u04e0\u04e2\u04e4\u04e6\u04e8\u04ea' + u'\u04ec\u04ee\u04f0\u04f2\u04f4\u04f6\u04f8\u04fa\u04fc\u04fe' + u'\u0500\u0502\u0504\u0506\u0508\u050a\u050c\u050e\u0510\u0512' + u'\u0514\u0516\u0518\u051a\u051c\u051e\u0520\u0522\u0531-\u0556' + u'\u10a0-\u10c5\u1e00\u1e02\u1e04\u1e06\u1e08\u1e0a\u1e0c\u1e0e' + u'\u1e10\u1e12\u1e14\u1e16\u1e18\u1e1a\u1e1c\u1e1e\u1e20\u1e22' + u'\u1e24\u1e26\u1e28\u1e2a\u1e2c\u1e2e\u1e30\u1e32\u1e34\u1e36' + u'\u1e38\u1e3a\u1e3c\u1e3e\u1e40\u1e42\u1e44\u1e46\u1e48\u1e4a' + u'\u1e4c\u1e4e\u1e50\u1e52\u1e54\u1e56\u1e58\u1e5a\u1e5c\u1e5e' + u'\u1e60\u1e62\u1e64\u1e66\u1e68\u1e6a\u1e6c\u1e6e\u1e70\u1e72' + u'\u1e74\u1e76\u1e78\u1e7a\u1e7c\u1e7e\u1e80\u1e82\u1e84\u1e86' + u'\u1e88\u1e8a\u1e8c\u1e8e\u1e90\u1e92\u1e94\u1e9e\u1ea0\u1ea2' + u'\u1ea4\u1ea6\u1ea8\u1eaa\u1eac\u1eae\u1eb0\u1eb2\u1eb4\u1eb6' + u'\u1eb8\u1eba\u1ebc\u1ebe\u1ec0\u1ec2\u1ec4\u1ec6\u1ec8\u1eca' + u'\u1ecc\u1ece\u1ed0\u1ed2\u1ed4\u1ed6\u1ed8\u1eda\u1edc\u1ede' + u'\u1ee0\u1ee2\u1ee4\u1ee6\u1ee8\u1eea\u1eec\u1eee\u1ef0\u1ef2' + u'\u1ef4\u1ef6\u1ef8\u1efa\u1efc\u1efe\u1f08-\u1f0f\u1f18-\u1f1d' + u'\u1f28-\u1f2f\u1f38-\u1f3f\u1f48-\u1f4d\u1f59-\u1f5f' + u'\u1f68-\u1f6f\u1fb8-\u1fbb\u1fc8-\u1fcb\u1fd8-\u1fdb' + u'\u1fe8-\u1fec\u1ff8-\u1ffb\u2102\u2107\u210b-\u210d\u2110-\u2112' + u'\u2115\u2119-\u211d\u2124\u2126\u2128\u212a-\u212d\u2130-\u2133' + u'\u213e-\u213f\u2145\u2183\u2c00-\u2c2e\u2c60\u2c62-\u2c64\u2c67' + u'\u2c69\u2c6b\u2c6d-\u2c6f\u2c72\u2c75\u2c80\u2c82\u2c84\u2c86' + u'\u2c88\u2c8a\u2c8c\u2c8e\u2c90\u2c92\u2c94\u2c96\u2c98\u2c9a' + u'\u2c9c\u2c9e\u2ca0\u2ca2\u2ca4\u2ca6\u2ca8\u2caa\u2cac\u2cae' + u'\u2cb0\u2cb2\u2cb4\u2cb6\u2cb8\u2cba\u2cbc\u2cbe\u2cc0\u2cc2' + u'\u2cc4\u2cc6\u2cc8\u2cca\u2ccc\u2cce\u2cd0\u2cd2\u2cd4\u2cd6' + u'\u2cd8\u2cda\u2cdc\u2cde\u2ce0\u2ce2\ua640\ua642\ua644\ua646' + u'\ua648\ua64a\ua64c\ua64e\ua650\ua652\ua654\ua656\ua658\ua65a' + u'\ua65c\ua65e\ua662\ua664\ua666\ua668\ua66a\ua66c\ua680\ua682' + u'\ua684\ua686\ua688\ua68a\ua68c\ua68e\ua690\ua692\ua694\ua696' + u'\ua722\ua724\ua726\ua728\ua72a\ua72c\ua72e\ua732\ua734\ua736' + u'\ua738\ua73a\ua73c\ua73e\ua740\ua742\ua744\ua746\ua748\ua74a' + u'\ua74c\ua74e\ua750\ua752\ua754\ua756\ua758\ua75a\ua75c\ua75e' + u'\ua760\ua762\ua764\ua766\ua768\ua76a\ua76c\ua76e\ua779\ua77b' + u'\ua77d-\ua77e\ua780\ua782\ua784\ua786\ua78b\uff21-\uff3a]') idrest = ur'%s(?:%s|[0-9])*(?:(?<=_)%s)?' % (letter, letter, op) @@ -161,7 +267,7 @@ class ScalaLexer(RegexLexer): (r'(type)(\s+)', bygroups(Keyword, Text), 'type'), (r'""".*?"""(?!")', String), (r'"(\\\\|\\"|[^"])*"', String), - (r"'\\.'|'[^\\]'|'\\u[0-9a-f]{4}'", String.Char), + (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Char), # (ur'(\.)(%s|%s|`[^`]+`)' % (idrest, op), bygroups(Operator, # Name.Attribute)), (idrest, Name), @@ -171,7 +277,7 @@ class ScalaLexer(RegexLexer): (op, Operator), (r'([0-9][0-9]*\.[0-9]*|\.[0-9]+)([eE][+-]?[0-9]+)?[fFdD]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+L?', Number.Integer), (r'\n', Text) ], @@ -249,7 +355,7 @@ class GosuLexer(RegexLexer): (r'(var|delegate|construct|function|private|internal|protected|' r'public|abstract|override|final|static|extends|transient|' r'implements|represents|readonly)\b', Keyword.Declaration), - (r'(property\s+)(get|set|)', Keyword.Declaration), + (r'(property\s+)(get|set)?', Keyword.Declaration), (r'(boolean|byte|char|double|float|int|long|short|void|block)\b', Keyword.Type), (r'(package)(\s+)', bygroups(Keyword.Namespace, Text)), @@ -357,13 +463,13 @@ class GroovyLexer(RegexLexer): (r"'(\\\\|\\'|[^'])*'", String.Single), (r'\$/((?!/\$).)*/\$', String), (r'/(\\\\|\\"|[^/])*/', String), - (r"'\\.'|'[^\\]'|'\\u[0-9a-f]{4}'", String.Char), + (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Char), (r'(\.)([a-zA-Z_][a-zA-Z0-9_]*)', bygroups(Operator, Name.Attribute)), (r'[a-zA-Z_][a-zA-Z0-9_]*:', Name.Label), (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), (r'[~\^\*!%&\[\]\(\)\{\}<>\|+=:;,./?-]', Operator), (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+L?', Number.Integer), (r'\n', Text) ], @@ -675,7 +781,7 @@ class ClojureLexer(RegexLexer): (r'::?' + valid_name, String.Symbol), # special operators - (r'~@|[`\'#^~&]', Operator), + (r'~@|[`\'#^~&@]', Operator), # highlight the special forms (_multi_escape(special_forms), Keyword), @@ -742,7 +848,7 @@ class TeaLangLexer(RegexLexer): (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), (r'(isa|[.]{3}|[.]{2}|[=#!<>+-/%&;,.\*\\\(\)\[\]\{\}])', Operator), (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+L?', Number.Integer), (r'\n', Text) ], @@ -754,6 +860,7 @@ class TeaLangLexer(RegexLexer): ], } + class CeylonLexer(RegexLexer): """ For `Ceylon <http://ceylon-lang.org/>`_ source code. @@ -781,10 +888,11 @@ class CeylonLexer(RegexLexer): (r'[^\S\n]+', Text), (r'//.*?\n', Comment.Single), (r'/\*.*?\*/', Comment.Multiline), - (r'(variable|shared|abstract|doc|by|formal|actual)', Name.Decorator), - (r'(break|case|catch|continue|default|else|finally|for|in|variable|' - r'if|return|switch|this|throw|try|while|is|exists|nonempty|then|outer)\b', - Keyword), + (r'(variable|shared|abstract|doc|by|formal|actual)', + Name.Decorator), + (r'(break|case|catch|continue|default|else|finally|for|in|' + r'variable|if|return|switch|this|throw|try|while|is|exists|' + r'nonempty|then|outer)\b', Keyword), (r'(abstracts|extends|satisfies|adapts|' r'super|given|of|out|assign|' r'transient|volatile)\b', Keyword.Declaration), @@ -792,20 +900,24 @@ class CeylonLexer(RegexLexer): Keyword.Type), (r'(package)(\s+)', bygroups(Keyword.Namespace, Text)), (r'(true|false|null)\b', Keyword.Constant), - (r'(class|interface|object)(\s+)', bygroups(Keyword.Declaration, Text), 'class'), + (r'(class|interface|object)(\s+)', + bygroups(Keyword.Declaration, Text), 'class'), (r'(import)(\s+)', bygroups(Keyword.Namespace, Text), 'import'), (r'"(\\\\|\\"|[^"])*"', String), - (r"'\\.'|'[^\\]'|'\\u[0-9a-f]{4}'", String.Quoted), - (r"`\\.`|`[^\\]`|`\\u[0-9a-f]{4}`", String.Char), - (r'(\.)([a-zA-Z_][a-zA-Z0-9_]*)', bygroups(Operator, Name.Attribute)), + (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Quoted), + (r"`\\.`|`[^\\]`|`\\u[0-9a-fA-F]{4}`", String.Char), + (r'(\.)([a-zA-Z_][a-zA-Z0-9_]*)', + bygroups(Operator, Name.Attribute)), (r'[a-zA-Z_][a-zA-Z0-9_]*:', Name.Label), (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), (r'[~\^\*!%&\[\]\(\)\{\}<>\|+=:;,./?-]', Operator), (r'\d{1,3}(_\d{3})+\.\d{1,3}(_\d{3})+[kMGTPmunpf]?', Number.Float), - (r'\d{1,3}(_\d{3})+\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', Number.Float), + (r'\d{1,3}(_\d{3})+\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', + Number.Float), (r'[0-9][0-9]*\.\d{1,3}(_\d{3})+[kMGTPmunpf]?', Number.Float), - (r'[0-9][0-9]*\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'[0-9][0-9]*\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', + Number.Float), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'\d{1,3}(_\d{3})+[kMGTP]?', Number.Integer), (r'[0-9]+[kMGTP]?', Number.Integer), (r'\n', Text) @@ -814,10 +926,12 @@ class CeylonLexer(RegexLexer): (r'[a-zA-Z_][a-zA-Z0-9_]*', Name.Class, '#pop') ], 'import': [ - (r'[a-zA-Z0-9_.]+\w+ \{([a-zA-Z,]+|\.\.\.)\}', Name.Namespace, '#pop') + (r'[a-zA-Z0-9_.]+\w+ \{([a-zA-Z,]+|\.\.\.)\}', + Name.Namespace, '#pop') ], } + class KotlinLexer(RegexLexer): """ For `Kotlin <http://confluence.jetbrains.net/display/Kotlin/>`_ @@ -977,7 +1091,7 @@ class XtendLexer(RegexLexer): (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), (r'[~\^\*!%&\[\]\(\)\{\}<>\|+=:;,./?-]', Operator), (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), - (r'0x[0-9a-f]+', Number.Hex), + (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+L?', Number.Integer), (r'\n', Text) ], diff --git a/pygments/lexers/math.py b/pygments/lexers/math.py index 3cba201b..537c6d0e 100644 --- a/pygments/lexers/math.py +++ b/pygments/lexers/math.py @@ -5,12 +5,13 @@ Lexers for math languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import re +from pygments.util import shebang_matches from pygments.lexer import Lexer, RegexLexer, bygroups, include, \ combined, do_insertions from pygments.token import Comment, String, Punctuation, Keyword, Name, \ @@ -23,10 +24,15 @@ from pygments.lexers import _stan_builtins __all__ = ['JuliaLexer', 'JuliaConsoleLexer', 'MuPADLexer', 'MatlabLexer', 'MatlabSessionLexer', 'OctaveLexer', 'ScilabLexer', 'NumPyLexer', 'RConsoleLexer', 'SLexer', 'JagsLexer', 'BugsLexer', 'StanLexer', - 'RdLexer'] + 'IDLLexer', 'RdLexer'] class JuliaLexer(RegexLexer): + """ + For `Julia <http://julialang.org/>`_ source code. + + *New in Pygments 1.6.* + """ name = 'Julia' aliases = ['julia','jl'] filenames = ['*.jl'] @@ -52,9 +58,9 @@ class JuliaLexer(RegexLexer): # keywords (r'(begin|while|for|in|return|break|continue|' r'macro|quote|let|if|elseif|else|try|catch|end|' - r'bitstype|ccall|do)\b', Keyword), + r'bitstype|ccall|do|using|module|import|export|' + r'importall|baremodule)\b', Keyword), (r'(local|global|const)\b', Keyword.Declaration), - (r'(module|import|export)\b', Keyword.Reserved), (r'(Bool|Int|Int8|Int16|Int32|Int64|Uint|Uint8|Uint16|Uint32|Uint64' r'|Float32|Float64|Complex64|Complex128|Any|Nothing|None)\b', Keyword.Type), @@ -78,7 +84,8 @@ class JuliaLexer(RegexLexer): (r'`(?s).*?`', String.Backtick), # chars - (r"'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,3}|\\u[a-fA-F0-9]{1,4}|\\U[a-fA-F0-9]{1,6}|[^\\\'\n])'", String.Char), + (r"'(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,3}|\\u[a-fA-F0-9]{1,4}|" + r"\\U[a-fA-F0-9]{1,6}|[^\\\'\n])'", String.Char), # try to match trailing transpose (r'(?<=[.\w\)\]])\'+', Operator), @@ -92,8 +99,8 @@ class JuliaLexer(RegexLexer): (r'[a-zA-Z_][a-zA-Z0-9_]*', Name), # numbers - (r'(\d+\.\d*|\d*\.\d+)([eE][+-]?[0-9]+)?', Number.Float), - (r'\d+[eE][+-]?[0-9]+', Number.Float), + (r'(\d+\.\d*|\d*\.\d+)([eEf][+-]?[0-9]+)?', Number.Float), + (r'\d+[eEf][+-]?[0-9]+', Number.Float), (r'0b[01]+', Number.Binary), (r'0o[0-7]+', Number.Oct), (r'0x[a-fA-F0-9]+', Number.Hex), @@ -137,6 +144,8 @@ line_re = re.compile('.*?\n') class JuliaConsoleLexer(Lexer): """ For Julia console sessions. Modeled after MatlabSessionLexer. + + *New in Pygments 1.6.* """ name = 'Julia console' aliases = ['jlcon'] @@ -335,6 +344,10 @@ class MatlabLexer(RegexLexer): # (not great, but handles common cases...) (r'(?<=[\w\)\]])\'', Operator), + (r'(\d+\.\d*|\d*\.\d+)([eEf][+-]?[0-9]+)?', Number.Float), + (r'\d+[eEf][+-]?[0-9]+', Number.Float), + (r'\d+', Number.Integer), + (r'(?<![\w\)\]])\'', String, 'string'), ('[a-zA-Z_][a-zA-Z0-9_]*', Name), (r'.', Text), @@ -781,6 +794,10 @@ class OctaveLexer(RegexLexer): (r'"[^"]*"', String), + (r'(\d+\.\d*|\d*\.\d+)([eEf][+-]?[0-9]+)?', Number.Float), + (r'\d+[eEf][+-]?[0-9]+', Number.Float), + (r'\d+', Number.Integer), + # quote can be transpose, instead of string: # (not great, but handles common cases...) (r'(?<=[\w\)\]])\'', Operator), @@ -802,8 +819,7 @@ class OctaveLexer(RegexLexer): def analyse_text(text): if re.match('^\s*[%#]', text, re.M): #Comment - return 0.9 - return 0.1 + return 0.1 class ScilabLexer(RegexLexer): @@ -852,6 +868,10 @@ class ScilabLexer(RegexLexer): (r'(?<=[\w\)\]])\'', Operator), (r'(?<![\w\)\]])\'', String, 'string'), + (r'(\d+\.\d*|\d*\.\d+)([eEf][+-]?[0-9]+)?', Number.Float), + (r'\d+[eEf][+-]?[0-9]+', Number.Float), + (r'\d+', Number.Integer), + ('[a-zA-Z_][a-zA-Z0-9_]*', Name), (r'.', Text), ], @@ -1092,7 +1112,8 @@ class SLexer(RegexLexer): class BugsLexer(RegexLexer): """ - Pygments Lexer for OpenBugs and WinBugs models. + Pygments Lexer for `OpenBugs <http://www.openbugs.info/w/>`_ and WinBugs + models. *New in Pygments 1.6.* """ @@ -1155,7 +1176,7 @@ class BugsLexer(RegexLexer): include('comments'), include('whitespace'), # Block start - (r'(?s)(model)(\s+)({)', + (r'(model)(\s+)({)', bygroups(Keyword.Namespace, Text, Punctuation)), # Reserved Words (r'(for|in)(?![0-9a-zA-Z\._])', Keyword.Reserved), @@ -1237,7 +1258,7 @@ class JagsLexer(RegexLexer): include('comments'), include('whitespace'), # Block start - (r'(?s)(model|data)(\s+)({)', + (r'(model|data)(\s+)({)', bygroups(Keyword.Namespace, Text, Punctuation)), (r'var(?![0-9a-zA-Z\._])', Keyword.Declaration), # Reserved Words @@ -1284,7 +1305,7 @@ class StanLexer(RegexLexer): filenames = ['*.stan'] _RESERVED = ('for', 'in', 'while', 'repeat', 'until', 'if', - 'then', 'else', 'true', 'false', 'T', + 'then', 'else', 'true', 'false', 'T', 'lower', 'upper', 'print') _TYPES = ('int', 'real', 'vector', 'simplex', 'ordered', 'row_vector', @@ -1307,7 +1328,7 @@ class StanLexer(RegexLexer): # block start include('whitespace'), # Block start - (r'(?s)(%s)(\s*)({)' % + (r'(%s)(\s*)({)' % r'|'.join(('data', r'transformed\s+?data', 'parameters', r'transformed\s+parameters', 'model', r'generated\s+quantities')), @@ -1335,7 +1356,7 @@ class StanLexer(RegexLexer): # Integer Literals (r'-?[0-9]+', Number.Integer), # Assignment operators - # SLexer makes these tokens Operators. + # SLexer makes these tokens Operators. (r'<-|~', Operator), # Infix and prefix operators (and = ) (r"\+|-|\.?\*|\.?/|\\|'|=", Operator), @@ -1351,6 +1372,247 @@ class StanLexer(RegexLexer): return 0.0 +class IDLLexer(RegexLexer): + """ + Pygments Lexer for IDL (Interactive Data Language). + + *New in Pygments 1.6.* + """ + name = 'IDL' + aliases = ['idl'] + filenames = ['*.pro'] + mimetypes = ['text/idl'] + + _RESERVED = ['and', 'begin', 'break', 'case', 'common', 'compile_opt', + 'continue', 'do', 'else', 'end', 'endcase', 'elseelse', + 'endfor', 'endforeach', 'endif', 'endrep', 'endswitch', + 'endwhile', 'eq', 'for', 'foreach', 'forward_function', + 'function', 'ge', 'goto', 'gt', 'if', 'inherits', 'le', + 'lt', 'mod', 'ne', 'not', 'of', 'on_ioerror', 'or', 'pro', + 'repeat', 'switch', 'then', 'until', 'while', 'xor'] + """Reserved words from: http://www.exelisvis.com/docs/reswords.html""" + + _BUILTIN_LIB = ['abs', 'acos', 'adapt_hist_equal', 'alog', 'alog10', + 'amoeba', 'annotate', 'app_user_dir', 'app_user_dir_query', + 'arg_present', 'array_equal', 'array_indices', 'arrow', + 'ascii_template', 'asin', 'assoc', 'atan', 'axis', + 'a_correlate', 'bandpass_filter', 'bandreject_filter', + 'barplot', 'bar_plot', 'beseli', 'beselj', 'beselk', + 'besely', 'beta', 'bilinear', 'binary_template', 'bindgen', + 'binomial', 'bin_date', 'bit_ffs', 'bit_population', + 'blas_axpy', 'blk_con', 'box_cursor', 'breakpoint', + 'broyden', 'butterworth', 'bytarr', 'byte', 'byteorder', + 'bytscl', 'caldat', 'calendar', 'call_external', + 'call_function', 'call_method', 'call_procedure', 'canny', + 'catch', 'cd', 'cdf_[0-9a-za-z_]*', 'ceil', 'chebyshev', + 'check_math', + 'chisqr_cvf', 'chisqr_pdf', 'choldc', 'cholsol', 'cindgen', + 'cir_3pnt', 'close', 'cluster', 'cluster_tree', 'clust_wts', + 'cmyk_convert', 'colorbar', 'colorize_sample', + 'colormap_applicable', 'colormap_gradient', + 'colormap_rotation', 'colortable', 'color_convert', + 'color_exchange', 'color_quan', 'color_range_map', 'comfit', + 'command_line_args', 'complex', 'complexarr', 'complexround', + 'compute_mesh_normals', 'cond', 'congrid', 'conj', + 'constrained_min', 'contour', 'convert_coord', 'convol', + 'convol_fft', 'coord2to3', 'copy_lun', 'correlate', 'cos', + 'cosh', 'cpu', 'cramer', 'create_cursor', 'create_struct', + 'create_view', 'crossp', 'crvlength', 'cti_test', + 'ct_luminance', 'cursor', 'curvefit', 'cvttobm', 'cv_coord', + 'cw_animate', 'cw_animate_getp', 'cw_animate_load', + 'cw_animate_run', 'cw_arcball', 'cw_bgroup', 'cw_clr_index', + 'cw_colorsel', 'cw_defroi', 'cw_field', 'cw_filesel', + 'cw_form', 'cw_fslider', 'cw_light_editor', + 'cw_light_editor_get', 'cw_light_editor_set', 'cw_orient', + 'cw_palette_editor', 'cw_palette_editor_get', + 'cw_palette_editor_set', 'cw_pdmenu', 'cw_rgbslider', + 'cw_tmpl', 'cw_zoom', 'c_correlate', 'dblarr', 'db_exists', + 'dcindgen', 'dcomplex', 'dcomplexarr', 'define_key', + 'define_msgblk', 'define_msgblk_from_file', 'defroi', + 'defsysv', 'delvar', 'dendrogram', 'dendro_plot', 'deriv', + 'derivsig', 'determ', 'device', 'dfpmin', 'diag_matrix', + 'dialog_dbconnect', 'dialog_message', 'dialog_pickfile', + 'dialog_printersetup', 'dialog_printjob', + 'dialog_read_image', 'dialog_write_image', 'digital_filter', + 'dilate', 'dindgen', 'dissolve', 'dist', 'distance_measure', + 'dlm_load', 'dlm_register', 'doc_library', 'double', + 'draw_roi', 'edge_dog', 'efont', 'eigenql', 'eigenvec', + 'ellipse', 'elmhes', 'emboss', 'empty', 'enable_sysrtn', + 'eof', 'eos_[0-9a-za-z_]*', 'erase', 'erf', 'erfc', 'erfcx', + 'erode', 'errorplot', 'errplot', 'estimator_filter', + 'execute', 'exit', 'exp', 'expand', 'expand_path', 'expint', + 'extrac', 'extract_slice', 'factorial', 'fft', 'filepath', + 'file_basename', 'file_chmod', 'file_copy', 'file_delete', + 'file_dirname', 'file_expand_path', 'file_info', + 'file_lines', 'file_link', 'file_mkdir', 'file_move', + 'file_poll_input', 'file_readlink', 'file_same', + 'file_search', 'file_test', 'file_which', 'findgen', + 'finite', 'fix', 'flick', 'float', 'floor', 'flow3', + 'fltarr', 'flush', 'format_axis_values', 'free_lun', + 'fstat', 'fulstr', 'funct', 'fv_test', 'fx_root', + 'fz_roots', 'f_cvf', 'f_pdf', 'gamma', 'gamma_ct', + 'gauss2dfit', 'gaussfit', 'gaussian_function', 'gaussint', + 'gauss_cvf', 'gauss_pdf', 'gauss_smooth', 'getenv', + 'getwindows', 'get_drive_list', 'get_dxf_objects', + 'get_kbrd', 'get_login_info', 'get_lun', 'get_screen_size', + 'greg2jul', 'grib_[0-9a-za-z_]*', 'grid3', 'griddata', + 'grid_input', 'grid_tps', 'gs_iter', + 'h5[adfgirst]_[0-9a-za-z_]*', 'h5_browser', 'h5_close', + 'h5_create', 'h5_get_libversion', 'h5_open', 'h5_parse', + 'hanning', 'hash', 'hdf_[0-9a-za-z_]*', 'heap_free', + 'heap_gc', 'heap_nosave', 'heap_refcount', 'heap_save', + 'help', 'hilbert', 'histogram', 'hist_2d', 'hist_equal', + 'hls', 'hough', 'hqr', 'hsv', 'h_eq_ct', 'h_eq_int', + 'i18n_multibytetoutf8', 'i18n_multibytetowidechar', + 'i18n_utf8tomultibyte', 'i18n_widechartomultibyte', + 'ibeta', 'icontour', 'iconvertcoord', 'idelete', 'identity', + 'idlexbr_assistant', 'idlitsys_createtool', 'idl_base64', + 'idl_validname', 'iellipse', 'igamma', 'igetcurrent', + 'igetdata', 'igetid', 'igetproperty', 'iimage', 'image', + 'image_cont', 'image_statistics', 'imaginary', 'imap', + 'indgen', 'intarr', 'interpol', 'interpolate', + 'interval_volume', 'int_2d', 'int_3d', 'int_tabulated', + 'invert', 'ioctl', 'iopen', 'iplot', 'ipolygon', + 'ipolyline', 'iputdata', 'iregister', 'ireset', 'iresolve', + 'irotate', 'ir_filter', 'isa', 'isave', 'iscale', + 'isetcurrent', 'isetproperty', 'ishft', 'isocontour', + 'isosurface', 'isurface', 'itext', 'itranslate', 'ivector', + 'ivolume', 'izoom', 'i_beta', 'journal', 'json_parse', + 'json_serialize', 'jul2greg', 'julday', 'keyword_set', + 'krig2d', 'kurtosis', 'kw_test', 'l64indgen', 'label_date', + 'label_region', 'ladfit', 'laguerre', 'laplacian', + 'la_choldc', 'la_cholmprove', 'la_cholsol', 'la_determ', + 'la_eigenproblem', 'la_eigenql', 'la_eigenvec', 'la_elmhes', + 'la_gm_linear_model', 'la_hqr', 'la_invert', + 'la_least_squares', 'la_least_square_equality', + 'la_linear_equation', 'la_ludc', 'la_lumprove', 'la_lusol', + 'la_svd', 'la_tridc', 'la_trimprove', 'la_triql', + 'la_trired', 'la_trisol', 'least_squares_filter', 'leefilt', + 'legend', 'legendre', 'linbcg', 'lindgen', 'linfit', + 'linkimage', 'list', 'll_arc_distance', 'lmfit', 'lmgr', + 'lngamma', 'lnp_test', 'loadct', 'locale_get', + 'logical_and', 'logical_or', 'logical_true', 'lon64arr', + 'lonarr', 'long', 'long64', 'lsode', 'ludc', 'lumprove', + 'lusol', 'lu_complex', 'machar', 'make_array', 'make_dll', + 'make_rt', 'map', 'mapcontinents', 'mapgrid', 'map_2points', + 'map_continents', 'map_grid', 'map_image', 'map_patch', + 'map_proj_forward', 'map_proj_image', 'map_proj_info', + 'map_proj_init', 'map_proj_inverse', 'map_set', + 'matrix_multiply', 'matrix_power', 'max', 'md_test', + 'mean', 'meanabsdev', 'mean_filter', 'median', 'memory', + 'mesh_clip', 'mesh_decimate', 'mesh_issolid', 'mesh_merge', + 'mesh_numtriangles', 'mesh_obj', 'mesh_smooth', + 'mesh_surfacearea', 'mesh_validate', 'mesh_volume', + 'message', 'min', 'min_curve_surf', 'mk_html_help', + 'modifyct', 'moment', 'morph_close', 'morph_distance', + 'morph_gradient', 'morph_hitormiss', 'morph_open', + 'morph_thin', 'morph_tophat', 'multi', 'm_correlate', + 'ncdf_[0-9a-za-z_]*', 'newton', 'noise_hurl', 'noise_pick', + 'noise_scatter', 'noise_slur', 'norm', 'n_elements', + 'n_params', 'n_tags', 'objarr', 'obj_class', 'obj_destroy', + 'obj_hasmethod', 'obj_isa', 'obj_new', 'obj_valid', + 'online_help', 'on_error', 'open', 'oplot', 'oploterr', + 'parse_url', 'particle_trace', 'path_cache', 'path_sep', + 'pcomp', 'plot', 'plot3d', 'ploterr', 'plots', 'plot_3dbox', + 'plot_field', 'pnt_line', 'point_lun', 'polarplot', + 'polar_contour', 'polar_surface', 'poly', 'polyfill', + 'polyfillv', 'polygon', 'polyline', 'polyshade', 'polywarp', + 'poly_2d', 'poly_area', 'poly_fit', 'popd', 'powell', + 'pref_commit', 'pref_get', 'pref_set', 'prewitt', 'primes', + 'print', 'printd', 'product', 'profile', 'profiler', + 'profiles', 'project_vol', 'psafm', 'pseudo', + 'ps_show_fonts', 'ptrarr', 'ptr_free', 'ptr_new', + 'ptr_valid', 'pushd', 'p_correlate', 'qgrid3', 'qhull', + 'qromb', 'qromo', 'qsimp', 'query_ascii', 'query_bmp', + 'query_csv', 'query_dicom', 'query_gif', 'query_image', + 'query_jpeg', 'query_jpeg2000', 'query_mrsid', 'query_pict', + 'query_png', 'query_ppm', 'query_srf', 'query_tiff', + 'query_wav', 'radon', 'randomn', 'randomu', 'ranks', + 'rdpix', 'read', 'reads', 'readu', 'read_ascii', + 'read_binary', 'read_bmp', 'read_csv', 'read_dicom', + 'read_gif', 'read_image', 'read_interfile', 'read_jpeg', + 'read_jpeg2000', 'read_mrsid', 'read_pict', 'read_png', + 'read_ppm', 'read_spr', 'read_srf', 'read_sylk', + 'read_tiff', 'read_wav', 'read_wave', 'read_x11_bitmap', + 'read_xwd', 'real_part', 'rebin', 'recall_commands', + 'recon3', 'reduce_colors', 'reform', 'region_grow', + 'register_cursor', 'regress', 'replicate', + 'replicate_inplace', 'resolve_all', 'resolve_routine', + 'restore', 'retall', 'return', 'reverse', 'rk4', 'roberts', + 'rot', 'rotate', 'round', 'routine_filepath', + 'routine_info', 'rs_test', 'r_correlate', 'r_test', + 'save', 'savgol', 'scale3', 'scale3d', 'scope_level', + 'scope_traceback', 'scope_varfetch', 'scope_varname', + 'search2d', 'search3d', 'sem_create', 'sem_delete', + 'sem_lock', 'sem_release', 'setenv', 'set_plot', + 'set_shading', 'sfit', 'shade_surf', 'shade_surf_irr', + 'shade_volume', 'shift', 'shift_diff', 'shmdebug', 'shmmap', + 'shmunmap', 'shmvar', 'show3', 'showfont', 'simplex', 'sin', + 'sindgen', 'sinh', 'size', 'skewness', 'skip_lun', + 'slicer3', 'slide_image', 'smooth', 'sobel', 'socket', + 'sort', 'spawn', 'spher_harm', 'sph_4pnt', 'sph_scat', + 'spline', 'spline_p', 'spl_init', 'spl_interp', 'sprsab', + 'sprsax', 'sprsin', 'sprstp', 'sqrt', 'standardize', + 'stddev', 'stop', 'strarr', 'strcmp', 'strcompress', + 'streamline', 'stregex', 'stretch', 'string', 'strjoin', + 'strlen', 'strlowcase', 'strmatch', 'strmessage', 'strmid', + 'strpos', 'strput', 'strsplit', 'strtrim', 'struct_assign', + 'struct_hide', 'strupcase', 'surface', 'surfr', 'svdc', + 'svdfit', 'svsol', 'swap_endian', 'swap_endian_inplace', + 'symbol', 'systime', 's_test', 't3d', 'tag_names', 'tan', + 'tanh', 'tek_color', 'temporary', 'tetra_clip', + 'tetra_surface', 'tetra_volume', 'text', 'thin', 'threed', + 'timegen', 'time_test2', 'tm_test', 'total', 'trace', + 'transpose', 'triangulate', 'trigrid', 'triql', 'trired', + 'trisol', 'tri_surf', 'truncate_lun', 'ts_coef', 'ts_diff', + 'ts_fcast', 'ts_smooth', 'tv', 'tvcrs', 'tvlct', 'tvrd', + 'tvscl', 'typename', 't_cvt', 't_pdf', 'uindgen', 'uint', + 'uintarr', 'ul64indgen', 'ulindgen', 'ulon64arr', 'ulonarr', + 'ulong', 'ulong64', 'uniq', 'unsharp_mask', 'usersym', + 'value_locate', 'variance', 'vector', 'vector_field', 'vel', + 'velovect', 'vert_t3d', 'voigt', 'voronoi', 'voxel_proj', + 'wait', 'warp_tri', 'watershed', 'wdelete', 'wf_draw', + 'where', 'widget_base', 'widget_button', 'widget_combobox', + 'widget_control', 'widget_displaycontextmen', 'widget_draw', + 'widget_droplist', 'widget_event', 'widget_info', + 'widget_label', 'widget_list', 'widget_propertysheet', + 'widget_slider', 'widget_tab', 'widget_table', + 'widget_text', 'widget_tree', 'widget_tree_move', + 'widget_window', 'wiener_filter', 'window', 'writeu', + 'write_bmp', 'write_csv', 'write_gif', 'write_image', + 'write_jpeg', 'write_jpeg2000', 'write_nrif', 'write_pict', + 'write_png', 'write_ppm', 'write_spr', 'write_srf', + 'write_sylk', 'write_tiff', 'write_wav', 'write_wave', + 'wset', 'wshow', 'wtn', 'wv_applet', 'wv_cwt', + 'wv_cw_wavelet', 'wv_denoise', 'wv_dwt', 'wv_fn_coiflet', + 'wv_fn_daubechies', 'wv_fn_gaussian', 'wv_fn_haar', + 'wv_fn_morlet', 'wv_fn_paul', 'wv_fn_symlet', + 'wv_import_data', 'wv_import_wavelet', 'wv_plot3d_wps', + 'wv_plot_multires', 'wv_pwt', 'wv_tool_denoise', + 'xbm_edit', 'xdisplayfile', 'xdxf', 'xfont', + 'xinteranimate', 'xloadct', 'xmanager', 'xmng_tmpl', + 'xmtool', 'xobjview', 'xobjview_rotate', + 'xobjview_write_image', 'xpalette', 'xpcolor', 'xplot3d', + 'xregistered', 'xroi', 'xsq_test', 'xsurface', 'xvaredit', + 'xvolume', 'xvolume_rotate', 'xvolume_write_image', + 'xyouts', 'zoom', 'zoom_24'] + """Functions from: http://www.exelisvis.com/docs/routines-1.html""" + + tokens = { + 'root': [ + (r'^\s*;.*?\n', Comment.Singleline), + (r'\b(' + '|'.join(_RESERVED) + r')\b', Keyword), + (r'\b(' + '|'.join(_BUILTIN_LIB) + r')\b', Name.Builtin), + (r'\+=|-=|\^=|\*=|/=|#=|##=|<=|>=|=', Operator), + (r'\+\+|--|->|\+|-|##|#|\*|/|<|>|&&|\^|~|\|\|\?|:', Operator), + (r'\b(mod=|lt=|le=|eq=|ne=|ge=|gt=|not=|and=|or=|xor=)', Operator), + (r'\b(mod|lt|le|eq|ne|ge|gt|not|and|or|xor)\b', Operator), + (r'\b[0-9](L|B|S|UL|ULL|LL)?\b', Number), + (r'.', Text), + ] + } + + class RdLexer(RegexLexer): """ Pygments Lexer for R documentation (Rd) files diff --git a/pygments/lexers/other.py b/pygments/lexers/other.py index 8dfb3145..c8557922 100644 --- a/pygments/lexers/other.py +++ b/pygments/lexers/other.py @@ -5,7 +5,7 @@ Lexers for other languages. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -19,6 +19,7 @@ from pygments.util import get_bool_opt from pygments.lexers.web import HtmlLexer from pygments.lexers._openedgebuiltins import OPENEDGEKEYWORDS +from pygments.lexers._robotframeworklexer import RobotFrameworkLexer # backwards compatibility from pygments.lexers.sql import SqlLexer, MySqlLexer, SqliteConsoleLexer @@ -32,7 +33,9 @@ __all__ = ['BrainfuckLexer', 'BefungeLexer', 'RedcodeLexer', 'MOOCodeLexer', 'AutohotkeyLexer', 'GoodDataCLLexer', 'MaqlLexer', 'ProtoBufLexer', 'HybrisLexer', 'AwkLexer', 'Cfengine3Lexer', 'SnobolLexer', 'ECLLexer', 'UrbiscriptLexer', 'OpenEdgeLexer', 'BroLexer', - 'MscgenLexer', 'KconfigLexer', 'VGLLexer', 'SourcePawnLexer'] + 'MscgenLexer', 'KconfigLexer', 'VGLLexer', 'SourcePawnLexer', + 'RobotFrameworkLexer', 'PuppetLexer', 'NSISLexer', 'RPMSpecLexer', + 'CbmBasicV2Lexer', 'AutoItLexer'] class ECLLexer(RegexLexer): @@ -1255,7 +1258,8 @@ class ModelicaLexer(RegexLexer): ], 'classes': [ (r'(block|class|connector|function|model|package|' - r'record|type)\b', Name.Class), + r'record|type)(\s+)([A-Za-z_]+)', + bygroups(Keyword, Text, Name.Class)) ], 'string': [ (r'"', String, '#pop'), @@ -1483,9 +1487,9 @@ class RebolLexer(RegexLexer): (r'}', Comment, '#pop'), ], 'commentBlock': [ - (r'\[',Comment, '#push'), - (r'\]',Comment, '#pop'), - (r'[^(\[\])]*', Comment), + (r'\[', Comment, '#push'), + (r'\]', Comment, '#pop'), + (r'[^(\[\])]+', Comment), ], } @@ -1736,6 +1740,7 @@ class NewspeakLexer(RegexLexer): ] } + class GherkinLexer(RegexLexer): """ For `Gherkin <http://github.com/aslakhellesoy/gherkin/>` syntax. @@ -1832,10 +1837,16 @@ class GherkinLexer(RegexLexer): include('table_vars'), include('numbers'), (r'(\s*)(@[^@\r\n\t ]+)', bygroups(Name.Function, Name.Tag)), - (step_keywords, bygroups(Name.Function, Keyword), "step_content_root"), - (feature_keywords, bygroups(Keyword, Keyword, Name.Function), 'narrative'), - (feature_element_keywords, bygroups(Name.Function, Keyword, Keyword, Name.Function), "feature_elements"), - (examples_keywords, bygroups(Name.Function, Keyword, Keyword, Name.Function), "examples_table"), + (step_keywords, bygroups(Name.Function, Keyword), + 'step_content_root'), + (feature_keywords, bygroups(Keyword, Keyword, Name.Function), + 'narrative'), + (feature_element_keywords, + bygroups(Name.Function, Keyword, Keyword, Name.Function), + 'feature_elements'), + (examples_keywords, + bygroups(Name.Function, Keyword, Keyword, Name.Function), + 'examples_table'), (r'(\s|.)', Name.Function), ] } @@ -2771,7 +2782,7 @@ class OpenEdgeLexer(RegexLexer): keywords = (r'(?i)(^|(?<=[^0-9a-z_\-]))(' + r'|'.join(OPENEDGEKEYWORDS) + - r')\s*($|(?=[^0-9a-z_\-]))') + r')\s*($|(?=[^0-9a-z_\-]))') tokens = { 'root': [ (r'/\*', Comment.Multiline, 'comment'), @@ -2834,7 +2845,7 @@ class BroLexer(RegexLexer): r'|pattern|port|record|set|string|subnet|table|time|timer' r'|vector)\b', Keyword.Type), (r'(T|F)\b', Keyword.Constant), - (r'(&)((?:add|delete|expire)_func|attr|(create|read|write)_expire' + (r'(&)((?:add|delete|expire)_func|attr|(?:create|read|write)_expire' r'|default|disable_print_hook|raw_output|encrypt|group|log' r'|mergeable|optional|persistent|priority|redef' r'|rotate_(?:interval|size)|synchronized)\b', bygroups(Punctuation, @@ -2882,6 +2893,44 @@ class BroLexer(RegexLexer): } +class CbmBasicV2Lexer(RegexLexer): + """ + For CBM BASIC V2 sources. + + *New in Pygments 1.6.* + """ + name = 'CBM BASIC V2' + aliases = ['cbmbas'] + filenames = ['*.bas'] + + flags = re.IGNORECASE + + tokens = { + 'root': [ + (r'rem.*\n', Comment.Single), + (r'\s+', Text), + (r'new|run|end|for|to|next|step|go(to|sub)?|on|return|stop|cont' + r'|if|then|input#?|read|wait|load|save|verify|poke|sys|print#?' + r'|list|clr|cmd|open|close|get#?', Keyword.Reserved), + (r'data|restore|dim|let|def|fn', Keyword.Declaration), + (r'tab|spc|sgn|int|abs|usr|fre|pos|sqr|rnd|log|exp|cos|sin|tan|atn' + r'|peek|len|val|asc|(str|chr|left|right|mid)\$', Name.Builtin), + (r'[-+*/^<>=]', Operator), + (r'not|and|or', Operator.Word), + (r'"[^"\n]*.', String), + (r'\d+|[-+]?\d*\.\d*(e[-+]?\d+)?', Number.Float), + (r'[\(\),:;]', Punctuation), + (r'\w+[$%]?', Name), + ] + } + + def analyse_text(self, text): + # if it starts with a line number, it shouldn't be a "modern" Basic + # like VB.net + if re.match(r'\d+', text): + return True + + class MscgenLexer(RegexLexer): """ For `Mscgen <http://www.mcternan.me.uk/mscgen/>`_ files. @@ -3156,5 +3205,463 @@ class SourcePawnLexer(RegexLexer): if value in self.SM_TYPES: token = Keyword.Type elif value in self._functions: - tokens = Name.Builtin + token = Name.Builtin yield index, token, value + + +class PuppetLexer(RegexLexer): + """ + For `Puppet <http://puppetlabs.com/>`__ configuration DSL. + + *New in Pygments 1.6.* + """ + name = 'Puppet' + aliases = ['puppet'] + filenames = ['*.pp'] + + tokens = { + 'root': [ + include('comments'), + include('keywords'), + include('names'), + include('numbers'), + include('operators'), + include('strings'), + + (r'[]{}:(),;[]', Punctuation), + (r'[^\S\n]+', Text), + ], + + 'comments': [ + (r'\s*#.*$', Comment), + (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline), + ], + + 'operators': [ + (r'(=>|\?|<|>|=|\+|-|/|\*|~|!|\|)', Operator), + (r'(in|and|or|not)\b', Operator.Word), + ], + + 'names': [ + ('[a-zA-Z_][a-zA-Z0-9_]*', Name.Attribute), + (r'(\$\S+)(\[)(\S+)(\])', bygroups(Name.Variable, Punctuation, + String, Punctuation)), + (r'\$\S+', Name.Variable), + ], + + 'numbers': [ + # Copypasta from the Python lexer + (r'(\d+\.\d*|\d*\.\d+)([eE][+-]?[0-9]+)?j?', Number.Float), + (r'\d+[eE][+-]?[0-9]+j?', Number.Float), + (r'0[0-7]+j?', Number.Oct), + (r'0[xX][a-fA-F0-9]+', Number.Hex), + (r'\d+L', Number.Integer.Long), + (r'\d+j?', Number.Integer) + ], + + 'keywords': [ + # Left out 'group' and 'require' + # Since they're often used as attributes + (r'(?i)(absent|alert|alias|audit|augeas|before|case|check|class|' + r'computer|configured|contained|create_resources|crit|cron|debug|' + r'default|define|defined|directory|else|elsif|emerg|err|exec|' + r'extlookup|fail|false|file|filebucket|fqdn_rand|generate|host|if|' + r'import|include|info|inherits|inline_template|installed|' + r'interface|k5login|latest|link|loglevel|macauthorization|' + r'mailalias|maillist|mcx|md5|mount|mounted|nagios_command|' + r'nagios_contact|nagios_contactgroup|nagios_host|' + r'nagios_hostdependency|nagios_hostescalation|nagios_hostextinfo|' + r'nagios_hostgroup|nagios_service|nagios_servicedependency|' + r'nagios_serviceescalation|nagios_serviceextinfo|' + r'nagios_servicegroup|nagios_timeperiod|node|noop|notice|notify|' + r'package|present|purged|realize|regsubst|resources|role|router|' + r'running|schedule|scheduled_task|search|selboolean|selmodule|' + r'service|sha1|shellquote|split|sprintf|ssh_authorized_key|sshkey|' + r'stage|stopped|subscribe|tag|tagged|template|tidy|true|undef|' + r'unmounted|user|versioncmp|vlan|warning|yumrepo|zfs|zone|' + r'zpool)\b', Keyword), + ], + + 'strings': [ + (r'"([^"])*"', String), + (r'\'([^\'])*\'', String), + ], + + } + + +class NSISLexer(RegexLexer): + """ + For `NSIS <http://nsis.sourceforge.net/>`_ scripts. + + *New in Pygments 1.6.* + """ + name = 'NSIS' + aliases = ['nsis', 'nsi', 'nsh'] + filenames = ['*.nsi', '*.nsh'] + mimetypes = ['text/x-nsis'] + + flags = re.IGNORECASE + + tokens = { + 'root': [ + (r'[;\#].*\n', Comment), + (r"'.*'", String.Single), + (r'"', String.Double, 'str_double'), + (r'`', String.Backtick, 'str_backtick'), + include('macro'), + include('interpol'), + include('basic'), + (r'\$\{[a-z_|][\w|]*\}', Keyword.Pseudo), + (r'/[a-z_]\w*', Name.Attribute), + ('.', Text), + ], + 'basic': [ + (r'(\n)(Function)(\s+)([._a-z][.\w]*)\b', + bygroups(Text, Keyword, Text, Name.Function)), + (r'\b([_a-z]\w*)(::)([a-z][a-z0-9]*)\b', + bygroups(Keyword.Namespace, Punctuation, Name.Function)), + (r'\b([_a-z]\w*)(:)', bygroups(Name.Label, Punctuation)), + (r'(\b[ULS]|\B)([\!\<\>=]?=|\<\>?|\>)\B', Operator), + (r'[|+-]', Operator), + (r'\\', Punctuation), + (r'\b(Abort|Add(?:BrandingImage|Size)|' + r'Allow(?:RootDirInstall|SkipFiles)|AutoCloseWindow|' + r'BG(?:Font|Gradient)|BrandingText|BringToFront|Call(?:InstDLL)?|' + r'(?:Sub)?Caption|ChangeUI|CheckBitmap|ClearErrors|CompletedText|' + r'ComponentText|CopyFiles|CRCCheck|' + r'Create(?:Directory|Font|Shortcut)|Delete(?:INI(?:Sec|Str)|' + r'Reg(?:Key|Value))?|DetailPrint|DetailsButtonText|' + r'Dir(?:Show|Text|Var|Verify)|(?:Disabled|Enabled)Bitmap|' + r'EnableWindow|EnumReg(?:Key|Value)|Exch|Exec(?:Shell|Wait)?|' + r'ExpandEnvStrings|File(?:BufSize|Close|ErrorText|Open|' + r'Read(?:Byte)?|Seek|Write(?:Byte)?)?|' + r'Find(?:Close|First|Next|Window)|FlushINI|Function(?:End)?|' + r'Get(?:CurInstType|CurrentAddress|DlgItem|DLLVersion(?:Local)?|' + r'ErrorLevel|FileTime(?:Local)?|FullPathName|FunctionAddress|' + r'InstDirError|LabelAddress|TempFileName)|' + r'Goto|HideWindow|Icon|' + r'If(?:Abort|Errors|FileExists|RebootFlag|Silent)|' + r'InitPluginsDir|Install(?:ButtonText|Colors|Dir(?:RegKey)?)|' + r'Inst(?:ProgressFlags|Type(?:[GS]etText)?)|Int(?:CmpU?|Fmt|Op)|' + r'IsWindow|LangString(?:UP)?|' + r'License(?:BkColor|Data|ForceSelection|LangString|Text)|' + r'LoadLanguageFile|LockWindow|Log(?:Set|Text)|MessageBox|' + r'MiscButtonText|Name|Nop|OutFile|(?:Uninst)?Page(?:Ex(?:End)?)?|' + r'PluginDir|Pop|Push|Quit|Read(?:(?:Env|INI|Reg)Str|RegDWORD)|' + r'Reboot|(?:Un)?RegDLL|Rename|RequestExecutionLevel|ReserveFile|' + r'Return|RMDir|SearchPath|Section(?:Divider|End|' + r'(?:(?:Get|Set)(?:Flags|InstTypes|Size|Text))|Group(?:End)?|In)?|' + r'SendMessage|Set(?:AutoClose|BrandingImage|Compress(?:ionLevel|' + r'or(?:DictSize)?)?|CtlColors|CurInstType|DatablockOptimize|' + r'DateSave|Details(?:Print|View)|Error(?:s|Level)|FileAttributes|' + r'Font|OutPath|Overwrite|PluginUnload|RebootFlag|ShellVarContext|' + r'Silent|StaticBkColor)|' + r'Show(?:(?:I|Uni)nstDetails|Window)|Silent(?:Un)?Install|Sleep|' + r'SpaceTexts|Str(?:CmpS?|Cpy|Len)|SubSection(?:End)?|' + r'Uninstall(?:ButtonText|(?:Sub)?Caption|EXEName|Icon|Text)|' + r'UninstPage|Var|VI(?:AddVersionKey|ProductVersion)|WindowIcon|' + r'Write(?:INIStr|Reg(:?Bin|DWORD|(?:Expand)?Str)|Uninstaller)|' + r'XPStyle)\b', Keyword), + (r'\b(CUR|END|(?:FILE_ATTRIBUTE_)?' + r'(?:ARCHIVE|HIDDEN|NORMAL|OFFLINE|READONLY|SYSTEM|TEMPORARY)|' + r'HK(CC|CR|CU|DD|LM|PD|U)|' + r'HKEY_(?:CLASSES_ROOT|CURRENT_(?:CONFIG|USER)|DYN_DATA|' + r'LOCAL_MACHINE|PERFORMANCE_DATA|USERS)|' + r'ID(?:ABORT|CANCEL|IGNORE|NO|OK|RETRY|YES)|' + r'MB_(?:ABORTRETRYIGNORE|DEFBUTTON[1-4]|' + r'ICON(?:EXCLAMATION|INFORMATION|QUESTION|STOP)|' + r'OK(?:CANCEL)?|RETRYCANCEL|RIGHT|SETFOREGROUND|TOPMOST|USERICON|' + r'YESNO(?:CANCEL)?)|SET|SHCTX|' + r'SW_(?:HIDE|SHOW(?:MAXIMIZED|MINIMIZED|NORMAL))|' + r'admin|all|auto|both|bottom|bzip2|checkbox|colored|current|false|' + r'force|hide|highest|if(?:diff|newer)|lastused|leave|left|' + r'listonly|lzma|nevershow|none|normal|off|on|pop|push|' + r'radiobuttons|right|show|silent|silentlog|smooth|textonly|top|' + r'true|try|user|zlib)\b', Name.Constant), + ], + 'macro': [ + (r'\!(addincludedir(?:dir)?|addplugindir|appendfile|cd|define|' + r'delfilefile|echo(?:message)?|else|endif|error|execute|' + r'if(?:macro)?n?(?:def)?|include|insertmacro|macro(?:end)?|packhdr|' + r'search(?:parse|replace)|system|tempfilesymbol|undef|verbose|' + r'warning)\b', Comment.Preproc), + ], + 'interpol': [ + (r'\$(R?[0-9])', Name.Builtin.Pseudo), # registers + (r'\$(ADMINTOOLS|APPDATA|CDBURN_AREA|COOKIES|COMMONFILES(?:32|64)|' + r'DESKTOP|DOCUMENTS|EXE(?:DIR|FILE|PATH)|FAVORITES|FONTS|HISTORY|' + r'HWNDPARENT|INTERNET_CACHE|LOCALAPPDATA|MUSIC|NETHOOD|PICTURES|' + r'PLUGINSDIR|PRINTHOOD|PROFILE|PROGRAMFILES(?:32|64)|QUICKLAUNCH|' + r'RECENT|RESOURCES(?:_LOCALIZED)?|SENDTO|SM(?:PROGRAMS|STARTUP)|' + r'STARTMENU|SYSDIR|TEMP(?:LATES)?|VIDEOS|WINDIR|\{NSISDIR\})', + Name.Builtin), + (r'\$(CMDLINE|INSTDIR|OUTDIR|LANGUAGE)', Name.Variable.Global), + (r'\$[a-z_]\w*', Name.Variable), + ], + 'str_double': [ + (r'"', String, '#pop'), + (r'\$(\\[nrt"]|\$)', String.Escape), + include('interpol'), + (r'.', String.Double), + ], + 'str_backtick': [ + (r'`', String, '#pop'), + (r'\$(\\[nrt"]|\$)', String.Escape), + include('interpol'), + (r'.', String.Double), + ], + } + + +class RPMSpecLexer(RegexLexer): + """ + For RPM *.spec files + + *New in Pygments 1.6.* + """ + + name = 'RPMSpec' + aliases = ['spec'] + filenames = ['*.spec'] + mimetypes = ['text/x-rpm-spec'] + + _directives = ('(?:package|prep|build|install|clean|check|pre[a-z]*|' + 'post[a-z]*|trigger[a-z]*|files)') + + tokens = { + 'root': [ + (r'#.*\n', Comment), + include('basic'), + ], + 'description': [ + (r'^(%' + _directives + ')(.*)$', + bygroups(Name.Decorator, Text), '#pop'), + (r'\n', Text), + (r'.', Text), + ], + 'changelog': [ + (r'\*.*\n', Generic.Subheading), + (r'^(%' + _directives + ')(.*)$', + bygroups(Name.Decorator, Text), '#pop'), + (r'\n', Text), + (r'.', Text), + ], + 'string': [ + (r'"', String.Double, '#pop'), + (r'\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})', String.Escape), + include('interpol'), + (r'.', String.Double), + ], + 'basic': [ + include('macro'), + (r'(?i)^(Name|Version|Release|Epoch|Summary|Group|License|Packager|' + r'Vendor|Icon|URL|Distribution|Prefix|Patch[0-9]*|Source[0-9]*|' + r'Requires\(?[a-z]*\)?|[a-z]+Req|Obsoletes|Provides|Conflicts|' + r'Build[a-z]+|[a-z]+Arch|Auto[a-z]+)(:)(.*)$', + bygroups(Generic.Heading, Punctuation, using(this))), + (r'^%description', Name.Decorator, 'description'), + (r'^%changelog', Name.Decorator, 'changelog'), + (r'^(%' + _directives + ')(.*)$', bygroups(Name.Decorator, Text)), + (r'%(attr|defattr|dir|doc(?:dir)?|setup|config(?:ure)?|' + r'make(?:install)|ghost|patch[0-9]+|find_lang|exclude|verify)', + Keyword), + include('interpol'), + (r"'.*'", String.Single), + (r'"', String.Double, 'string'), + (r'.', Text), + ], + 'macro': [ + (r'%define.*\n', Comment.Preproc), + (r'%\{\!\?.*%define.*\}', Comment.Preproc), + (r'(%(?:if(?:n?arch)?|else(?:if)?|endif))(.*)$', + bygroups(Comment.Preproc, Text)), + ], + 'interpol': [ + (r'%\{?__[a-z_]+\}?', Name.Function), + (r'%\{?_([a-z_]+dir|[a-z_]+path|prefix)\}?', Keyword.Pseudo), + (r'%\{\?[A-Za-z0-9_]+\}', Name.Variable), + (r'\$\{?RPM_[A-Z0-9_]+\}?', Name.Variable.Global), + (r'%\{[a-zA-Z][a-zA-Z0-9_]+\}', Keyword.Constant), + ] + } + + +class AutoItLexer(RegexLexer): + """ + For `AutoIt <http://www.autoitscript.com/site/autoit/>`_ files. + + AutoIt is a freeware BASIC-like scripting language + designed for automating the Windows GUI and general scripting + + *New in Pygments 1.6.* + """ + name = 'AutoIt' + aliases = ['autoit', 'Autoit'] + filenames = ['*.au3'] + mimetypes = ['text/x-autoit'] + + # Keywords, functions, macros from au3.keywords.properties + # which can be found in AutoIt installed directory, e.g. + # c:\Program Files (x86)\AutoIt3\SciTE\au3.keywords.properties + + keywords = """\ + #include-once #include #endregion #forcedef #forceref #region + and byref case continueloop dim do else elseif endfunc endif + endselect exit exitloop for func global + if local next not or return select step + then to until wend while exit""".split() + + functions = """\ + abs acos adlibregister adlibunregister asc ascw asin assign atan + autoitsetoption autoitwingettitle autoitwinsettitle beep binary binarylen + binarymid binarytostring bitand bitnot bitor bitrotate bitshift bitxor + blockinput break call cdtray ceiling chr chrw clipget clipput consoleread + consolewrite consolewriteerror controlclick controlcommand controldisable + controlenable controlfocus controlgetfocus controlgethandle controlgetpos + controlgettext controlhide controllistview controlmove controlsend + controlsettext controlshow controltreeview cos dec dircopy dircreate + dirgetsize dirmove dirremove dllcall dllcalladdress dllcallbackfree + dllcallbackgetptr dllcallbackregister dllclose dllopen dllstructcreate + dllstructgetdata dllstructgetptr dllstructgetsize dllstructsetdata + drivegetdrive drivegetfilesystem drivegetlabel drivegetserial drivegettype + drivemapadd drivemapdel drivemapget drivesetlabel drivespacefree + drivespacetotal drivestatus envget envset envupdate eval execute exp + filechangedir fileclose filecopy filecreatentfslink filecreateshortcut + filedelete fileexists filefindfirstfile filefindnextfile fileflush + filegetattrib filegetencoding filegetlongname filegetpos filegetshortcut + filegetshortname filegetsize filegettime filegetversion fileinstall filemove + fileopen fileopendialog fileread filereadline filerecycle filerecycleempty + filesavedialog fileselectfolder filesetattrib filesetpos filesettime + filewrite filewriteline floor ftpsetproxy guicreate guictrlcreateavi + guictrlcreatebutton guictrlcreatecheckbox guictrlcreatecombo + guictrlcreatecontextmenu guictrlcreatedate guictrlcreatedummy + guictrlcreateedit guictrlcreategraphic guictrlcreategroup guictrlcreateicon + guictrlcreateinput guictrlcreatelabel guictrlcreatelist + guictrlcreatelistview guictrlcreatelistviewitem guictrlcreatemenu + guictrlcreatemenuitem guictrlcreatemonthcal guictrlcreateobj + guictrlcreatepic guictrlcreateprogress guictrlcreateradio + guictrlcreateslider guictrlcreatetab guictrlcreatetabitem + guictrlcreatetreeview guictrlcreatetreeviewitem guictrlcreateupdown + guictrldelete guictrlgethandle guictrlgetstate guictrlread guictrlrecvmsg + guictrlregisterlistviewsort guictrlsendmsg guictrlsendtodummy + guictrlsetbkcolor guictrlsetcolor guictrlsetcursor guictrlsetdata + guictrlsetdefbkcolor guictrlsetdefcolor guictrlsetfont guictrlsetgraphic + guictrlsetimage guictrlsetlimit guictrlsetonevent guictrlsetpos + guictrlsetresizing guictrlsetstate guictrlsetstyle guictrlsettip guidelete + guigetcursorinfo guigetmsg guigetstyle guiregistermsg guisetaccelerators + guisetbkcolor guisetcoord guisetcursor guisetfont guisethelp guiseticon + guisetonevent guisetstate guisetstyle guistartgroup guiswitch hex hotkeyset + httpsetproxy httpsetuseragent hwnd inetclose inetget inetgetinfo inetgetsize + inetread inidelete iniread inireadsection inireadsectionnames + inirenamesection iniwrite iniwritesection inputbox int isadmin isarray + isbinary isbool isdeclared isdllstruct isfloat ishwnd isint iskeyword + isnumber isobj isptr isstring log memgetstats mod mouseclick mouseclickdrag + mousedown mousegetcursor mousegetpos mousemove mouseup mousewheel msgbox + number objcreate objcreateinterface objevent objevent objget objname + onautoitexitregister onautoitexitunregister opt ping pixelchecksum + pixelgetcolor pixelsearch pluginclose pluginopen processclose processexists + processgetstats processlist processsetpriority processwait processwaitclose + progressoff progresson progressset ptr random regdelete regenumkey + regenumval regread regwrite round run runas runaswait runwait send + sendkeepactive seterror setextended shellexecute shellexecutewait shutdown + sin sleep soundplay soundsetwavevolume splashimageon splashoff splashtexton + sqrt srandom statusbargettext stderrread stdinwrite stdioclose stdoutread + string stringaddcr stringcompare stringformat stringfromasciiarray + stringinstr stringisalnum stringisalpha stringisascii stringisdigit + stringisfloat stringisint stringislower stringisspace stringisupper + stringisxdigit stringleft stringlen stringlower stringmid stringregexp + stringregexpreplace stringreplace stringright stringsplit stringstripcr + stringstripws stringtoasciiarray stringtobinary stringtrimleft + stringtrimright stringupper tan tcpaccept tcpclosesocket tcpconnect + tcplisten tcpnametoip tcprecv tcpsend tcpshutdown tcpstartup timerdiff + timerinit tooltip traycreateitem traycreatemenu traygetmsg trayitemdelete + trayitemgethandle trayitemgetstate trayitemgettext trayitemsetonevent + trayitemsetstate trayitemsettext traysetclick trayseticon traysetonevent + traysetpauseicon traysetstate traysettooltip traytip ubound udpbind + udpclosesocket udpopen udprecv udpsend udpshutdown udpstartup vargettype + winactivate winactive winclose winexists winflash wingetcaretpos + wingetclasslist wingetclientsize wingethandle wingetpos wingetprocess + wingetstate wingettext wingettitle winkill winlist winmenuselectitem + winminimizeall winminimizeallundo winmove winsetontop winsetstate + winsettitle winsettrans winwait winwaitactive winwaitclose + winwaitnotactive""".split() + + macros = """\ + @appdatacommondir @appdatadir @autoitexe @autoitpid @autoitversion + @autoitx64 @com_eventobj @commonfilesdir @compiled @computername @comspec + @cpuarch @cr @crlf @desktopcommondir @desktopdepth @desktopdir + @desktopheight @desktoprefresh @desktopwidth @documentscommondir @error + @exitcode @exitmethod @extended @favoritescommondir @favoritesdir + @gui_ctrlhandle @gui_ctrlid @gui_dragfile @gui_dragid @gui_dropid + @gui_winhandle @homedrive @homepath @homeshare @hotkeypressed @hour + @ipaddress1 @ipaddress2 @ipaddress3 @ipaddress4 @kblayout @lf + @logondnsdomain @logondomain @logonserver @mday @min @mon @msec @muilang + @mydocumentsdir @numparams @osarch @osbuild @oslang @osservicepack @ostype + @osversion @programfilesdir @programscommondir @programsdir @scriptdir + @scriptfullpath @scriptlinenumber @scriptname @sec @startmenucommondir + @startmenudir @startupcommondir @startupdir @sw_disable @sw_enable @sw_hide + @sw_lock @sw_maximize @sw_minimize @sw_restore @sw_show @sw_showdefault + @sw_showmaximized @sw_showminimized @sw_showminnoactive @sw_showna + @sw_shownoactivate @sw_shownormal @sw_unlock @systemdir @tab @tempdir + @tray_id @trayiconflashing @trayiconvisible @username @userprofiledir @wday + @windowsdir @workingdir @yday @year""".split() + + tokens = { + 'root': [ + (r';.*\n', Comment.Single), + (r'(#comments-start|#cs).*?(#comments-end|#ce)', Comment.Multiline), + (r'[\[\]{}(),;]', Punctuation), + (r'(and|or|not)\b', Operator.Word), + (r'[\$|@][a-zA-Z_][a-zA-Z0-9_]*', Name.Variable), + (r'!=|==|:=|\.=|<<|>>|[-~+/*%=<>&^|?:!.]', Operator), + include('commands'), + include('labels'), + include('builtInFunctions'), + include('builtInMarcros'), + (r'"', String, combined('stringescape', 'dqs')), + include('numbers'), + (r'[a-zA-Z_#@$][a-zA-Z0-9_#@$]*', Name), + (r'\\|\'', Text), + (r'\`([\,\%\`abfnrtv\-\+;])', String.Escape), + (r'_\n', Text), # Line continuation + include('garbage'), + ], + 'commands': [ + (r'(?i)(\s*)(%s)\b' % '|'.join(keywords), + bygroups(Text, Name.Builtin)), + ], + 'builtInFunctions': [ + (r'(?i)(%s)\b' % '|'.join(functions), + Name.Function), + ], + 'builtInMarcros': [ + (r'(?i)(%s)\b' % '|'.join(macros), + Name.Variable.Global), + ], + 'labels': [ + # sendkeys + (r'(^\s*)({\S+?})', bygroups(Text, Name.Label)), + ], + 'numbers': [ + (r'(\d+\.\d*|\d*\.\d+)([eE][+-]?[0-9]+)?', Number.Float), + (r'\d+[eE][+-]?[0-9]+', Number.Float), + (r'0\d+', Number.Oct), + (r'0[xX][a-fA-F0-9]+', Number.Hex), + (r'\d+L', Number.Integer.Long), + (r'\d+', Number.Integer) + ], + 'stringescape': [ + (r'\"\"|\`([\,\%\`abfnrtv])', String.Escape), + ], + 'strings': [ + (r'[^"\n]+', String), + ], + 'dqs': [ + (r'"', String, '#pop'), + include('strings') + ], + 'garbage': [ + (r'[^\S\n]', Text), + ], + } diff --git a/pygments/lexers/parsers.py b/pygments/lexers/parsers.py index 2b5f954f..c1ad710f 100644 --- a/pygments/lexers/parsers.py +++ b/pygments/lexers/parsers.py @@ -5,7 +5,7 @@ Lexers for parser generators. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -28,7 +28,8 @@ __all__ = ['RagelLexer', 'RagelEmbeddedLexer', 'RagelCLexer', 'RagelDLexer', 'AntlrPerlLexer', 'AntlrRubyLexer', 'AntlrCppLexer', #'AntlrCLexer', 'AntlrCSharpLexer', 'AntlrObjectiveCLexer', - 'AntlrJavaLexer', "AntlrActionScriptLexer"] + 'AntlrJavaLexer', "AntlrActionScriptLexer", + 'TreetopLexer'] class RagelLexer(RegexLexer): @@ -693,3 +694,85 @@ class AntlrActionScriptLexer(DelegatingLexer): def analyse_text(text): return AntlrLexer.analyse_text(text) and \ re.search(r'^\s*language\s*=\s*ActionScript\s*;', text, re.M) + +class TreetopBaseLexer(RegexLexer): + """ + A base lexer for `Treetop <http://treetop.rubyforge.org/>`_ grammars. + Not for direct use; use TreetopLexer instead. + + *New in Pygments 1.6.* + """ + + tokens = { + 'root': [ + include('space'), + (r'require[ \t]+[^\n\r]+[\n\r]', Other), + (r'module\b', Keyword.Namespace, 'module'), + (r'grammar\b', Keyword, 'grammar'), + ], + 'module': [ + include('space'), + include('end'), + (r'module\b', Keyword, '#push'), + (r'grammar\b', Keyword, 'grammar'), + (r'[A-Z][A-Za-z_0-9]*(?:::[A-Z][A-Za-z_0-9]*)*', Name.Namespace), + ], + 'grammar': [ + include('space'), + include('end'), + (r'rule\b', Keyword, 'rule'), + (r'include\b', Keyword, 'include'), + (r'[A-Z][A-Za-z_0-9]*', Name), + ], + 'include': [ + include('space'), + (r'[A-Z][A-Za-z_0-9]*(?:::[A-Z][A-Za-z_0-9]*)*', Name.Class, '#pop'), + ], + 'rule': [ + include('space'), + include('end'), + (r'"(\\\\|\\"|[^"])*"', String.Double), + (r"'(\\\\|\\'|[^'])*'", String.Single), + (r'([A-Za-z_][A-Za-z_0-9]*)(:)', bygroups(Name.Label, Punctuation)), + (r'[A-Za-z_][A-Za-z_0-9]*', Name), + (r'[()]', Punctuation), + (r'[?+*/&!~]', Operator), + (r'\[(?:\\.|\[:\^?[a-z]+:\]|[^\\\]])+\]', String.Regex), + (r'([0-9]*)(\.\.)([0-9]*)', + bygroups(Number.Integer, Operator, Number.Integer)), + (r'(<)([^>]+)(>)', bygroups(Punctuation, Name.Class, Punctuation)), + (r'{', Punctuation, 'inline_module'), + (r'\.', String.Regex), + ], + 'inline_module': [ + (r'{', Other, 'ruby'), + (r'}', Punctuation, '#pop'), + (r'[^{}]+', Other), + ], + 'ruby': [ + (r'{', Other, '#push'), + (r'}', Other, '#pop'), + (r'[^{}]+', Other), + ], + 'space': [ + (r'[ \t\n\r]+', Whitespace), + (r'#[^\n]*', Comment.Single), + ], + 'end': [ + (r'end\b', Keyword, '#pop'), + ], + } + +class TreetopLexer(DelegatingLexer): + """ + A lexer for `Treetop <http://treetop.rubyforge.org/>`_ grammars. + + *New in Pygments 1.6.* + """ + + name = 'Treetop' + aliases = ['treetop'] + filenames = ['*.treetop', '*.tt'] + + def __init__(self, **options): + super(TreetopLexer, self).__init__(RubyLexer, TreetopBaseLexer, **options) diff --git a/pygments/lexers/shell.py b/pygments/lexers/shell.py index 52df5f00..b95faf93 100644 --- a/pygments/lexers/shell.py +++ b/pygments/lexers/shell.py @@ -5,7 +5,7 @@ Lexers for various shells. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -18,7 +18,7 @@ from pygments.util import shebang_matches __all__ = ['BashLexer', 'BashSessionLexer', 'TcshLexer', 'BatchLexer', - 'PowerShellLexer'] + 'PowerShellLexer', 'ShellSessionLexer'] line_re = re.compile('.*?\n') @@ -153,6 +153,52 @@ class BashSessionLexer(Lexer): yield pos+i, t, v +class ShellSessionLexer(Lexer): + """ + Lexer for shell sessions that works with different command prompts + + *New in Pygments 1.6.* + """ + + name = 'Shell Session' + aliases = ['shell-session'] + filenames = ['*.shell-session'] + mimetypes = ['application/x-sh-session'] + + def get_tokens_unprocessed(self, text): + bashlexer = BashLexer(**self.options) + + pos = 0 + curcode = '' + insertions = [] + + for match in line_re.finditer(text): + line = match.group() + m = re.match(r'^((?:\[?\S+@[^$#%]+)[$#%])(.*\n?)', line) + if m: + # To support output lexers (say diff output), the output + # needs to be broken by prompts whenever the output lexer + # changes. + if not insertions: + pos = match.start() + + insertions.append((len(curcode), + [(0, Generic.Prompt, m.group(1))])) + curcode += m.group(2) + else: + if insertions: + toks = bashlexer.get_tokens_unprocessed(curcode) + for i, t, v in do_insertions(insertions, toks): + yield pos+i, t, v + yield match.start(), Generic.Output, line + insertions = [] + curcode = '' + if insertions: + for i, t, v in do_insertions(insertions, + bashlexer.get_tokens_unprocessed(curcode)): + yield pos+i, t, v + + class BatchLexer(RegexLexer): """ Lexer for the DOS/Windows Batch file format. diff --git a/pygments/lexers/special.py b/pygments/lexers/special.py index 785ab73c..9b3cd508 100644 --- a/pygments/lexers/special.py +++ b/pygments/lexers/special.py @@ -5,7 +5,7 @@ Special lexers. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/sql.py b/pygments/lexers/sql.py index e7e02fce..dcfd8fa8 100644 --- a/pygments/lexers/sql.py +++ b/pygments/lexers/sql.py @@ -34,7 +34,7 @@ The ``tests/examplefiles`` contains a few test files with data to be parsed by these lexers. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/lexers/templates.py b/pygments/lexers/templates.py index 6c55edbf..b3e70d05 100644 --- a/pygments/lexers/templates.py +++ b/pygments/lexers/templates.py @@ -5,7 +5,7 @@ Lexers for various template engines' markup. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -224,12 +224,14 @@ class VelocityLexer(RegexLexer): 'variable': [ (identifier, Name.Variable), (r'\(', Punctuation, 'funcparams'), - (r'(\.)(' + identifier + r')', bygroups(Punctuation, Name.Variable), '#push'), + (r'(\.)(' + identifier + r')', + bygroups(Punctuation, Name.Variable), '#push'), (r'\}', Punctuation, '#pop'), (r'', Other, '#pop') ], 'directiveparams': [ - (r'(&&|\|\||==?|!=?|[-<>+*%&\|\^/])|\b(eq|ne|gt|lt|ge|le|not|in)\b', Operator), + (r'(&&|\|\||==?|!=?|[-<>+*%&\|\^/])|\b(eq|ne|gt|lt|ge|le|not|in)\b', + Operator), (r'\[', Operator, 'rangeoperator'), (r'\b' + identifier + r'\b', Name.Function), include('funcparams') @@ -261,7 +263,8 @@ class VelocityLexer(RegexLexer): rv += 0.15 if re.search(r'#\{?foreach\}?\(.+?\).*?#\{?end\}?', text): rv += 0.15 - if re.search(r'\$\{?[a-zA-Z_][a-zA-Z0-9_]*(\([^)]*\))?(\.[a-zA-Z0-9_]+(\([^)]*\))?)*\}?', text): + if re.search(r'\$\{?[a-zA-Z_][a-zA-Z0-9_]*(\([^)]*\))?' + r'(\.[a-zA-Z0-9_]+(\([^)]*\))?)*\}?', text): rv += 0.01 return rv @@ -1638,6 +1641,8 @@ class LassoHtmlLexer(DelegatingLexer): `HtmlLexer`. Nested JavaScript and CSS is also highlighted. + + *New in Pygments 1.6.* """ name = 'HTML+Lasso' @@ -1649,7 +1654,6 @@ class LassoHtmlLexer(DelegatingLexer): 'application/x-httpd-lasso[89]'] def __init__(self, **options): - options['requiredelimiters'] = True super(LassoHtmlLexer, self).__init__(HtmlLexer, LassoLexer, **options) def analyse_text(text): @@ -1665,6 +1669,8 @@ class LassoXmlLexer(DelegatingLexer): """ Subclass of the `LassoLexer` which highlights unhandled data with the `XmlLexer`. + + *New in Pygments 1.6.* """ name = 'XML+Lasso' @@ -1674,7 +1680,6 @@ class LassoXmlLexer(DelegatingLexer): mimetypes = ['application/xml+lasso'] def __init__(self, **options): - options['requiredelimiters'] = True super(LassoXmlLexer, self).__init__(XmlLexer, LassoLexer, **options) def analyse_text(text): @@ -1688,6 +1693,8 @@ class LassoCssLexer(DelegatingLexer): """ Subclass of the `LassoLexer` which highlights unhandled data with the `CssLexer`. + + *New in Pygments 1.6.* """ name = 'CSS+Lasso' @@ -1712,6 +1719,8 @@ class LassoJavascriptLexer(DelegatingLexer): """ Subclass of the `LassoLexer` which highlights unhandled data with the `JavascriptLexer`. + + *New in Pygments 1.6.* """ name = 'JavaScript+Lasso' diff --git a/pygments/lexers/text.py b/pygments/lexers/text.py index cff4ddd0..5e340893 100644 --- a/pygments/lexers/text.py +++ b/pygments/lexers/text.py @@ -5,7 +5,7 @@ Lexers for non-source code file types. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -25,7 +25,7 @@ __all__ = ['IniLexer', 'PropertiesLexer', 'SourcesListLexer', 'BaseMakefileLexer 'RstLexer', 'VimLexer', 'GettextLexer', 'SquidConfLexer', 'DebianControlLexer', 'DarcsPatchLexer', 'YamlLexer', 'LighttpdConfLexer', 'NginxConfLexer', 'CMakeLexer', 'HttpLexer', - 'PyPyLogLexer'] + 'PyPyLogLexer', 'RegeditLexer', 'HxmlLexer'] class IniLexer(RegexLexer): @@ -41,7 +41,7 @@ class IniLexer(RegexLexer): tokens = { 'root': [ (r'\s+', Text), - (r'[;#].*?$', Comment), + (r'[;#].*', Comment.Single), (r'\[.*?\]$', Keyword), (r'(.*?)([ \t]*)(=)([ \t]*)(.*(?:\n[ \t].+)*)', bygroups(Name.Attribute, Text, Operator, Text, String)) @@ -55,6 +55,49 @@ class IniLexer(RegexLexer): return text[0] == '[' and text[npos-1] == ']' +class RegeditLexer(RegexLexer): + """ + Lexer for `Windows Registry + <http://en.wikipedia.org/wiki/Windows_Registry#.REG_files>`_ files produced + by regedit. + + *New in Pygments 1.6.* + """ + + name = 'reg' + aliases = ['registry'] + filenames = ['*.reg'] + mimetypes = ['text/x-windows-registry'] + + tokens = { + 'root': [ + (r'Windows Registry Editor.*', Text), + (r'\s+', Text), + (r'[;#].*', Comment.Single), + (r'(\[)(-?)(HKEY_[A-Z_]+)(.*?\])$', + bygroups(Keyword, Operator, Name.Builtin, Keyword)), + # String keys, which obey somewhat normal escaping + (r'("(?:\\"|\\\\|[^"])+")([ \t]*)(=)([ \t]*)', + bygroups(Name.Attribute, Text, Operator, Text), + 'value'), + # Bare keys (includes @) + (r'(.*?)([ \t]*)(=)([ \t]*)', + bygroups(Name.Attribute, Text, Operator, Text), + 'value'), + ], + 'value': [ + (r'-', Operator, '#pop'), # delete value + (r'(dword|hex(?:\([0-9a-fA-F]\))?)(:)([0-9a-fA-F,]+)', + bygroups(Name.Variable, Punctuation, Number), '#pop'), + # As far as I know, .reg files do not support line continuation. + (r'.*', String, '#pop'), + ] + } + + def analyse_text(text): + return text.startswith('Windows Registry Editor') + + class PropertiesLexer(RegexLexer): """ Lexer for configuration files in Java's properties format. @@ -573,7 +616,7 @@ class MoinWikiLexer(RegexLexer): (r'(\'\'\'?|\|\||`|__|~~|\^|,,|::)', Comment), # Formatting # Lists (r'^( +)([.*-])( )', bygroups(Text, Name.Builtin, Text)), - (r'^( +)([a-zivx]{1,5}\.)( )', bygroups(Text, Name.Builtin, Text)), + (r'^( +)([a-z]{1,5}\.)( )', bygroups(Text, Name.Builtin, Text)), # Other Formatting (r'\[\[\w+.*?\]\]', Keyword), # Macro (r'(\[[^\s\]]+)(\s+[^\]]+?)?(\])', @@ -1706,8 +1749,8 @@ class PyPyLogLexer(RegexLexer): ], "jit-log": [ (r"\[\w+\] jit-log-.*?}$", Keyword, "#pop"), - (r"^\+\d+: ", Comment), + (r"--end of the loop--", Comment), (r"[ifp]\d+", Name), (r"ptr\d+", Name), (r"(\()(\w+(?:\.\w+)?)(\))", @@ -1717,7 +1760,7 @@ class PyPyLogLexer(RegexLexer): (r"-?\d+", Number.Integer), (r"'.*'", String), (r"(None|descr|ConstClass|ConstPtr|TargetToken)", Name), - (r"<.*?>", Name.Builtin), + (r"<.*?>+", Name.Builtin), (r"(label|debug_merge_point|jump|finish)", Name.Class), (r"(int_add_ovf|int_add|int_sub_ovf|int_sub|int_mul_ovf|int_mul|" r"int_floordiv|int_mod|int_lshift|int_rshift|int_and|int_or|" @@ -1757,3 +1800,44 @@ class PyPyLogLexer(RegexLexer): (r"#.*?$", Comment), ], } + + +class HxmlLexer(RegexLexer): + """ + Lexer for `haXe build <http://haxe.org/doc/compiler>`_ files. + + *New in Pygments 1.6.* + """ + name = 'Hxml' + aliases = ['haxeml', 'hxml'] + filenames = ['*.hxml'] + + tokens = { + 'root': [ + # Seperator + (r'(--)(next)', bygroups(Punctuation, Generic.Heading)), + # Compiler switches with one dash + (r'(-)(prompt|debug|v)', bygroups(Punctuation, Keyword.Keyword)), + # Compilerswitches with two dashes + (r'(--)(neko-source|flash-strict|flash-use-stage|no-opt|no-traces|' + r'no-inline|times|no-output)', bygroups(Punctuation, Keyword)), + # Targets and other options that take an argument + (r'(-)(cpp|js|neko|x|as3|swf9?|swf-lib|php|xml|main|lib|D|resource|' + r'cp|cmd)( +)(.+)', + bygroups(Punctuation, Keyword, Whitespace, String)), + # Options that take only numerical arguments + (r'(-)(swf-version)( +)(\d+)', + bygroups(Punctuation, Keyword, Number.Integer)), + # An Option that defines the size, the fps and the background + # color of an flash movie + (r'(-)(swf-header)( +)(\d+)(:)(\d+)(:)(\d+)(:)([A-Fa-f0-9]{6})', + bygroups(Punctuation, Keyword, Whitespace, Number.Integer, + Punctuation, Number.Integer, Punctuation, Number.Integer, + Punctuation, Number.Hex)), + # options with two dashes that takes arguments + (r'(--)(js-namespace|php-front|php-lib|remap|gen-hx-classes)( +)' + r'(.+)', bygroups(Punctuation, Keyword, Whitespace, String)), + # Single line comment, multiline ones are not allowed. + (r'#.*', Comment.Single) + ] + } diff --git a/pygments/lexers/web.py b/pygments/lexers/web.py index 4ce27a94..fb23a4f6 100644 --- a/pygments/lexers/web.py +++ b/pygments/lexers/web.py @@ -5,7 +5,7 @@ Lexers for web-related languages and markup. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -27,7 +27,7 @@ __all__ = ['HtmlLexer', 'XmlLexer', 'JavascriptLexer', 'JsonLexer', 'CssLexer', 'MxmlLexer', 'HaxeLexer', 'HamlLexer', 'SassLexer', 'ScssLexer', 'ObjectiveJLexer', 'CoffeeScriptLexer', 'LiveScriptLexer', 'DuelLexer', 'ScamlLexer', 'JadeLexer', 'XQueryLexer', - 'DtdLexer', 'DartLexer', 'LassoLexer'] + 'DtdLexer', 'DartLexer', 'LassoLexer', 'QmlLexer', 'TypeScriptLexer'] class JavascriptLexer(RegexLexer): @@ -791,7 +791,7 @@ class PhpLexer(RegexLexer): name = 'PHP' aliases = ['php', 'php3', 'php4', 'php5'] - filenames = ['*.php', '*.php[345]'] + filenames = ['*.php', '*.php[345]', '*.inc'] mimetypes = ['text/x-php'] flags = re.IGNORECASE | re.DOTALL | re.MULTILINE @@ -969,7 +969,8 @@ class DtdLexer(RegexLexer): 'attlist': [ include('common'), - (r'CDATA|IDREFS|IDREF|ID|NMTOKENS|NMTOKEN|ENTITIES|ENTITY|NOTATION', Keyword.Constant), + (r'CDATA|IDREFS|IDREF|ID|NMTOKENS|NMTOKEN|ENTITIES|ENTITY|NOTATION', + Keyword.Constant), (r'#REQUIRED|#IMPLIED|#FIXED', Keyword.Constant), (r'xml:space|xml:lang', Keyword.Reserved), (r'[^>\s\|()?+*,]+', Name.Attribute), @@ -2264,7 +2265,7 @@ class ScssLexer(RegexLexer): (r'(@include)( [\w-]+)', bygroups(Keyword, Name.Decorator), 'value'), (r'@extend', Keyword, 'selector'), (r'@[a-z0-9_-]+', Keyword, 'selector'), - (r'(\$[\w-]\w*)([ \t]*:)', bygroups(Name.Variable, Operator), 'value'), + (r'(\$[\w-]*\w)([ \t]*:)', bygroups(Name.Variable, Operator), 'value'), (r'(?=[^;{}][;}])', Name.Attribute, 'attr'), (r'(?=[^;{}:]+:[^a-z])', Name.Attribute, 'attr'), (r'', Text, 'selector'), @@ -3438,27 +3439,25 @@ class DartLexer(RegexLexer): tokens = { 'root': [ + include('string_literal'), (r'#!(.*?)$', Comment.Preproc), - (r'(#)(import|library|source)', bygroups(Text, Keyword)), + (r'\b(import|export)\b', Keyword, 'import_decl'), + (r'\b(library|source|part of|part)\b', Keyword), (r'[^\S\n]+', Text), (r'//.*?\n', Comment.Single), (r'/\*.*?\*/', Comment.Multiline), - (r'(class|interface)(\s+)', + (r'\b(class)\b(\s+)', bygroups(Keyword.Declaration, Text), 'class'), - (r'(assert|break|case|catch|continue|default|do|else|finally|for|' + (r'\b(assert|break|case|catch|continue|default|do|else|finally|for|' r'if|in|is|new|return|super|switch|this|throw|try|while)\b', Keyword), - (r'(abstract|const|extends|factory|final|get|implements|' + (r'\b(abstract|const|extends|factory|final|get|implements|' r'native|operator|set|static|typedef|var)\b', Keyword.Declaration), - (r'(bool|double|Dynamic|int|num|Object|String|void)', Keyword.Type), - (r'(false|null|true)', Keyword.Constant), - (r'@"(\\\\|\\"|[^"])*"', String.Double), # raw string - (r"@'(\\\\|\\'|[^'])*'", String.Single), # raw string - (r'"', String.Double, 'string_double'), - (r"'", String.Single, 'string_single'), + (r'\b(bool|double|Dynamic|int|num|Object|String|void)\b', Keyword.Type), + (r'\b(false|null|true)\b', Keyword.Constant), + (r'[~!%^&*+=|?:<>/-]|as', Operator), (r'[a-zA-Z_$][a-zA-Z0-9_]*:', Name.Label), (r'[a-zA-Z_$][a-zA-Z0-9_]*', Name), - (r'[~!%^&*+=|?:<>/-]', Operator), (r'[(){}\[\],.;]', Punctuation), (r'0[xX][0-9a-fA-F]+', Number.Hex), # DIGIT+ (‘.’ DIGIT*)? EXPONENT? @@ -3470,21 +3469,132 @@ class DartLexer(RegexLexer): 'class': [ (r'[a-zA-Z_$][a-zA-Z0-9_]*', Name.Class, '#pop') ], - 'string_double': [ - (r'"', String.Double, '#pop'), - (r'[^"$]+', String.Double), + 'import_decl': [ + include('string_literal'), + (r'\s+', Text), + (r'\b(as|show|hide)\b', Keyword), + (r'[a-zA-Z_$][a-zA-Z0-9_]*', Name), + (r'\,', Punctuation), + (r'\;', Punctuation, '#pop') + ], + 'string_literal': [ + # Raw strings. + (r'r"""([\s|\S]*?)"""', String.Double), + (r"r'''([\s|\S]*?)'''", String.Single), + (r'r"(.*?)"', String.Double), + (r"r'(.*?)'", String.Single), + # Normal Strings. + (r'"""', String.Double, 'string_double_multiline'), + (r"'''", String.Single, 'string_single_multiline'), + (r'"', String.Double, 'string_double'), + (r"'", String.Single, 'string_single') + ], + 'string_common': [ + (r"\\(x[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}|u\{[0-9A-Fa-f]*\}|[a-z\'\"$\\])", + String.Escape), (r'(\$)([a-zA-Z_][a-zA-Z0-9_]*)', bygroups(String.Interpol, Name)), (r'(\$\{)(.*?)(\})', - bygroups(String.Interpol, using(this), String.Interpol)), + bygroups(String.Interpol, using(this), String.Interpol)) + ], + 'string_double': [ + (r'"', String.Double, '#pop'), + (r'[^\"$\\\n]+', String.Double), + include('string_common'), (r'\$+', String.Double) ], + 'string_double_multiline': [ + (r'"""', String.Double, '#pop'), + (r'[^\"$\\]+', String.Double), + include('string_common'), + (r'(\$|\")+', String.Double) + ], 'string_single': [ (r"'", String.Single, '#pop'), - (r"[^'$]+", String.Single), - (r'(\$)([a-zA-Z_][a-zA-Z0-9_]*)', bygroups(String.Interpol, Name)), - (r'(\$\{)(.*?)(\})', - bygroups(String.Interpol, using(this), String.Interpol)), + (r"[^\'$\\\n]+", String.Single), + include('string_common'), (r'\$+', String.Single) + ], + 'string_single_multiline': [ + (r"'''", String.Single, '#pop'), + (r'[^\'$\\]+', String.Single), + include('string_common'), + (r'(\$|\')+', String.Single) + ] + } + + +class TypeScriptLexer(RegexLexer): + """ + For `TypeScript <http://www.python.org>`_ source code. + + *New in Pygments 1.6.* + """ + + name = 'TypeScript' + aliases = ['ts'] + filenames = ['*.ts'] + mimetypes = ['text/x-typescript'] + + flags = re.DOTALL + tokens = { + 'commentsandwhitespace': [ + (r'\s+', Text), + (r'<!--', Comment), + (r'//.*?\n', Comment.Single), + (r'/\*.*?\*/', Comment.Multiline) + ], + 'slashstartsregex': [ + include('commentsandwhitespace'), + (r'/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/' + r'([gim]+\b|\B)', String.Regex, '#pop'), + (r'(?=/)', Text, ('#pop', 'badregex')), + (r'', Text, '#pop') + ], + 'badregex': [ + (r'\n', Text, '#pop') + ], + 'root': [ + (r'^(?=\s|/|<!--)', Text, 'slashstartsregex'), + include('commentsandwhitespace'), + (r'\+\+|--|~|&&|\?|:|\|\||\\(?=\n)|' + r'(<<|>>>?|==?|!=?|[-<>+*%&\|\^/])=?', Operator, 'slashstartsregex'), + (r'[{(\[;,]', Punctuation, 'slashstartsregex'), + (r'[})\].]', Punctuation), + (r'(for|in|while|do|break|return|continue|switch|case|default|if|else|' + r'throw|try|catch|finally|new|delete|typeof|instanceof|void|' + r'this)\b', Keyword, 'slashstartsregex'), + (r'(var|let|with|function)\b', Keyword.Declaration, 'slashstartsregex'), + (r'(abstract|boolean|byte|char|class|const|debugger|double|enum|export|' + r'extends|final|float|goto|implements|import|int|interface|long|native|' + r'package|private|protected|public|short|static|super|synchronized|throws|' + r'transient|volatile)\b', Keyword.Reserved), + (r'(true|false|null|NaN|Infinity|undefined)\b', Keyword.Constant), + (r'(Array|Boolean|Date|Error|Function|Math|netscape|' + r'Number|Object|Packages|RegExp|String|sun|decodeURI|' + r'decodeURIComponent|encodeURI|encodeURIComponent|' + r'Error|eval|isFinite|isNaN|parseFloat|parseInt|document|this|' + r'window)\b', Name.Builtin), + # Match stuff like: module name {...} + (r'\b(module)(\s*)(\s*[a-zA-Z0-9_?.$][\w?.$]*)(\s*)', + bygroups(Keyword.Reserved, Text, Name.Other, Text), 'slashstartsregex'), + # Match variable type keywords + (r'\b(string|bool|number)\b', Keyword.Type), + # Match stuff like: constructor + (r'\b(constructor|declare|interface|as|AS)\b', Keyword.Reserved), + # Match stuff like: super(argument, list) + (r'(super)(\s*)(\([a-zA-Z0-9,_?.$\s]+\s*\))', + bygroups(Keyword.Reserved, Text), 'slashstartsregex'), + # Match stuff like: function() {...} + (r'([a-zA-Z_?.$][\w?.$]*)\(\) \{', Name.Other, 'slashstartsregex'), + # Match stuff like: (function: return type) + (r'([a-zA-Z0-9_?.$][\w?.$]*)(\s*:\s*)([a-zA-Z0-9_?.$][\w?.$]*)', + bygroups(Name.Other, Text, Keyword.Type)), + (r'[$a-zA-Z_][a-zA-Z0-9_]*', Name.Other), + (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), + (r'0x[0-9a-fA-F]+', Number.Hex), + (r'[0-9]+', Number.Integer), + (r'"(\\\\|\\"|[^"])*"', String.Double), + (r"'(\\\\|\\'|[^'])*'", String.Single), ] } @@ -3517,19 +3627,14 @@ class LassoLexer(RegexLexer): tokens = { 'root': [ (r'^#!.+lasso9\b', Comment.Preproc, 'lasso'), - (r'\s+', Other), (r'\[no_square_brackets\]', Comment.Preproc, 'nosquarebrackets'), (r'\[noprocess\]', Comment.Preproc, ('delimiters', 'noprocess')), (r'\[', Comment.Preproc, ('delimiters', 'squarebrackets')), (r'<\?(LassoScript|lasso|=)', Comment.Preproc, ('delimiters', 'anglebrackets')), (r'<', Other, 'delimiters'), - include('lasso'), - ], - 'nosquarebrackets': [ - (r'<\?(LassoScript|lasso|=)', Comment.Preproc, 'anglebrackets'), - (r'<', Other), - (r'[^<]+', Other), + (r'\s+', Other), + (r'', Other, ('delimiters', 'lassofile')), ], 'delimiters': [ (r'\[no_square_brackets\]', Comment.Preproc, 'nosquarebrackets'), @@ -3539,6 +3644,11 @@ class LassoLexer(RegexLexer): (r'<', Other), (r'[^[<]+', Other), ], + 'nosquarebrackets': [ + (r'<\?(LassoScript|lasso|=)', Comment.Preproc, 'anglebrackets'), + (r'<', Other), + (r'[^<]+', Other), + ], 'noprocess': [ (r'\[/noprocess\]', Comment.Preproc, '#pop'), (r'\[', Other), @@ -3552,12 +3662,20 @@ class LassoLexer(RegexLexer): (r'\?>', Comment.Preproc, '#pop'), include('lasso'), ], - 'lasso': [ - # whitespace/comments + 'lassofile': [ + (r'\]', Comment.Preproc, '#pop'), + (r'\?>', Comment.Preproc, '#pop'), + include('lasso'), + ], + 'whitespacecomments': [ (r'\s+', Text), (r'//.*?\n', Comment.Single), (r'/\*\*!.*?\*/', String.Doc), (r'/\*.*?\*/', Comment.Multiline), + ], + 'lasso': [ + # whitespace/comments + include('whitespacecomments'), # literals (r'\d*\.\d+(e[+-]?\d+)?', Number.Float), @@ -3600,10 +3718,9 @@ class LassoLexer(RegexLexer): (r'(define)(\s+)([a-z_][\w.]*)', bygroups(Keyword.Declaration, Text, Name.Function), 'signature'), - (r'(public|protected|private|provide)(\s+)([a-z_][\w.]*=?|' - r'[-+*/%<>]|==)(\s*)(\()', - bygroups(Keyword, Text, Name.Function, Text, Punctuation), - ('signature', 'parameter')), + (r'(public|protected|private|provide)(\s+)(([a-z_][\w.]*=?|' + r'[-+*/%<>]|==)(?=\s*\())', bygroups(Keyword, Text, Name.Function), + 'signature'), (r'(public|protected|private)(\s+)([a-z_][\w.]*)', bygroups(Keyword, Text, Name.Function)), @@ -3614,6 +3731,9 @@ class LassoLexer(RegexLexer): r'null)\b', Keyword.Type), (r'([a-z_][\w.]*)(\s+)(in)\b', bygroups(Name, Text, Keyword)), (r'(let|into)(\s+)([a-z_][\w.]*)', bygroups(Keyword, Text, Name)), + (r'require\b', Keyword, 'requiresection'), + (r'(/?)(Namespace_Using)\b', + bygroups(Punctuation, Keyword.Namespace)), (r'(/?)(Cache|Database_Names|Database_SchemaNames|' r'Database_TableNames|Define_Tag|Define_Type|Email_Batch|' r'Encode_Set|HTML_Comment|Handle|Handle_Error|Header|If|Inline|' @@ -3621,19 +3741,21 @@ class LassoLexer(RegexLexer): r'Link_CurrentRecord|Link_Detail|Link_FirstGroup|' r'Link_FirstRecord|Link_LastGroup|Link_LastRecord|Link_NextGroup|' r'Link_NextRecord|Link_PrevGroup|Link_PrevRecord|Log|Loop|' - r'Namespace_Using|NoProcess|Output_None|Portal|Private|Protect|' - r'Records|Referer|Referrer|Repeating|ResultSet|Rows|Search_Args|' - r'Search_Arguments|Select|Sort_Args|Sort_Arguments|Thread_Atomic|' - r'Value_List|While|Abort|Case|Else|If_Empty|If_False|If_Null|' - r'If_True|Loop_Abort|Loop_Continue|Loop_Count|Params|Params_Up|' - r'Return|Return_Value|Run_Children|SOAP_DefineTag|' - r'SOAP_LastRequest|SOAP_LastResponse|Tag_Name|ascending|average|' - r'by|define|descending|do|equals|frozen|group|handle_failure|' - r'import|in|into|join|let|match|max|min|on|order|parent|protected|' - r'provide|public|require|skip|split_thread|sum|take|thread|to|' - r'trait|type|where|with|yield)\b', bygroups(Punctuation, Keyword)), + r'NoProcess|Output_None|Portal|Private|Protect|Records|Referer|' + r'Referrer|Repeating|ResultSet|Rows|Search_Args|Search_Arguments|' + r'Select|Sort_Args|Sort_Arguments|Thread_Atomic|Value_List|While|' + r'Abort|Case|Else|If_Empty|If_False|If_Null|If_True|Loop_Abort|' + r'Loop_Continue|Loop_Count|Params|Params_Up|Return|Return_Value|' + r'Run_Children|SOAP_DefineTag|SOAP_LastRequest|SOAP_LastResponse|' + r'Tag_Name|ascending|average|by|define|descending|do|equals|' + r'frozen|group|handle_failure|import|in|into|join|let|match|max|' + r'min|on|order|parent|protected|provide|public|require|skip|' + r'split_thread|sum|take|thread|to|trait|type|where|with|yield)\b', + bygroups(Punctuation, Keyword)), # other + (r'(([a-z_][\w.]*=?|[-+*/%<>]|==)(?=\s*\([^)]*\)\s*=>))', + Name.Function, 'signature'), (r'(and|or|not)\b', Operator.Word), (r'([a-z_][\w.]*)(\s*)(::\s*)([a-z_][\w.]*)(\s*)(=)', bygroups(Name, Text, Punctuation, Name.Label, Text, Operator)), @@ -3662,16 +3784,34 @@ class LassoLexer(RegexLexer): r'[abefnrtv?\"\'\\]|$)', String.Escape), ], 'signature': [ - (r'[(,]', Punctuation, 'parameter'), (r'=>', Operator, '#pop'), + (r'\)', Punctuation, '#pop'), + (r'[(,]', Punctuation, 'parameter'), include('lasso'), ], 'parameter': [ - (r'\.\.\.', Name.Builtin.Pseudo), - (r'-?[a-z_][\w.]*', Name.Attribute, '#pop'), (r'\)', Punctuation, '#pop'), + (r'-?[a-z_][\w.]*', Name.Attribute, '#pop'), + (r'\.\.\.', Name.Builtin.Pseudo), include('lasso'), ], + 'requiresection': [ + (r'(([a-z_][\w.]*=?|[-+*/%<>]|==)(?=\s*\())', Name, 'requiresignature'), + (r'(([a-z_][\w.]*=?|[-+*/%<>]|==)(?=(\s*::\s*[\w.]+)?\s*,))', Name), + (r'[a-z_][\w.]*=?|[-+*/%<>]|==', Name, '#pop'), + (r'(::\s*)([a-z_][\w.]*)', bygroups(Punctuation, Name.Label)), + (r',', Punctuation), + include('whitespacecomments'), + ], + 'requiresignature': [ + (r'(\)(?=(\s*::\s*[\w.]+)?\s*,))', Punctuation, '#pop'), + (r'\)', Punctuation, '#pop:2'), + (r'-?[a-z_][\w.]*', Name.Attribute), + (r'(::\s*)([a-z_][\w.]*)', bygroups(Punctuation, Name.Label)), + (r'\.\.\.', Name.Builtin.Pseudo), + (r'[(,]', Punctuation), + include('whitespacecomments'), + ], } def __init__(self, **options): @@ -3710,3 +3850,76 @@ class LassoLexer(RegexLexer): if re.search(r'\[\n|\?>', text): rv += 0.4 return rv + + +class QmlLexer(RegexLexer): + """ + For QML files. See http://doc.qt.digia.com/4.7/qdeclarativeintroduction.html. + + *New in Pygments 1.6.* + """ + + # QML is based on javascript, so much of this is taken from the + # JavascriptLexer above. + + name = 'QML' + aliases = ['qml', 'Qt Meta Language', 'Qt modeling Language'] + filenames = ['*.qml',] + mimetypes = [ 'application/x-qml',] + + + # pasted from JavascriptLexer, with some additions + flags = re.DOTALL + tokens = { + 'commentsandwhitespace': [ + (r'\s+', Text), + (r'<!--', Comment), + (r'//.*?\n', Comment.Single), + (r'/\*.*?\*/', Comment.Multiline) + ], + 'slashstartsregex': [ + include('commentsandwhitespace'), + (r'/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/' + r'([gim]+\b|\B)', String.Regex, '#pop'), + (r'(?=/)', Text, ('#pop', 'badregex')), + (r'', Text, '#pop') + ], + 'badregex': [ + (r'\n', Text, '#pop') + ], + 'root' : [ + (r'^(?=\s|/|<!--)', Text, 'slashstartsregex'), + include('commentsandwhitespace'), + (r'\+\+|--|~|&&|\?|:|\|\||\\(?=\n)|' + r'(<<|>>>?|==?|!=?|[-<>+*%&\|\^/])=?', Operator, 'slashstartsregex'), + (r'[{(\[;,]', Punctuation, 'slashstartsregex'), + (r'[})\].]', Punctuation), + + # QML insertions + (r'\bid\s*:\s*[A-Za-z][_A-Za-z.0-9]*',Keyword.Declaration, + 'slashstartsregex'), + (r'\b[A-Za-z][_A-Za-z.0-9]*\s*:',Keyword, 'slashstartsregex'), + + # the rest from JavascriptLexer + (r'(for|in|while|do|break|return|continue|switch|case|default|if|else|' + r'throw|try|catch|finally|new|delete|typeof|instanceof|void|' + r'this)\b', Keyword, 'slashstartsregex'), + (r'(var|let|with|function)\b', Keyword.Declaration, 'slashstartsregex'), + (r'(abstract|boolean|byte|char|class|const|debugger|double|enum|export|' + r'extends|final|float|goto|implements|import|int|interface|long|native|' + r'package|private|protected|public|short|static|super|synchronized|throws|' + r'transient|volatile)\b', Keyword.Reserved), + (r'(true|false|null|NaN|Infinity|undefined)\b', Keyword.Constant), + (r'(Array|Boolean|Date|Error|Function|Math|netscape|' + r'Number|Object|Packages|RegExp|String|sun|decodeURI|' + r'decodeURIComponent|encodeURI|encodeURIComponent|' + r'Error|eval|isFinite|isNaN|parseFloat|parseInt|document|this|' + r'window)\b', Name.Builtin), + (r'[$a-zA-Z_][a-zA-Z0-9_]*', Name.Other), + (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), + (r'0x[0-9a-fA-F]+', Number.Hex), + (r'[0-9]+', Number.Integer), + (r'"(\\\\|\\"|[^"])*"', String.Double), + (r"'(\\\\|\\'|[^'])*'", String.Single), + ] + } diff --git a/pygments/plugin.py b/pygments/plugin.py index ea606b9c..58662e96 100644 --- a/pygments/plugin.py +++ b/pygments/plugin.py @@ -32,7 +32,7 @@ yourfilter = yourfilter:YourFilter - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ try: diff --git a/pygments/scanner.py b/pygments/scanner.py index 2acd6e6b..f469e694 100644 --- a/pygments/scanner.py +++ b/pygments/scanner.py @@ -12,7 +12,7 @@ Have a look at the `DelphiLexer` to get an idea of how to use this scanner. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import re diff --git a/pygments/style.py b/pygments/style.py index 0d9e6eb6..0fc01b40 100644 --- a/pygments/style.py +++ b/pygments/style.py @@ -5,7 +5,7 @@ Basic style object. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/__init__.py b/pygments/styles/__init__.py index 2be93da4..3d6ef73c 100644 --- a/pygments/styles/__init__.py +++ b/pygments/styles/__init__.py @@ -5,7 +5,7 @@ Contains built-in styles. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/autumn.py b/pygments/styles/autumn.py index a311487c..3960536b 100644 --- a/pygments/styles/autumn.py +++ b/pygments/styles/autumn.py @@ -5,7 +5,7 @@ A colorful style, inspired by the terminal highlighting style. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/borland.py b/pygments/styles/borland.py index ce6120b5..9858034e 100644 --- a/pygments/styles/borland.py +++ b/pygments/styles/borland.py @@ -5,7 +5,7 @@ Style similar to the style used in the Borland IDEs. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/bw.py b/pygments/styles/bw.py index fdec8a2f..170442ad 100644 --- a/pygments/styles/bw.py +++ b/pygments/styles/bw.py @@ -5,7 +5,7 @@ Simple black/white only style. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/colorful.py b/pygments/styles/colorful.py index 819e81ba..eb595467 100644 --- a/pygments/styles/colorful.py +++ b/pygments/styles/colorful.py @@ -5,7 +5,7 @@ A colorful style, inspired by CodeRay. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/default.py b/pygments/styles/default.py index d90f08d8..77bdac0d 100644 --- a/pygments/styles/default.py +++ b/pygments/styles/default.py @@ -5,7 +5,7 @@ The default highlighting style. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/emacs.py b/pygments/styles/emacs.py index 96640f87..9f8b4074 100644 --- a/pygments/styles/emacs.py +++ b/pygments/styles/emacs.py @@ -5,7 +5,7 @@ A highlighting style for Pygments, inspired by Emacs. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/friendly.py b/pygments/styles/friendly.py index 25dda6ca..732a1252 100644 --- a/pygments/styles/friendly.py +++ b/pygments/styles/friendly.py @@ -5,7 +5,7 @@ A modern style based on the VIM pyte theme. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/fruity.py b/pygments/styles/fruity.py index 6f6f2864..45334159 100644 --- a/pygments/styles/fruity.py +++ b/pygments/styles/fruity.py @@ -5,7 +5,7 @@ pygments version of my "fruity" vim theme. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/manni.py b/pygments/styles/manni.py index 2f689419..036a2120 100644 --- a/pygments/styles/manni.py +++ b/pygments/styles/manni.py @@ -8,7 +8,7 @@ This is a port of the style used in the `php port`_ of pygments by Manni. The style is called 'default' there. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/monokai.py b/pygments/styles/monokai.py index b5338122..31dc83b2 100644 --- a/pygments/styles/monokai.py +++ b/pygments/styles/monokai.py @@ -7,7 +7,7 @@ http://www.monokai.nl/blog/2006/07/15/textmate-color-theme/ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/murphy.py b/pygments/styles/murphy.py index 03494fb6..dbf4eba9 100644 --- a/pygments/styles/murphy.py +++ b/pygments/styles/murphy.py @@ -5,7 +5,7 @@ Murphy's style from CodeRay. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/native.py b/pygments/styles/native.py index 53749e64..0de84386 100644 --- a/pygments/styles/native.py +++ b/pygments/styles/native.py @@ -5,7 +5,7 @@ pygments version of my "native" vim theme. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/pastie.py b/pygments/styles/pastie.py index ca8f6206..2a2f386f 100644 --- a/pygments/styles/pastie.py +++ b/pygments/styles/pastie.py @@ -7,7 +7,7 @@ .. _pastie: http://pastie.caboo.se/ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/perldoc.py b/pygments/styles/perldoc.py index 73f9bd3c..b8b67b29 100644 --- a/pygments/styles/perldoc.py +++ b/pygments/styles/perldoc.py @@ -7,7 +7,7 @@ .. _perldoc: http://perldoc.perl.org/ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/rrt.py b/pygments/styles/rrt.py index 1c363858..1a2fc6a4 100644 --- a/pygments/styles/rrt.py +++ b/pygments/styles/rrt.py @@ -5,7 +5,7 @@ pygments "rrt" theme, based on Zap and Emacs defaults. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/tango.py b/pygments/styles/tango.py index fea7ae5d..7b1c4f3c 100644 --- a/pygments/styles/tango.py +++ b/pygments/styles/tango.py @@ -33,7 +33,7 @@ have been chosen to have the same style. Similarly, keywords (Keyword.*), and Operator.Word (and, or, in) have been assigned the same style. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/trac.py b/pygments/styles/trac.py index 468a5822..714e36cc 100644 --- a/pygments/styles/trac.py +++ b/pygments/styles/trac.py @@ -5,7 +5,7 @@ Port of the default trac highlighter design. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/vim.py b/pygments/styles/vim.py index 0aab8bcd..a5462db3 100644 --- a/pygments/styles/vim.py +++ b/pygments/styles/vim.py @@ -5,7 +5,7 @@ A highlighting style for Pygments, inspired by vim. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/styles/vs.py b/pygments/styles/vs.py index 5cdda4ae..14a56faa 100644 --- a/pygments/styles/vs.py +++ b/pygments/styles/vs.py @@ -5,7 +5,7 @@ Simple style with MS Visual Studio colors. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/token.py b/pygments/token.py index cf78cca2..19a83f2e 100644 --- a/pygments/token.py +++ b/pygments/token.py @@ -5,7 +5,7 @@ Basic token types and the standard tokens. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/pygments/unistring.py b/pygments/unistring.py index b6f53e89..4b07028e 100644 --- a/pygments/unistring.py +++ b/pygments/unistring.py @@ -8,7 +8,7 @@ Inspired by chartypes_create.py from the MoinMoin project. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ from pygments.util import u_prefix diff --git a/pygments/util.py b/pygments/util.py index 127f6e87..caac1144 100644 --- a/pygments/util.py +++ b/pygments/util.py @@ -5,7 +5,7 @@ Utility functions. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/scripts/check_sources.py b/scripts/check_sources.py index e1c7544d..d9e5c2ae 100755 --- a/scripts/check_sources.py +++ b/scripts/check_sources.py @@ -7,7 +7,7 @@ Make sure each Python file has a correct file header including copyright and license information. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -30,7 +30,7 @@ def checker(*suffixes, **kwds): name_mail_re = r'[\w ]+(<.*?>)?' -copyright_re = re.compile(r'^ :copyright: Copyright 2006-2012 by ' +copyright_re = re.compile(r'^ :copyright: Copyright 2006-2013 by ' r'the Pygments team, see AUTHORS\.$', re.UNICODE) copyright_2_re = re.compile(r'^ %s(, %s)*[,.]$' % (name_mail_re, name_mail_re), re.UNICODE) diff --git a/scripts/detect_missing_analyse_text.py b/scripts/detect_missing_analyse_text.py index fc3fef71..1312648f 100644 --- a/scripts/detect_missing_analyse_text.py +++ b/scripts/detect_missing_analyse_text.py @@ -8,6 +8,8 @@ def main(): for name, aliases, filenames, mimetypes in get_all_lexers(): cls = find_lexer_class(name) + if not cls.aliases: + print cls, "has no aliases" for f in filenames: if f not in uses: uses[f] = [] diff --git a/scripts/find_codetags.py b/scripts/find_codetags.py index 35db143f..2fb18333 100755 --- a/scripts/find_codetags.py +++ b/scripts/find_codetags.py @@ -7,7 +7,7 @@ Find code tags in specified files and/or directories and create a report in HTML format. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/scripts/find_error.py b/scripts/find_error.py index 453f16ed..00923569 100755 --- a/scripts/find_error.py +++ b/scripts/find_error.py @@ -8,17 +8,16 @@ the text where Error tokens are being generated, along with some context. - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ import sys, os -try: - import pygments -except ImportError: - # try parent path - sys.path.append(os.path.join(os.path.dirname(__file__), '..')) +# always prefer Pygments from source if exists +srcpath = os.path.join(os.path.dirname(__file__), '..') +if os.path.isdir(os.path.join(srcpath, 'pygments')): + sys.path.insert(0, srcpath) from pygments.lexer import RegexLexer @@ -4,3 +4,4 @@ tag_date = true [aliases] release = egg_info -RDb '' +upload = upload --sign --identity=36580288 @@ -21,9 +21,9 @@ The `Pygments tip`_ is installable with ``easy_install Pygments==dev``. .. _Pygments tip: - http://bitbucket.org/birkenfeld/pygments-main/get/tip.zip#egg=Pygments-dev + http://bitbucket.org/birkenfeld/pygments-main/get/default.zip#egg=Pygments-dev - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -60,7 +60,7 @@ else: setup( name = 'Pygments', - version = '1.5', + version = '1.6', url = 'http://pygments.org/', license = 'BSD License', author = 'Georg Brandl', @@ -77,11 +77,13 @@ setup( 'Intended Audience :: Developers', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: System Administrators', - 'Development Status :: 5 - Production/Stable', + 'Development Status :: 6 - Mature', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 3', 'Operating System :: OS Independent', + 'Topic :: Text Processing :: Filters', + 'Topic :: Utilities', ], cmdclass = {'build_py': build_py}, **add_keywords diff --git a/tests/examplefiles/BOM.js b/tests/examplefiles/BOM.js new file mode 100644 index 00000000..930599c1 --- /dev/null +++ b/tests/examplefiles/BOM.js @@ -0,0 +1 @@ +/* There is a BOM at the beginning of this file. */
\ No newline at end of file diff --git a/tests/examplefiles/autoit_submit.au3 b/tests/examplefiles/autoit_submit.au3 new file mode 100644 index 00000000..e5054dea --- /dev/null +++ b/tests/examplefiles/autoit_submit.au3 @@ -0,0 +1,25 @@ +#include <IE.au3>
+;http://msdn.microsoft.com/en-us/library/Aa752084.aspx
+$ourl="http://localhost:5000/"
+
+$oIE = _IEAttach ($ourl,"url")
+If @error = $_IEStatus_NoMatch Then
+ $oIE = _IECreate ($ourl & "sample.html")
+endIf
+
+$oForm = _IEFormGetObjByName ($oIE, "form1")
+;username, call DOM directly
+$oIE.document.getElementById("username").value="helloAutoIT"
+;state select
+$oSelect = _IEFormElementGetObjByName ($oForm, "state")
+_IEFormElementOptionSelect ($oSelect, "S2", 1, "byText")
+;options raido
+_IEFormElementRadioSelect($oForm, "2nd", "type", 1, "byValue")
+
+ConsoleWrite(@Error)
+Sleep(10000)
+_IEFormSubmit($oForm, 0)
+_IELoadWait($oIE)
+Sleep(60000)
+_IEQuit($oIE)
+
diff --git a/tests/examplefiles/bigtest.nsi b/tests/examplefiles/bigtest.nsi new file mode 100644 index 00000000..62f5211c --- /dev/null +++ b/tests/examplefiles/bigtest.nsi @@ -0,0 +1,308 @@ +; bigtest.nsi +; +; This script attempts to test most of the functionality of the NSIS exehead. + +;-------------------------------- + +!ifdef HAVE_UPX +!packhdr tmp.dat "upx\upx -9 tmp.dat" +!endif + +!ifdef NOCOMPRESS +SetCompress off +!endif + +;-------------------------------- + +Name "BigNSISTest" +Caption "NSIS Big Test" +Icon "${NSISDIR}\Contrib\Graphics\Icons\nsis1-install.ico" +OutFile "bigtest.exe" + +SetDateSave on +SetDatablockOptimize on +CRCCheck on +SilentInstall normal +BGGradient 000000 800000 FFFFFF +InstallColors FF8080 000030 +XPStyle on + +InstallDir "$PROGRAMFILES\NSISTest\BigNSISTest" +InstallDirRegKey HKLM "Software\NSISTest\BigNSISTest" "Install_Dir" + +CheckBitmap "${NSISDIR}\Contrib\Graphics\Checks\classic-cross.bmp" + +LicenseText "A test text, make sure it's all there" +LicenseData "bigtest.nsi" + +RequestExecutionLevel admin + +;-------------------------------- + +Page license +Page components +Page directory +Page instfiles + +UninstPage uninstConfirm +UninstPage instfiles + +;-------------------------------- + +!ifndef NOINSTTYPES ; only if not defined + InstType "Most" + InstType "Full" + InstType "More" + InstType "Base" + ;InstType /NOCUSTOM + ;InstType /COMPONENTSONLYONCUSTOM +!endif + +AutoCloseWindow false +ShowInstDetails show + +;-------------------------------- + +Section "" ; empty string makes it hidden, so would starting with - + + ; write reg info + StrCpy $1 "POOOOOOOOOOOP" + DetailPrint "I like to be able to see what is going on (debug) $1" + WriteRegStr HKLM SOFTWARE\NSISTest\BigNSISTest "Install_Dir" "$INSTDIR" + + ; write uninstall strings + WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\BigNSISTest" "DisplayName" "BigNSISTest (remove only)" + WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\BigNSISTest" "UninstallString" '"$INSTDIR\bt-uninst.exe"' + + SetOutPath $INSTDIR + File /a "silent.nsi" + CreateDirectory "$INSTDIR\MyProjectFamily\MyProject" ; 2 recursively create a directory for fun. + WriteUninstaller "bt-uninst.exe" + + Nop ; for fun + +SectionEnd + +Section "TempTest" + +SectionIn 1 2 3 + Start: MessageBox MB_OK "Start:" + + MessageBox MB_YESNO "Goto MyLabel" IDYES MyLabel + + MessageBox MB_OK "Right before MyLabel:" + + MyLabel: MessageBox MB_OK "MyLabel:" + + MessageBox MB_OK "Right after MyLabel:" + + MessageBox MB_YESNO "Goto Start:?" IDYES Start + +SectionEnd + +SectionGroup /e SectionGroup1 + +Section "Test Registry/INI functions" + +SectionIn 1 4 3 + + WriteRegStr HKLM SOFTWARE\NSISTest\BigNSISTest "StrTest_INSTDIR" "$INSTDIR" + WriteRegDword HKLM SOFTWARE\NSISTest\BigNSISTest "DwordTest_0xDEADBEEF" 0xdeadbeef + WriteRegDword HKLM SOFTWARE\NSISTest\BigNSISTest "DwordTest_123456" 123456 + WriteRegDword HKLM SOFTWARE\NSISTest\BigNSISTest "DwordTest_0123" 0123 + WriteRegBin HKLM SOFTWARE\NSISTest\BigNSISTest "BinTest_deadbeef01f00dbeef" "DEADBEEF01F00DBEEF" + StrCpy $8 "$SYSDIR\IniTest" + WriteINIStr "$INSTDIR\test.ini" "MySection" "Value1" $8 + WriteINIStr "$INSTDIR\test.ini" "MySectionIni" "Value1" $8 + WriteINIStr "$INSTDIR\test.ini" "MySectionIni" "Value2" $8 + WriteINIStr "$INSTDIR\test.ini" "IniOn" "Value1" $8 + + Call MyFunctionTest + + DeleteINIStr "$INSTDIR\test.ini" "IniOn" "Value1" + DeleteINISec "$INSTDIR\test.ini" "MySectionIni" + + ReadINIStr $1 "$INSTDIR\test.ini" "MySectionIni" "Value1" + StrCmp $1 "" INIDelSuccess + MessageBox MB_OK "DeleteINISec failed" + INIDelSuccess: + + ClearErrors + ReadRegStr $1 HKCR "software\microsoft" xyz_cc_does_not_exist + IfErrors 0 NoError + MessageBox MB_OK "could not read from HKCR\software\microsoft\xyz_cc_does_not_exist" + Goto ErrorYay + NoError: + MessageBox MB_OK "read '$1' from HKCR\software\microsoft\xyz_cc_does_not_exist" + ErrorYay: + +SectionEnd + +Section "Test CreateShortCut" + + SectionIn 1 2 3 + + Call CSCTest + +SectionEnd + +SectionGroup Group2 + +Section "Test Branching" + + BeginTestSection: + SectionIn 1 2 3 + + SetOutPath $INSTDIR + + IfFileExists "$INSTDIR\LogicLib.nsi" 0 BranchTest69 + + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to overwrite $INSTDIR\LogicLib.nsi?" IDNO NoOverwrite ; skipped if file doesn't exist + + BranchTest69: + + SetOverwrite ifnewer ; NOT AN INSTRUCTION, NOT COUNTED IN SKIPPINGS + + NoOverwrite: + + File "LogicLib.nsi" ; skipped if answered no + SetOverwrite try ; NOT AN INSTRUCTION, NOT COUNTED IN SKIPPINGS + + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to skip the rest of this section?" IDYES EndTestBranch + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to go back to the beginning of this section?" IDYES BeginTestSection + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to hide the installer and wait five seconds?" IDNO NoHide + + HideWindow + Sleep 5000 + BringToFront + + NoHide: + + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to call the function 5 times?" IDNO NoRecurse + + StrCpy $1 "x" + + LoopTest: + + Call myfunc + StrCpy $1 "x$1" + StrCmp $1 "xxxxxx" 0 LoopTest + + NoRecurse: + + EndTestBranch: + +SectionEnd + +SectionGroupEnd + +Section "Test CopyFiles" + + SectionIn 1 2 3 + + SetOutPath $INSTDIR\cpdest + CopyFiles "$WINDIR\*.ini" "$INSTDIR\cpdest" 0 + +SectionEnd + +SectionGroupEnd + +Section "Test Exec functions" TESTIDX + + SectionIn 1 2 3 + + SearchPath $1 notepad.exe + + MessageBox MB_OK "notepad.exe=$1" + Exec '"$1"' + ExecShell "open" '"$INSTDIR"' + Sleep 500 + BringToFront + +SectionEnd + +Section "Test ActiveX control registration" + + SectionIn 2 + + UnRegDLL "$SYSDIR\spin32.ocx" + Sleep 1000 + RegDLL "$SYSDIR\spin32.ocx" + Sleep 1000 + +SectionEnd + +;-------------------------------- + +Function "CSCTest" + + CreateDirectory "$SMPROGRAMS\Big NSIS Test" + SetOutPath $INSTDIR ; for working directory + CreateShortCut "$SMPROGRAMS\Big NSIS Test\Uninstall BIG NSIS Test.lnk" "$INSTDIR\bt-uninst.exe" ; use defaults for parameters, icon, etc. + ; this one will use notepad's icon, start it minimized, and give it a hotkey (of Ctrl+Shift+Q) + CreateShortCut "$SMPROGRAMS\Big NSIS Test\silent.nsi.lnk" "$INSTDIR\silent.nsi" "" "$WINDIR\notepad.exe" 0 SW_SHOWMINIMIZED CONTROL|SHIFT|Q + CreateShortCut "$SMPROGRAMS\Big NSIS Test\TheDir.lnk" "$INSTDIR\" "" "" 0 SW_SHOWMAXIMIZED CONTROL|SHIFT|Z + +FunctionEnd + +Function myfunc + + StrCpy $2 "MyTestVar=$1" + MessageBox MB_OK "myfunc: $2" + +FunctionEnd + +Function MyFunctionTest + + ReadINIStr $1 "$INSTDIR\test.ini" "MySectionIni" "Value1" + StrCmp $1 $8 NoFailedMsg + MessageBox MB_OK "WriteINIStr failed" + + NoFailedMsg: + +FunctionEnd + +Function .onSelChange + + SectionGetText ${TESTIDX} $0 + StrCmp $0 "" e + SectionSetText ${TESTIDX} "" + Goto e2 +e: + SectionSetText ${TESTIDX} "TextInSection" +e2: + +FunctionEnd + +;-------------------------------- + +; Uninstaller + +UninstallText "This will uninstall example2. Hit next to continue." +UninstallIcon "${NSISDIR}\Contrib\Graphics\Icons\nsis1-uninstall.ico" + +Section "Uninstall" + + DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\BigNSISTest" + DeleteRegKey HKLM "SOFTWARE\NSISTest\BigNSISTest" + Delete "$INSTDIR\silent.nsi" + Delete "$INSTDIR\LogicLib.nsi" + Delete "$INSTDIR\bt-uninst.exe" + Delete "$INSTDIR\test.ini" + Delete "$SMPROGRAMS\Big NSIS Test\*.*" + RMDir "$SMPROGRAMS\BiG NSIS Test" + + MessageBox MB_YESNO|MB_ICONQUESTION "Would you like to remove the directory $INSTDIR\cpdest?" IDNO NoDelete + Delete "$INSTDIR\cpdest\*.*" + RMDir "$INSTDIR\cpdest" ; skipped if no + NoDelete: + + RMDir "$INSTDIR\MyProjectFamily\MyProject" + RMDir "$INSTDIR\MyProjectFamily" + RMDir "$INSTDIR" + + IfFileExists "$INSTDIR" 0 NoErrorMsg + MessageBox MB_OK "Note: $INSTDIR could not be removed!" IDOK 0 ; skipped if file doesn't exist + NoErrorMsg: + +SectionEnd diff --git a/tests/examplefiles/ca65_example b/tests/examplefiles/ca65_example new file mode 100644 index 00000000..67c6313e --- /dev/null +++ b/tests/examplefiles/ca65_example @@ -0,0 +1,284 @@ +;-------------------------------------- +; Lychrel numbers. +; +; :author: Marc 'BlackJack' Rintsch +; :date: 2008-03-07 +; :version: 0.1 +; +; Prints all `Lychrel numbers`_ between 1 and 100000. +; +; The numbers are stored as array of "digits" in little endian +; order. Each digit is a byte with a value between 0 and 9. +; +; Runtime on C64: 00:21:01 +; +; .. _Lychrel numbers: http://en.wikipedia.org/wiki/Lychrel_number +; +; .. cl65 -l -tnone -C simple.cfg lychrel.s -o lychrel.prg +;-------------------------------------- + +;-------------------------------------- +; External addresses. +;-------------------------------------- + chrout = $ffd2 + +;-------------------------------------- +; Constants. +;-------------------------------------- + TO = 100000 + TO_DIGITS = 10 + ITERATIONS = 100 + MAX_DIGITS = TO_DIGITS + ITERATIONS + +;-------------------------------------- +; Global variables. +;-------------------------------------- +.zeropage +; +; Length of the currently tested `n` in digits. +; +n_length: + .res 1 +; +; Length of the number(s) `xa` and `xb` while testing. +; +length: + .res 1 + +.bss +; +; Number to be tested as digits i.e. bytes with values between +; 0 and 9. The length is stored in `n_length`. +; +n: + .res TO_DIGITS +; +; Space for calculating the reversed and added values. +; In the `main` code the current number is copied into `xa` +; and then repeatedly `reverse_add`\ed to itself with the +; result of that adding stored in `xb`. +; +xa: + .res MAX_DIGITS +xb: + .res MAX_DIGITS + +;-------------------------------------- +; BASIC header. +;-------------------------------------- +.code + .word 0800h ; Load address. + .byte 0 + .word @line_end + .word 2008 ; Line number. + .byte $9e ; SYS token. + .byte "2080 " ; SYS argument. + .byte "LYCHREL NUMBERS/BJ" +@line_end: + .byte 0, 0, 0 ; Line and program end marker. + +;-------------------------------------- +; Main program. +;-------------------------------------- +.proc main + +.zeropage +; +; Three byte counter for `TO` iterations (100000 = $0186a0). +; +i: + .res 3 + +.code +; +; Clear and set `n` and `i` to 1. +; + lda #0 ; n := 0; n := 1; i := 1 + sta i+1 + sta i+2 + ldx #TO_DIGITS +clear_n: + sta n-1,x + dex + bne clear_n + inx + stx i + stx n + stx n_length + +mainloop: + jsr is_lychrel + bcc no_lychrel + jsr print_n +no_lychrel: + jsr increase_n + + inc i ; INC(i) + bne skip + inc i+1 + bne skip + inc i+2 +skip: + lda i + cmp #<TO + bne mainloop + lda i+1 + cmp #>TO + bne mainloop + lda i+2 + cmp #^TO + bne mainloop + + rts +.endproc + +;-------------------------------------- +; Print `n` and a trailing newline. +; +; :in: `n_length`, `n` +;-------------------------------------- +.proc print_n + ldy n_length +L1: + lda n-1,y + ora #%110000 ; = '0' + jsr chrout + dey + bne L1 + + lda #13 + jmp chrout +.endproc + +;-------------------------------------- +; Increase `n` by one. +; +; This procedure expects n[n_length] == 0 in case the number gets +; one digit longer. +; +; :in: `n`, `n_length` +; :out: `n`, `n_length` +;-------------------------------------- +.proc increase_n + ldx #0 +L1: + inc n,x ; Increase digit. + lda n,x + cmp #10 ; If "carry", store 0 and go to next digit. + bne return + lda #0 + sta n,x + inx + bne L1 +return: + cpx n_length ; If "carry" after last digit, increase length. + bcc skip + inc n_length +skip: + rts +.endproc + +;-------------------------------------- +; Tests if `n` is a Lychrel number. +; +; :in: `n`, `n_length` +; :out: C is set if yes, cleared otherwise. +; :uses: `length`, `xa`, `xb` +;-------------------------------------- +.proc is_lychrel +.zeropage +i: + .res 1 + +.code + ldx n_length ; xa := n; length := n_length + stx length +L1: + lda n-1,x + sta xa-1,x + dex + bne L1 + + lda #ITERATIONS ; i := ITERATIONS + sta i +L2: + jsr reverse_add + jsr is_palindrome + bne no_palindrome + clc + rts +no_palindrome: + ldx length ; a := b +L3: + lda xb-1,x + sta xa-1,x + dex + bne L3 + + dec i ; Loop body end. + bne L2 + + sec + rts +.endproc + +;-------------------------------------- +; Add the reverse to `xa` to itself and store the result in `xb`. +; +; :in: `length`, `xa` +; :out: `length`, `xb` +;-------------------------------------- +.proc reverse_add +.code + ldx #0 + ldy length + clc +L1: + lda xa,x + adc xa-1,y + + cmp #10 + bcc no_adjust + sbc #10 +no_adjust: + sta xb,x + + dey + inx + txa ; ``eor`` instead of ``cpx`` to keep the carry flag + eor length ; of the addition above. + bne L1 + + bcc no_carry + lda #1 + sta xb,x + inc length +no_carry: + rts +.endproc + +;-------------------------------------- +; Checks if `xb` is a palindrome. +; +; :in: `length`, `xb` +; :out: Z flag set if `xb` is a palindrome, cleared otherwise. +;-------------------------------------- +.proc is_palindrome +.code + ldx #0 + lda length + tay + lsr + sta L1+1 ; Self modifying code! +L1: + cpx #0 ; <<< 0 replaced by (`length` / 2). + beq return + lda xb,x + cmp xb-1,y + bne return + dey + inx + bne L1 +return: + rts +.endproc diff --git a/tests/examplefiles/cbmbas_example b/tests/examplefiles/cbmbas_example new file mode 100644 index 00000000..8d8ded9e --- /dev/null +++ b/tests/examplefiles/cbmbas_example @@ -0,0 +1,9 @@ +10 rem cbm basic v2 example +20 rem comment with keywords: for, data +30 dim a$(20) +35 rem the typical space efficient form of leaving spaces out: +40 fort=0to15:poke646,t:print"{revers on} ";:next +50 geta$:ifa$=chr$(0):goto40 +55 rem it is legal to omit the closing " on line end +60 print"{white}":print"bye... +70 end diff --git a/tests/examplefiles/classes.dylan b/tests/examplefiles/classes.dylan index 6dd55ff2..83faf69c 100644 --- a/tests/examplefiles/classes.dylan +++ b/tests/examplefiles/classes.dylan @@ -1,12 +1,26 @@ +module: sample +comment: for make sure that does not highlight per word. + and it continues on to the next line. + define class <car> (<object>) slot serial-number :: <integer> = unique-serial-number(); - slot model-name :: <string>, + constant slot model-name :: <string>, required-init-keyword: model:; - slot has-sunroof? :: <boolean>, + each-subclass slot has-sunroof? :: <boolean>, init-keyword: sunroof?:, init-value: #f; + keyword foo:; + required keyword bar:; end class <car>; +define class <flying-car> (<car>) +end class <flying-car>; + +let flying-car = make(<flying-car>); +let car? :: <car?> = #f; +let prefixed-car :: <vehicles/car> = #f; +let model :: <car-911> = #f; + define constant $empty-string = ""; define constant $escaped-backslash = '\\'; define constant $escaped-single-quote = '\''; @@ -31,10 +45,81 @@ define method foo() => _ :: <boolean>; #t end method; -define method \+() -end; +define method \+ + (offset1 :: <time-offset>, offset2 :: <time-offset>) + => (sum :: <time-offset>) + let sum = offset1.total-seconds + offset2.total-seconds; + make(<time-offset>, total-seconds: sum); +end method \+; + +define method bar () + 1 | 2 & 3 +end + +if (bar) + 1 +elseif (foo) + 2 +else + 3 +end if; + +select (foo by instance?) + <integer> => 1 + otherwise => 3 +end select; + +/* multi + line + comment +*/ + +/* multi line comments + /* can be */ + nested */ define constant $symbol = #"hello"; define variable *vector* = #[3.5, 5] define constant $list = #(1, 2); define constant $pair = #(1 . "foo") + +let octal-number = #o238; +let hex-number = #x3890ADEF; +let binary-number = #b1010; +let float-exponent = 3.5e10; + +block (return) + with-lock (lock) + return(); + end; +exception (e :: <error>) + format-out("Oh no"); +cleanup + return(); +afterwards + format-out("Hello"); +end; + +define macro repeat + { repeat ?:body end } + => { block (?=stop!) + local method again() ?body; again() end; + again(); + end } +end macro repeat; + +define macro with-decoded-seconds + { + with-decoded-seconds + (?max:variable, ?min:variable, ?sec:variable = ?time:expression) + ?:body + end + } + => { + let (?max, ?min, ?sec) = decode-total-seconds(?time); + ?body + } +end macro; + +let x = "This size call should be seen as a builtin despite the odd case.".siZe; + diff --git a/tests/examplefiles/example.cob b/tests/examplefiles/example.cob new file mode 100644 index 00000000..3f65e498 --- /dev/null +++ b/tests/examplefiles/example.cob @@ -0,0 +1,3556 @@ + IDENTIFICATION DIVISION.
+ PROGRAM-ID. OCic.
+ *****************************************************************
+ ** This program provides a Textual User Interface (TUI) to the **
+ ** process of compiling and (optionally) executing an OpenCOBOL**
+ ** program. **
+ ** **
+ ** This programs execution syntax is as follows: **
+ ** **
+ ** ocic <program-path-and-filename> [ <switch>... ] **
+ ** **
+ ** Once executed, a display screen will be presented showing **
+ ** the compilation options that will be used. The user will **
+ ** have the opportunity to change options, specify new ones **
+ ** and specify any program execution arguments to be used if **
+ ** you select the "Execute" option. When you press the Enter **
+ ** key the program will be compiled. **
+ ** **
+ ** The SCREEN SECTION contains an image of the screen. **
+ ** **
+ ** The "010-Parse-Args" section in the PROCEDURE DIVISION has **
+ ** documentation on switches and their function. **
+ *****************************************************************
+ ** **
+ ** AUTHOR: GARY L. CUTLER **
+ ** CutlerGL@gmail.com **
+ ** Copyright (C) 2009-2010, Gary L. Cutler, GPL **
+ ** **
+ ** DATE-WRITTEN: June 14, 2009 **
+ ** **
+ *****************************************************************
+ ** Note: Depending on which extended DISPLAY handler you're **
+ ** using (PDCurses, Curses, ...), you may need to un- **
+ ** comment any source lines tagged with "SCROLL" in cols **
+ ** 1-6 in order to have error messages scroll properly **
+ ** in the OCic shell window. **
+ *****************************************************************
+ ** DATE CHANGE DESCRIPTION **
+ ** ====== ==================================================== **
+ ** GC0609 Don't display compiler messages file if compilation **
+ ** Is successful. Also don't display messages if the **
+ ** output file is busy (just put a message on the **
+ ** screen, leave the OC screen up & let the user fix **
+ ** the problem & resubmit. **
+ ** GC0709 When 'EXECUTE' is selected, a 'FILE BUSY' error will **
+ ** still cause the (old) executable to be launched. **
+ ** Also, the 'EXTRA SWITCHES' field is being ignored. **
+ ** Changed the title bar to lowlighted reverse video & **
+ ** the message area to highlighted reverse-video. **
+ ** GC0809 Add a SPACE in from of command-line args when **
+ ** executing users program. Add a SPACE after the **
+ ** -ftraceall switch when building cobc command. **
+ ** GC0909 Convert to work on Cygwin/Linux as well as MinGW **
+ ** GC0310 Virtualized the key codes for S-F1 thru S-F7 as they **
+ ** differ depending upon whether PDCurses or NCurses is **
+ ** being used. **
+ ** GC0410 Introduced the cross-reference and source listing **
+ ** features. Also fixed a bug in @EXTRA switch proces- **
+ ** sing where garbage will result if more than the **
+ ** @EXTRA switch is specified. **
+ *****************************************************************
+ ENVIRONMENT DIVISION.
+ CONFIGURATION SECTION.
+ REPOSITORY.
+ FUNCTION ALL INTRINSIC.
+ INPUT-OUTPUT SECTION.
+ FILE-CONTROL.
+ SELECT Bat-File ASSIGN TO Bat-File-Name
+ ORGANIZATION IS LINE SEQUENTIAL.
+
+ SELECT Cobc-Output ASSIGN TO Cobc-Output-File
+ ORGANIZATION IS LINE SEQUENTIAL.
+
+ SELECT Source-Code ASSIGN TO File-Name
+ ORGANIZATION IS LINE SEQUENTIAL
+ FILE STATUS IS FSM-Status.
+ DATA DIVISION.
+ FILE SECTION.
+ FD Bat-File.
+ 01 Bat-File-Rec PIC X(2048).
+
+ FD Cobc-Output.
+ 01 Cobc-Output-Rec PIC X(256).
+
+ FD Source-Code.
+ 01 Source-Code-Record PIC X(80).
+
+ WORKING-STORAGE SECTION.
+ COPY screenio.
+
+ 01 Bat-File-Name PIC X(256).
+
+GC0909 01 Cmd PIC X(512).
+
+ 01 Cobc-Cmd PIC X(256).
+
+ 01 Cobc-Output-File PIC X(256).
+
+ 01 Command-Line-Args PIC X(256).
+
+ 01 Config-File PIC X(12).
+
+GC0310 01 Config-Keys.
+GC0310 05 CK-S-F1 PIC 9(4).
+GC0310 05 CK-S-F2 PIC 9(4).
+GC0310 05 CK-S-F3 PIC 9(4).
+GC0310 05 CK-S-F4 PIC 9(4).
+GC0310 05 CK-S-F5 PIC 9(4).
+GC0310 05 CK-S-F6 PIC 9(4).
+GC0310 05 CK-S-F7 PIC 9(4).
+
+GC0909 01 Dir-Char PIC X(1).
+
+ 01 Dummy PIC X(1).
+
+ 01 Env-TEMP PIC X(256).
+
+ 01 File-Name.
+ 05 FN-Char OCCURS 256 TIMES PIC X(1).
+
+ 01 File-Status-Message.
+ 05 FILLER PIC X(13) VALUE 'Status Code: '.
+ 05 FSM-Status PIC 9(2).
+ 05 FILLER PIC X(11) VALUE ', Meaning: '.
+ 05 FSM-Msg PIC X(25).
+
+ 01 Flags.
+ 05 F-Compilation-Succeeded PIC X(1).
+ 88 88-Compile-OK VALUE 'Y'.
+GC0909 88 88-Compile-OK-Warn VALUE 'W'.
+ 88 88-Compile-Failed VALUE 'N'.
+GC0609 05 F-Complete PIC X(1).
+GC0609 88 88-Complete VALUE 'Y'.
+GC0609 88 88-Not-Complete VALUE 'N'.
+GC0809 05 F-IDENT-DIVISION PIC X(1).
+GC0809 88 88-1st-Prog-Complete VALUE 'Y'.
+GC0809 88 88-More-To-1st-Prog VALUE 'N'.
+ 05 F-LINKAGE-SECTION PIC X(1).
+ 88 88-Compile-As-Subpgm VALUE 'Y'.
+ 88 88-Compile-As-Mainpgm VALUE 'N'.
+ 05 F-No-Switch-Changes PIC X(1).
+ 88 88-No-Switch-Changes VALUE 'Y'.
+ 88 88-Switch-Changes VALUE 'N'.
+GC0709 05 F-Output-File-Busy PIC X(1).
+GC0709 88 88-Output-File-Busy VALUE 'Y'.
+GC0709 88 88-Output-File-Avail VALUE 'N'.
+GC0809 05 F-Source-Record-Type PIC X(1).
+GC0809 88 88-Source-Rec-Linkage VALUE 'L'.
+GC0809 88 88-Source-Rec-Ident VALUE 'I'.
+GC0809 88 88-Source-Rec-IgnoCOB-COLOR-RED VALUE ' '.
+ 05 F-Switch-Error PIC X(1).
+ 88 88-Switch-Is-Bad VALUE 'Y'.
+ 88 88-Switch-Is-Good VALUE 'N'.
+
+GC0909 01 Horizontal-Line PIC X(80).
+GC0909
+ 01 I USAGE BINARY-LONG.
+
+ 01 J USAGE BINARY-LONG.
+
+GC0909 01 MS USAGE BINARY-LONG.
+
+GC0909 01 ML USAGE BINARY-LONG.
+
+ 01 OC-Compiled PIC XXXX/XX/XXBXX/XX.
+
+GC0909 01 OS-Type USAGE BINARY-LONG.
+GC0909 88 OS-Unknown VALUE 0.
+GC0909 88 OS-Windows VALUE 1.
+GC0909 88 OS-Cygwin VALUE 2.
+GC0909 88 OS-UNIX VALUE 3.
+
+GC0909 01 OS-Type-Literal PIC X(7).
+
+ 01 Output-Message PIC X(80).
+
+ 01 Path-Delimiter PIC X(1).
+
+ 01 Prog-Folder PIC X(256).
+
+ 01 Prog-Extension PIC X(30).
+
+ 01 Prog-File-Name PIC X(40).
+
+ 01 Prog-Name PIC X(31).
+
+ 78 Selection-Char VALUE '>'.
+
+ 01 Switch-Display.
+ 05 SD-Switch-And-Value PIC X(19).
+ 05 FILLER PIC X(1).
+ 05 SD-Description PIC X(60).
+
+ 01 Switch-Keyword PIC X(12).
+GC0410 88 Switch-Is-CONFIG VALUE '@CONFIG', '@C'.
+GC0410 88 Switch-Is-DEBUG VALUE '@DEBUG', '@D'.
+GC0410 88 Switch-Is-DLL VALUE '@DLL'.
+GC0410 88 Switch-Is-EXECUTE VALUE '@EXECUTE', '@E'.
+GC0410 88 Switch-Is-EXTRA VALUE '@EXTRA', '@EX'.
+GC0410 88 Switch-Is-NOTRUNC VALUE '@NOTRUNC', '@N'.
+GC0410 88 Switch-Is-TRACE VALUE '@TRACE', '@T'.
+GC0410 88 Switch-Is-SOURCE VALUE '@SOURCE', '@S'.
+GC0410 88 Switch-Is-XREF VALUE '@XREF', '@X'.
+
+ 01 Switch-Keyword-And-Value PIC X(256).
+
+ 01 Switch-Value.
+ 05 SV-1 PIC X(1).
+ 05 FILLER PIC X(255).
+ 01 Switch-Value-Alt REDEFINES Switch-Value
+ PIC X(256).
+ 88 Valid-Config-Filename
+ VALUE 'BS2000', 'COBOL85', 'COBOL2002', 'DEFAULT',
+ 'IBM', 'MF', 'MVS'.
+
+ 01 Switches.
+ 05 S-ARGS PIC X(75) VALUE SPACES.
+ 05 S-CfgS.
+ 10 S-Cfg-BS2000 PIC X(1) VALUE ' '.
+ 10 S-Cfg-COBOL85 PIC X(1) VALUE ' '.
+ 10 S-Cfg-COBOL2002 PIC X(1) VALUE ' '.
+ 10 S-Cfg-DEFAULT PIC X(1) VALUE Selection-Char.
+ 10 S-Cfg-IBM PIC X(1) VALUE ' '.
+ 10 S-Cfg-MF PIC X(1) VALUE ' '.
+ 10 S-Cfg-MVS PIC X(1) VALUE ' '.
+ 05 S-EXTRA PIC X(75) VALUE SPACES.
+ 05 S-Yes-No-Switches.
+ 10 S-DEBUG PIC X(1) VALUE 'N'.
+ 10 S-DLL PIC X(1) VALUE 'N'.
+GC0410 10 S-XREF PIC X(1) VALUE 'N'.
+GC0410 10 S-SOURCE PIC X(1) VALUE 'N'.
+ 10 S-EXECUTE PIC X(1) VALUE 'N'.
+ 10 S-NOTRUNC PIC X(1) VALUE 'Y'.
+ 10 S-SUBROUTINE PIC X(1) VALUE 'A'.
+ 10 S-TRACE PIC X(1) VALUE 'N'.
+ 10 S-TRACEALL PIC X(1) VALUE 'N'.
+
+ 01 Tally USAGE BINARY-LONG.
+
+ SCREEN SECTION.
+ *>
+ *> Here is the layout of the OCic screen.
+ *>
+ *> Note that this program can utilize the traditional PC line-drawing characters,
+ *> if they are available.
+ *>
+ *> If this program is run on Windows, it must run with codepage 437 activated to
+ *> display the line-drawing characters. With a native Windows build or a
+ *> Windows/MinGW build, one could use the command "chcp 437" to set that codepage
+ *> for display within a Windows console window (that should be the default, though).
+ *> With a Windows/Cygwin build, set the environment variable CYGWIN to a value of
+ *> "codepage:oem" (this cannot be done from within the program though - you will
+ *> have to use the "Computer/Advanced System Settings/Environment Variables" (Vista or
+ *> Windows 7) function to define the variable. XP Users: use "My Computer/Properties/
+ *> Advanced/Environment Variables".
+ *>
+ *> To use OCic without the line-drawing characters, comment-out the first set of
+ *> 78 "LD" items and uncomment the second.
+ *>
+ *> The following sample screen layout shows how the screen looks with line-drawing
+ *> characters disabled.
+ *>
+ *>===================================================================================
+ *> OCic (2010/04/02 11:36) - OpenCOBOL V1.1 Interactive Compilation Windows 01
+ *> +-----------------------------------------------------------------------------+ 02
+ *> | Program: OCic F-Key: Select Opt | 03
+ *> | Folder: E:\OpenCOBOL\Samples Enter: Compile | 04
+ *> | Filename: OCic.cbl Esc: Quit | 05
+ *> +-----------------------------------------------------------------------------+ 06
+ *> On/Off Switches: Configuration: 07
+ *> +---------------------------------------------------------+-------------------+ 08
+ *> | F1 Compile debug lines F8 Produce source listing | S-F1 BS2000 | 09
+ *> | F2 Always make DLLs F9 Produce xref listing | S-F2 COBOL85 | 10
+ *> | F3 Pgm is a SUBROUTINE | S-F3 COBOL2002 | 11
+ *> | F4 Execute if compile OK | S-F4 > Default | 12
+ *> | F5 > No COMP/BINARY trunc | S-F5 IBM | 13
+ *> | F6 Trace procedures | S-F6 MicroFocus | 14
+ *> | F7 Trace proc + stmnts | S-F7 MVS | 15
+ *> +---------------------------------------------------------+-------------------+ 16
+ *> Additional "cobc" Switches (if any): 17
+ *> +-----------------------------------------------------------------------------+ 18
+ *> | -O2________________________________________________________________________ | 19
+ *> +-----------------------------------------------------------------------------+ 20
+ *> Program Execution Arguments (if any): 21
+ *> +-----------------------------------------------------------------------------+ 22
+ *> | ___________________________________________________________________________ | 23
+ *> +-----------------------------------------------------------------------------+ 24
+ *> OCic Copyright (C) 2009-2010, Gary L. Cutler, GPL 25
+ *>===================================================================================
+ *>12345678901234567890123456789012345678901234567890123456789012345678901234567890
+ *> 1 2 3 4 5 6 7 8
+ *>
+ *> USE THESE CHARS FOR LINE-DRAWING IF YOU HAVE ACCESS TO PC-DOS CODEPAGE 437:
+ *>
+ 78 LD-UL-Corner VALUE X"DA".
+ 78 LD-LL-Corner VALUE X"C0".
+ 78 LD-UR-Corner VALUE X"BF".
+ 78 LD-LR-Corner VALUE X"D9".
+ 78 LD-Upper-T VALUE X"C2".
+ 78 LD-Lower-T VALUE X"C1".
+ 78 LD-Horiz-Line VALUE X"C4".
+ 78 LD-Vert-Line VALUE X"B3".
+ *>
+ *> USE THESE CHARS FOR LINE-DRAWING IF YOU DO NOT HAVE ACCESS TO PC-DOS CODEPAGE 437:
+ *>
+ *> 78 LD-UL-Corner VALUE '+'.
+ *> 78 LD-LL-Corner VALUE '+'.
+ *> 78 LD-UR-Corner VALUE '+'.
+ *> 78 LD-LR-Corner VALUE '+'.
+ *> 78 LD-Upper-T VALUE '+'.
+ *> 78 LD-Lower-T VALUE '+'.
+ *> 78 LD-Horiz-Line VALUE '-'.
+ *> 78 LD-Vert-Line VALUE '|'.
+ *>
+ 01 Blank-Screen LINE 1 COLUMN 1 BLANK SCREEN.
+
+ 01 Switches-Screen BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-WHITE AUTO.
+ *>
+ *> GENERAL SCREEN FRAMEWORK
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-BLUE HIGHLIGHT.
+ 05 LINE 02 COL 02 VALUE LD-UL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-UR-Corner.
+
+ 05 LINE 03 COL 02 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 04 COL 02 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 05 COL 02 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 06 COL 02 VALUE LD-LL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-LR-Corner.
+
+ 05 LINE 08 COL 02 VALUE LD-UL-Corner.
+ 05 PIC X(57) FROM Horizontal-Line.
+ 05 COL 60 VALUE LD-Upper-T.
+ 05 PIC X(19) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-UR-Corner.
+
+ 05 LINE 09 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 10 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 11 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 12 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 13 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 14 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 15 COL 02 VALUE LD-Vert-Line.
+ 05 COL 60 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 16 COL 02 VALUE LD-LL-Corner.
+ 05 PIC X(57) FROM Horizontal-Line.
+ 05 COL 60 VALUE LD-Lower-T.
+ 05 PIC X(19) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-LR-Corner.
+
+ 05 LINE 18 COL 02 VALUE LD-UL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-UR-Corner.
+
+ 05 LINE 19 COL 02 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 20 COL 02 VALUE LD-LL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-LR-Corner.
+
+ 05 LINE 22 COL 02 VALUE LD-UL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-UR-Corner.
+
+ 05 LINE 23 COL 02 VALUE LD-Vert-Line.
+ 05 COL 80 VALUE LD-Vert-Line.
+
+ 05 LINE 24 COL 02 VALUE LD-LL-Corner.
+ 05 PIC X(77) FROM Horizontal-Line.
+ 05 COL 80 VALUE LD-LR-Corner.
+ *>
+ *> TOP AND BOTTOM LINES
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLUE BLINK
+ FOREGROUND-COLOR COB-COLOR-WHITE HIGHLIGHT.
+GC0410 05 LINE 01 COL 01 VALUE ' OCic ('.
+GC0410 05 PIC X(16) FROM OC-Compiled.
+GC0410 05 VALUE ') OpenCOBOL V1.1 06FEB2009 ' &
+GC0410 'Interactive Compilation '.
+GC0410 05 LINE 25 COL 01 PIC X(81) FROM Output-Message.
+ *>
+ *> LABELS
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-CYAN HIGHLIGHT.
+ 05 LINE 07 COL 04 VALUE 'On/Off Switches:'.
+ 05 COL 62 VALUE 'Configuration:'.
+ 05 LINE 17 COL 04 VALUE 'Additional "cobc" Switches (if any
+ - '):'.
+ 05 LINE 21 COL 04 VALUE 'Program Execution Arguments (if an
+ - 'y):'.
+ *>
+ *> TOP SECTION BACKGROUND
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-CYAN LOWLIGHT.
+ 05 LINE 03 COL 04 VALUE 'Program: '.
+ 05 LINE 04 COL 04 VALUE 'Folder: '.
+ 05 LINE 05 COL 04 VALUE 'Filename: '.
+
+ 05 LINE 03 COL 62 VALUE 'F-Key: Select Opt'.
+ 05 LINE 04 COL 62 VALUE 'Enter: Compile '.
+ 05 LINE 05 COL 62 VALUE 'Esc: Quit '.
+ *>
+ *> TOP SECTION PROGRAM INFO
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-WHITE HIGHLIGHT.
+ 05 LINE 03 COL 14 PIC X(47) FROM Prog-Name.
+ 05 LINE 04 COL 14 PIC X(47) FROM Prog-Folder.
+ 05 LINE 05 COL 14 PIC X(47) FROM Prog-File-Name.
+ *>
+ *> MIDDLE LEFT SECTION F-KEYS
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-WHITE HIGHLIGHT.
+ 05 LINE 09 COL 04 VALUE 'F1'.
+ 05 LINE 10 COL 04 VALUE 'F2'.
+ 05 LINE 11 COL 04 VALUE 'F3'.
+ 05 LINE 12 COL 04 VALUE 'F4'.
+ 05 LINE 13 COL 04 VALUE 'F5'.
+ 05 LINE 14 COL 04 VALUE 'F6'.
+ 05 LINE 15 COL 04 VALUE 'F7'.
+ 05 LINE 09 COL 32 VALUE 'F8'.
+ 05 LINE 10 COL 32 VALUE 'F9'.
+ *>
+ *> MIDDLE LEFT SECTION SWITCHES
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-RED HIGHLIGHT.
+ 05 LINE 09 COL 07 PIC X(1) FROM S-DEBUG.
+ 05 LINE 10 COL 07 PIC X(1) FROM S-DLL.
+ 05 LINE 11 COL 07 PIC X(1) FROM S-SUBROUTINE.
+ 05 LINE 12 COL 07 PIC X(1) FROM S-EXECUTE.
+ 05 LINE 13 COL 07 PIC X(1) FROM S-NOTRUNC.
+ 05 LINE 14 COL 07 PIC X(1) FROM S-TRACE.
+ 05 LINE 15 COL 07 PIC X(1) FROM S-TRACEALL.
+ 05 LINE 09 COL 35 PIC X(1) FROM S-SOURCE.
+ 05 LINE 10 COL 35 PIC X(1) FROM S-XREF.
+ *>
+ *> MIDDLE LEFT SECTION BACKGROUND
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-CYAN LOWLIGHT.
+ 05 LINE 09 COL 09 VALUE 'Compile debug lines '.
+ 05 LINE 10 COL 09 VALUE 'Always make DLLs '.
+ 05 LINE 11 COL 09 VALUE 'Pgm is a SUBROUTINE '.
+ 05 LINE 12 COL 09 VALUE 'Execute if compile OK '.
+ 05 LINE 13 COL 09 VALUE 'No COMP/BINARY trunc '.
+ 05 LINE 14 COL 09 VALUE 'Trace procedures '.
+ 05 LINE 15 COL 09 VALUE 'Trace proc + stmnts '.
+ 05 LINE 09 COL 37 VALUE 'Produce source listing'.
+ 05 LINE 10 COL 37 VALUE 'Produce xref listing '.
+ *>
+ *> MIDDLE RIGHT SECTION F-KEYS
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-WHITE HIGHLIGHT.
+ 05 LINE 09 COL 62 VALUE 'S-F1'.
+ 05 LINE 10 COL 62 VALUE 'S-F2'.
+ 05 LINE 11 COL 62 VALUE 'S-F3'.
+ 05 LINE 12 COL 62 VALUE 'S-F4'.
+ 05 LINE 13 COL 62 VALUE 'S-F5'.
+ 05 LINE 14 COL 62 VALUE 'S-F6'.
+ 05 LINE 15 COL 62 VALUE 'S-F7'.
+ *>
+ *> MIDDLE RIGHT SECTION SWITCHES
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-RED HIGHLIGHT.
+ 05 LINE 09 COL 67 PIC X(1) FROM S-Cfg-BS2000.
+ 05 LINE 10 COL 67 PIC X(1) FROM S-Cfg-COBOL85.
+ 05 LINE 11 COL 67 PIC X(1) FROM S-Cfg-COBOL2002.
+ 05 LINE 12 COL 67 PIC X(1) FROM S-Cfg-DEFAULT.
+ 05 LINE 13 COL 67 PIC X(1) FROM S-Cfg-IBM.
+ 05 LINE 14 COL 67 PIC X(1) FROM S-Cfg-MF.
+ 05 LINE 15 COL 67 PIC X(1) FROM S-Cfg-MVS.
+ *>
+ *> MIDDLE RIGHT SECTION BACKGROUND
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-CYAN LOWLIGHT.
+ 05 LINE 09 COL 69 VALUE 'BS2000 '.
+ 05 LINE 10 COL 69 VALUE 'COBOL85 '.
+ 05 LINE 11 COL 69 VALUE 'COBOL2002 '.
+ 05 LINE 12 COL 69 VALUE 'Default '.
+ 05 LINE 13 COL 69 VALUE 'IBM '.
+ 05 LINE 14 COL 69 VALUE 'MicroFocus'.
+ 05 LINE 15 COL 69 VALUE 'MVS '.
+ *>
+ *> FREE-FORM OPTIONS FIELDS
+ *>
+ 03 BACKGROUND-COLOR COB-COLOR-BLACK
+ FOREGROUND-COLOR COB-COLOR-WHITE HIGHLIGHT.
+ 05 LINE 19 COL 04 PIC X(75) USING S-EXTRA.
+ 05 LINE 23 COL 04 PIC X(75) USING S-ARGS.
+ /
+ PROCEDURE DIVISION.
+ *****************************************************************
+ ** Legend to procedure names: **
+ ** **
+ ** 00x-xxx All MAIN driver procedures **
+ ** 0xx-xxx All GLOBAL UTILITY procedures **
+ ** 1xx-xxx All INITIALIZATION procedures **
+ ** 2xx-xxx All CORE PROCESSING procedures **
+ ** 9xx-xxx All TERMINATION procedures **
+ *****************************************************************
+ DECLARATIVES.
+ 000-File-Error SECTION.
+ USE AFTER STANDARD ERROR PROCEDURE ON Source-Code.
+ 000-Handle-Error.
+ COPY FileStat-Msgs
+ REPLACING STATUS BY FSM-Status
+ MSG BY FSM-Msg.
+ MOVE SPACES TO Output-Message
+ IF FSM-Status = 35
+ DISPLAY
+ 'File not found: "'
+ TRIM(File-Name,TRAILING)
+ '"'
+ END-DISPLAY
+ ELSE
+ DISPLAY
+ 'Error accessing file: "'
+ TRIM(File-Name,TRAILING)
+ '"'
+ END-DISPLAY
+ END-IF
+ GOBACK
+ .
+ END DECLARATIVES.
+ /
+ 000-Main SECTION.
+
+ PERFORM 100-Initialization
+GC0609 SET 88-Not-Complete TO TRUE
+GC0609 PERFORM UNTIL 88-Complete
+GC0609 PERFORM 200-Let-User-Set-Switches
+GC0609 PERFORM 210-Run-Compiler
+GC0410 IF (88-Compile-OK OR 88-Compile-OK-Warn)
+GC0410 AND (S-XREF NOT = SPACE OR S-SOURCE NOT = SPACE)
+GC0410 PERFORM 220-Make-Listing
+GC0410 END-IF
+GC0709 IF (S-EXECUTE NOT = SPACES)
+GC0709 AND (88-Output-File-Avail)
+GC0609 PERFORM 230-Run-Program
+GC0609 END-IF
+GC0609 END-PERFORM
+ .
+
+ 009-Done.
+ PERFORM 900-Terminate
+ .
+ * -- Control will NOT return
+ /
+ 010-Parse-Args SECTION.
+ *****************************************************************
+ ** Process a sequence of KEYWORD=VALUE items. These are items **
+ ** specified on the command-line to provide the initial **
+ ** options shown selected on the screen. When integrating **
+ ** OCic into an edirot or framework, include these switches on **
+ ** the ocic.exe command the editor/framework executes. Any **
+ ** underlined choice is the default value for that switch. **
+ ** **
+ ** @CONFIG=BS2000|COBOL85|COBOL2002|DEFAULT|IBM|MF|MVS **
+ ** ======= **
+ ** This switch specifies the default cobc compiler configura- **
+ ** tion file to be used **
+ ** **
+ ** @DEBUG=YES|NO **
+ ** == **
+ ** This switch specifies whether (YES) or not (NO) debugging **
+ ** lines (those with a "D" in column 7) will be compiled. **
+ ** **
+ ** @DLL=YES|NO **
+ ** == **
+ ** Use this switch to force ALL compiled programs to be built **
+ ** as DLLs ("@DLL=YES"). When main programs are built as DLLs **
+ ** they must be executed using the cobcrun utility. When **
+ ** "@DLL=NO" is in effect, main programs are generated as **
+ ** actual "exe" files and only subprograms will be generated **
+ ** as DLLs. **
+ ** **
+ ** @EXECUTE=YES|NO **
+ ** == **
+ ** This switch specifies whether ("@EXECUTE=YES") or not **
+ ** ("@EXECUTE=NO") the program will be executed after it is **
+ ** successfully compiled. **
+ ** **
+ ** @EXTRA=extra cobc argument(s) **
+ ** **
+ ** This switch allows you to specify additional cobc arguments **
+ ** that aren't managed by the other OC switches. If used, **
+ ** this must be the last switch specified on the command line, **
+ ** as everything that follows the "=" will be placed on the **
+ ** cobc command generated by OC. **
+ ** **
+ ** @NOTRUNC=YES|NO **
+ ** === **
+ ** This switch specifies whether (YES) or not (NO) the sup- **
+ ** pression of binary field truncation will occur. If a PIC **
+ ** 99 COMP field (one byte of storage), for example, is given **
+ ** the value 123, it may have its value truncated to 23 when **
+ ** DISPLAYed. Regardless of the NOTRUNC setting, internally **
+ ** the full precision of the field (allowing a maximum value **
+ ** of 255) will be preserved. Even though truncation - if it **
+ ** does occur - would appear to have a minimal disruption on **
+ ** program operation, it has a significant effect on program **
+ ** run-time speed. **
+ ** **
+ ** @TRACE=YES|NO|ALL **
+ ** == **
+ ** This switch controls whether or not code will be added to **
+ ** the object program to produce execution-time logic traces. **
+ ** A specification of "@TRACE=NO" means no such code will be **
+ ** produced. By specifying "@TRACE=YES", code will be genera- **
+ ** ted to display procedure names as they are entered. A **
+ ** "@TRACE=ALL" specification will generate not only procedure **
+ ** traces (as "@TRACE=YES" would) but also statement-level **
+ ** traces too! All trace output is written to STDERR, so **
+ ** adding a "2>file" to the execution of the program will pipe **
+ ** the trace output to a file. You may find it valuable to **
+ ** add your own DISPLAY statements to the debugging output via **
+ ** "DISPLAY xx UPON SYSERR" The SYSERR device corresponds to **
+ ** the Windows or UNIX STDERR device and will therefore honor **
+ ** any "2>file" placed at the end of your program's execution. **
+ ** Add a "D" in column 7 and you can control the generation or **
+ ** ignoring of these DISPLAY statements via the "@DEBUG" **
+ ** switch. **
+ ** **
+GC0410** @SOURCE=YES|NO **
+GC0410** == **
+GC0410** Use this switch to produce a source listing of the program, **
+GC0410** PROVIDED it compiles without errors. **
+ ** **
+GC0410** @XREF=YES|NO **
+GC0410** == **
+GC0410** Use this switch to produce a cross-reference listing of the **
+GC0410** program, PROVIDED it compiles without errors. **
+ *****************************************************************
+
+ 011-Init.
+ MOVE 1 TO I
+ .
+
+ 012-Extract-Kwd-And-Value.
+ PERFORM UNTIL I NOT < LENGTH(Command-Line-Args)
+ MOVE I TO J
+ UNSTRING Command-Line-Args
+ DELIMITED BY ALL SPACES
+ INTO Switch-Keyword-And-Value
+ WITH POINTER I
+ END-UNSTRING
+ IF Switch-Keyword-And-Value NOT = SPACES
+ UNSTRING Switch-Keyword-And-Value
+ DELIMITED BY '='
+ INTO Switch-Keyword, Switch-Value
+ END-UNSTRING
+ PERFORM 030-Process-Keyword
+ END-IF
+ END-PERFORM
+ .
+
+ 019-Done.
+ EXIT.
+
+ *****************************************************************
+ ** Since this program uses the SCREEN SECTION, it cannot do **
+ ** conventional console DISPLAY operations. This routine **
+ ** (which, I admit, is like using an H-bomb to hunt rabbits) **
+ ** will submit an "ECHO" command to the system to simulate a **
+ ** DISPLAY. **
+ *****************************************************************
+ 021-Build-And-Issue-Command.
+ DISPLAY
+ Output-Message
+ END-DISPLAY
+ .
+
+ 029-Done.
+ EXIT.
+ /
+ 030-Process-Keyword SECTION.
+ *****************************************************************
+ ** Process a single KEYWORD=VALUE item. **
+ *****************************************************************
+
+ 031-Init.
+ MOVE UPPER-CASE(Switch-Keyword) TO Switch-Keyword
+ SET 88-Switch-Is-Good TO TRUE
+ .
+
+ 032-Process.
+ EVALUATE TRUE
+ WHEN Switch-Is-EXTRA
+GC0410 MOVE J TO I
+ UNSTRING Command-Line-Args DELIMITED BY '='
+ INTO Dummy, S-EXTRA
+GC0410 WITH POINTER I
+GC0410 END-UNSTRING
+ MOVE LENGTH(Command-Line-Args) TO I
+ WHEN Switch-Is-CONFIG
+ MOVE 'CONFIG' TO Switch-Keyword
+ MOVE UPPER-CASE(Switch-Value)
+ TO Switch-Value
+ EVALUATE Switch-Value
+ WHEN 'BS2000'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-BS2000
+ WHEN 'COBOL85'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-COBOL85
+ WHEN 'COBOL2002'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-COBOL2002
+ WHEN 'DEFAULT'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-DEFAULT
+ WHEN 'IBM'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-IBM
+ WHEN 'MF'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-MF
+ WHEN 'MVS'
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-MVS
+ WHEN OTHER
+ MOVE 'An invalid /CONFIG switch value ' &
+ 'was specified on the command line ' &
+ '- ignored'
+ TO Output-Message
+ END-EVALUATE
+ WHEN Switch-Is-DEBUG
+ MOVE 'DEBUG' TO Switch-Keyword
+ MOVE UPPER-CASE(Switch-Value)
+ TO Switch-Value
+ PERFORM 040-Process-Yes-No-Value
+ IF 88-Switch-Is-Good
+ MOVE SV-1 TO S-DEBUG
+ END-IF
+GC0410 WHEN Switch-Is-DLL
+GC0410 MOVE 'DLL' TO Switch-Keyword
+GC0410 MOVE UPPER-CASE(Switch-Value)
+GC0410 TO Switch-Value
+GC0410 PERFORM 040-Process-Yes-No-Value
+GC0410 IF 88-Switch-Is-Good
+GC0410 MOVE SV-1 TO S-DLL
+GC0410 END-IF
+ WHEN Switch-Is-EXECUTE
+ MOVE 'EXECUTE' TO Switch-Keyword
+ MOVE UPPER-CASE(Switch-Value)
+ TO Switch-Value
+ PERFORM 040-Process-Yes-No-Value
+ IF 88-Switch-Is-Good
+ MOVE SV-1 TO S-EXECUTE
+ END-IF
+ WHEN Switch-Is-NOTRUNC
+ MOVE 'NOTRUNC' TO Switch-Keyword
+ MOVE UPPER-CASE(Switch-Value)
+ TO Switch-Value
+ PERFORM 040-Process-Yes-No-Value
+ IF 88-Switch-Is-Good
+ MOVE SV-1 TO S-NOTRUNC
+ END-IF
+GC0410 WHEN Switch-Is-SOURCE
+GC0410 MOVE 'SOURCE' TO Switch-Keyword
+GC0410 MOVE UPPER-CASE(Switch-Value)
+GC0410 TO Switch-Value
+GC0410 PERFORM 050-Process-Yes-No-All
+GC0410 IF 88-Switch-Is-Good
+GC0410 MOVE SV-1 TO S-SOURCE
+GC0410 END-IF
+ WHEN Switch-Is-TRACE
+ MOVE 'TRACE' TO Switch-Keyword
+ MOVE UPPER-CASE(Switch-Value)
+ TO Switch-Value
+ PERFORM 050-Process-Yes-No-All
+ IF 88-Switch-Is-Good
+ MOVE SV-1 TO S-TRACE
+ END-IF
+GC0410 WHEN Switch-Is-XREF
+GC0410 MOVE 'XREF' TO Switch-Keyword
+GC0410 MOVE UPPER-CASE(Switch-Value)
+GC0410 TO Switch-Value
+GC0410 PERFORM 050-Process-Yes-No-All
+GC0410 IF 88-Switch-Is-Good
+GC0410 MOVE SV-1 TO S-XREF
+GC0410 END-IF
+ WHEN OTHER
+ MOVE SPACES TO Output-Message
+ STRING '"'
+ TRIM(Switch-Keyword)
+ '" is not a valid switch ' &
+ '- ignored'
+ DELIMITED SIZE
+ INTO Output-Message
+ END-STRING
+ SET 88-Switch-Is-Bad TO TRUE
+ END-EVALUATE
+ .
+
+ 039-Done.
+ EXIT.
+ /
+ 040-Process-Yes-No-Value SECTION.
+ *****************************************************************
+ ** Process a switch value of YES or NO **
+ *****************************************************************
+
+ 042-Process.
+ EVALUATE SV-1
+ WHEN 'Y'
+ MOVE 'YES' TO Switch-Value
+ WHEN 'N'
+ MOVE 'NO' To Switch-Value
+ WHEN OTHER
+ MOVE SPACES TO Output-Message
+ STRING '*ERROR: "' TRIM(Switch-Value)
+ '" is not a valid value for the "'
+ TRIM(Switch-Keyword) '" switch'
+ DELIMITED SPACES
+ INTO Output-Message
+ END-STRING
+ SET 88-Switch-Is-Bad TO TRUE
+ END-EVALUATE
+ .
+
+ 049-Done.
+ EXIT.
+ /
+ 050-Process-Yes-No-All SECTION.
+ *****************************************************************
+ ** Process a switch value of YES, NO or ALL **
+ *****************************************************************
+
+ 052-Process.
+ IF SV-1 = 'A'
+ MOVE 'ALL' TO Switch-Value
+ ELSE
+ PERFORM 040-Process-Yes-No-Value
+ END-IF
+ .
+
+ 059-Done.
+ EXIT.
+ /
+ 060-Process-Yes-No-Auto SECTION.
+ *****************************************************************
+ ** Process a switch value of YES, NO or AUTO **
+ *****************************************************************
+
+ 061-Init.
+ IF SV-1 = 'A'
+ PERFORM 070-Find-LINKAGE-SECTION
+ IF 88-Compile-As-Subpgm
+ MOVE 'Y' TO Switch-Value
+ ELSE
+ MOVE 'N' TO Switch-Value
+ END-IF
+ ELSE
+ PERFORM 040-Process-Yes-No-Value
+ END-IF
+ .
+ /
+ 070-Find-LINKAGE-SECTION SECTION.
+ *****************************************************************
+ ** Determine if the program being compiled is a MAIN program **
+ *****************************************************************
+
+ 071-Init.
+ OPEN INPUT Source-Code
+ SET 88-Compile-As-Mainpgm TO TRUE
+ SET 88-More-To-1st-Prog TO TRUE
+ PERFORM UNTIL 88-1st-Prog-Complete
+ READ Source-Code AT END
+ CLOSE Source-Code
+ EXIT SECTION
+ END-READ
+ CALL 'CHECKSOURCE' USING Source-Code-Record
+ F-Source-Record-Type
+ END-CALL
+ IF 88-Source-Rec-Ident
+ SET 88-1st-Prog-Complete TO TRUE
+ END-IF
+ END-PERFORM
+ .
+
+ 072-Process-Source.
+ SET 88-Source-Rec-IgnoCOB-COLOR-RED TO TRUE
+ PERFORM UNTIL 88-Source-Rec-Linkage
+ OR 88-Source-Rec-Ident
+ READ Source-Code AT END
+ CLOSE Source-Code
+ EXIT SECTION
+ END-READ
+ CALL 'CHECKSOURCE' USING Source-Code-Record
+ F-Source-Record-Type
+ END-CALL
+ END-PERFORM
+ CLOSE Source-Code
+ IF 88-Source-Rec-Linkage
+ SET 88-Compile-As-Subpgm TO TRUE
+ END-IF
+ .
+
+ 079-Done.
+ EXIT.
+ /
+ 100-Initialization SECTION.
+ *****************************************************************
+ ** Perform all program-wide initialization operations **
+ *****************************************************************
+
+
+GC0909 101-Determine-OS-Type.
+GC0909 CALL 'GETOSTYPE'
+GC0909 END-CALL
+GC0909 MOVE RETURN-CODE TO OS-Type
+GC0909 EVALUATE TRUE
+GC0909 WHEN OS-Unknown
+GC0909 MOVE '\' TO Dir-Char
+GC0909 MOVE 'Unknown' TO OS-Type-Literal
+GC0310 MOVE COB-SCR-F11 TO CK-S-F1
+GC0310 MOVE COB-SCR-F12 TO CK-S-F2
+GC0310 MOVE COB-SCR-F13 TO CK-S-F3
+GC0310 MOVE COB-SCR-F14 TO CK-S-F4
+GC0310 MOVE COB-SCR-F15 TO CK-S-F5
+GC0310 MOVE COB-SCR-F16 TO CK-S-F6
+GC0310 MOVE COB-SCR-F17 TO CK-S-F7
+GC0909 WHEN OS-Windows
+GC0909 MOVE '\' TO Dir-Char
+GC0909 MOVE 'Windows' TO OS-Type-Literal
+GC0310 MOVE COB-SCR-F13 TO CK-S-F1
+GC0310 MOVE COB-SCR-F14 TO CK-S-F2
+GC0310 MOVE COB-SCR-F15 TO CK-S-F3
+GC0310 MOVE COB-SCR-F16 TO CK-S-F4
+GC0310 MOVE COB-SCR-F17 TO CK-S-F5
+GC0310 MOVE COB-SCR-F18 TO CK-S-F6
+GC0310 MOVE COB-SCR-F19 TO CK-S-F7
+GC0909 WHEN OS-Cygwin
+GC0909 MOVE '/' TO Dir-Char
+GC0410 MOVE 'Cygwin' TO OS-Type-Literal
+GC0310 MOVE COB-SCR-F11 TO CK-S-F1
+GC0310 MOVE COB-SCR-F12 TO CK-S-F2
+GC0310 MOVE COB-SCR-F13 TO CK-S-F3
+GC0310 MOVE COB-SCR-F14 TO CK-S-F4
+GC0310 MOVE COB-SCR-F15 TO CK-S-F5
+GC0310 MOVE COB-SCR-F16 TO CK-S-F6
+GC0310 MOVE COB-SCR-F17 TO CK-S-F7
+GC0909 WHEN OS-UNIX
+GC0909 MOVE '/' TO Dir-Char
+GC0410 MOVE 'UNIX ' TO OS-Type-Literal
+GC0310 MOVE COB-SCR-F11 TO CK-S-F1
+GC0310 MOVE COB-SCR-F12 TO CK-S-F2
+GC0310 MOVE COB-SCR-F13 TO CK-S-F3
+GC0310 MOVE COB-SCR-F14 TO CK-S-F4
+GC0310 MOVE COB-SCR-F15 TO CK-S-F5
+GC0310 MOVE COB-SCR-F16 TO CK-S-F6
+GC0310 MOVE COB-SCR-F17 TO CK-S-F7
+GC0909 END-EVALUATE
+GC0909 .
+
+ 102-Set-Environment-Vars.
+ SET ENVIRONMENT 'COB_SCREEN_EXCEPTIONS' TO 'Y'
+ SET ENVIRONMENT 'COB_SCREEN_ESC' TO 'Y'
+ .
+
+ 103-Generate-Cobc-Output-Fn.
+ ACCEPT Env-TEMP
+ FROM ENVIRONMENT "TEMP"
+ END-ACCEPT
+ MOVE SPACES TO Cobc-Output-File
+ STRING TRIM(Env-TEMP,TRAILING)
+GC0909 Dir-Char
+GC0909 'OC-Messages.TXT'
+ DELIMITED SIZE
+ INTO Cobc-Output-File
+ END-STRING
+ .
+
+ 104-Generate-Banner-Line-Info.
+ MOVE WHEN-COMPILED (1:12) TO OC-Compiled
+ INSPECT OC-Compiled
+ REPLACING ALL '/' BY ':'
+ AFTER INITIAL SPACE
+ .
+
+ 105-Establish-Switch-Settings.
+ ACCEPT Command-Line-Args
+ FROM COMMAND-LINE
+ END-ACCEPT
+ MOVE TRIM(Command-Line-Args, Leading)
+ TO Command-Line-Args
+ MOVE 0 TO Tally
+GC0410 INSPECT Command-Line-Args TALLYING Tally FOR ALL '@'
+ IF Tally = 0
+ MOVE Command-Line-Args TO File-Name
+ MOVE SPACES TO Command-Line-Args
+ ELSE
+GC0410 UNSTRING Command-Line-Args DELIMITED BY '@'
+ INTO File-Name, Dummy
+ END-UNSTRING
+ INSPECT Command-Line-Args
+GC0410 REPLACING FIRST '@' BY LOW-VALUES
+ UNSTRING Command-Line-Args
+ DELIMITED BY LOW-VALUES
+ INTO Dummy, Cmd
+ END-UNSTRING
+ MOVE SPACES TO Command-Line-Args
+GC0410 STRING '@' Cmd DELIMITED SIZE
+ INTO Command-Line-Args
+ END-STRING
+ END-IF
+ IF File-Name = SPACES
+ DISPLAY
+ 'No program filename was specified'
+ END-DISPLAY
+ PERFORM 900-Terminate
+ END-IF
+ PERFORM 010-Parse-Args
+ IF S-SUBROUTINE = 'A'
+ MOVE 'S' TO Switch-Keyword
+ MOVE 'A' TO Switch-Value
+ PERFORM 070-Find-LINKAGE-SECTION
+ IF 88-Compile-As-Subpgm
+ MOVE 'Y' TO S-SUBROUTINE
+ ELSE
+ MOVE 'N' TO S-SUBROUTINE
+ END-IF
+ END-IF
+ INSPECT S-Yes-No-Switches REPLACING ALL 'Y' BY Selection-Char
+ INSPECT S-Yes-No-Switches REPLACING ALL 'N' BY ' '
+ .
+
+ 106-Determine-Folder-Path.
+ Move 256 TO I
+GC0909 IF OS-Cygwin AND File-Name (2:1) = ':'
+GC0909 MOVE '\' TO Dir-Char
+GC0909 END-IF
+ PERFORM UNTIL I = 0 OR FN-Char (I) = Dir-Char
+ SUBTRACT 1 FROM I
+ END-PERFORM
+ IF I = 0
+ MOVE SPACES TO Prog-Folder
+ MOVE File-Name TO Prog-File-Name
+ ELSE
+ MOVE '*' TO FN-Char (I)
+ UNSTRING File-Name DELIMITED BY '*'
+ INTO Prog-Folder
+ Prog-File-Name
+ END-UNSTRING
+ MOVE Dir-Char TO FN-Char (I)
+ END-IF
+ UNSTRING Prog-File-Name DELIMITED BY '.'
+ INTO Prog-Name, Prog-Extension
+ END-UNSTRING
+ IF Prog-Folder = SPACES
+ ACCEPT Prog-Folder
+ FROM ENVIRONMENT 'CD'
+ END-ACCEPT
+GC0909 ELSE
+GC0909 CALL "CBL_CHANGE_DIR"
+GC0909 USING TRIM(Prog-Folder,TRAILING)
+GC0909 END-CALL
+ END-IF
+GC0909 IF OS-Cygwin AND File-Name (2:1) = ':'
+GC0909 MOVE '/' TO Dir-Char
+GC0909 END-IF
+ .
+
+GC0909 107-Other.
+GC0909 MOVE ALL LD-Horiz-Line TO Horizontal-Line.
+GC0410 MOVE CONCATENATE(' OCic for ',
+GC0410 TRIM(OS-Type-Literal,Trailing),
+GC0410 ' Copyright (C) 2009-2010, Gary L. Cutler,',
+GC0410 ' GPL')
+GC0410 TO Output-Message.
+GC0909 .
+GC0909
+ 109-Done.
+ EXIT.
+ /
+ 200-Let-User-Set-Switches SECTION.
+ *****************************************************************
+ ** Show the user the current switch settings and allow them to **
+ ** be changed. **
+ *****************************************************************
+
+ 201-Init.
+ SET 88-Switch-Changes TO TRUE
+ .
+
+ 202-Show-And-Change-Switches.
+ PERFORM UNTIL 88-No-Switch-Changes
+ ACCEPT
+ Switches-Screen
+ END-ACCEPT
+ IF COB-CRT-STATUS > 0
+ EVALUATE COB-CRT-STATUS
+ WHEN COB-SCR-F1
+ IF S-DEBUG = SPACE
+ MOVE Selection-Char TO S-DEBUG
+ ELSE
+ MOVE ' ' TO S-DEBUG
+ END-IF
+ WHEN COB-SCR-F2
+ IF S-DLL = SPACE
+ MOVE Selection-Char TO S-DLL
+ ELSE
+ MOVE ' ' TO S-DLL
+ END-IF
+ WHEN COB-SCR-F3
+ IF S-SUBROUTINE = SPACE
+ MOVE Selection-Char TO S-SUBROUTINE
+ MOVE ' ' TO S-EXECUTE
+ ELSE
+ MOVE ' ' TO S-SUBROUTINE
+ END-IF
+ WHEN COB-SCR-F4
+ IF S-EXECUTE = SPACE
+ AND S-SUBROUTINE = SPACE
+ MOVE Selection-Char TO S-EXECUTE
+ ELSE
+ MOVE ' ' TO S-EXECUTE
+ END-IF
+ WHEN COB-SCR-F5
+ IF S-NOTRUNC = SPACE
+ MOVE Selection-Char TO S-NOTRUNC
+ ELSE
+ MOVE ' ' TO S-NOTRUNC
+ END-IF
+ WHEN COB-SCR-F6
+ IF S-TRACE = SPACE
+ MOVE Selection-Char TO S-TRACE
+ MOVE ' ' TO S-TRACEALL
+ ELSE
+ MOVE ' ' TO S-TRACE
+ END-IF
+ WHEN COB-SCR-F7
+ IF S-TRACEALL = SPACE
+ MOVE Selection-Char TO S-TRACEALL
+ MOVE ' ' TO S-TRACE
+ ELSE
+ MOVE ' ' TO S-TRACEALL
+ END-IF
+GC0410 WHEN COB-SCR-F8
+GC0410 IF S-SOURCE = SPACE
+GC0410 MOVE Selection-Char TO S-SOURCE
+GC0410 ELSE
+GC0410 MOVE ' ' TO S-SOURCE
+GC0410 END-IF
+GC0410 WHEN COB-SCR-F9
+GC0410 IF S-XREF = SPACE
+GC0410 MOVE Selection-Char TO S-XREF
+GC0410 ELSE
+GC0410 MOVE ' ' TO S-XREF
+GC0410 END-IF
+ WHEN COB-SCR-ESC
+ PERFORM 900-Terminate
+GC0310 WHEN CK-S-F1
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-BS2000
+GC0310 WHEN CK-S-F2
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-COBOL85
+GC0310 WHEN CK-S-F3
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-COBOL2002
+GC0310 WHEN CK-S-F4
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-DEFAULT
+GC0310 WHEN CK-S-F5
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-IBM
+GC0310 WHEN CK-S-F6
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-MF
+GC0310 WHEN CK-S-F7
+ MOVE SPACES TO S-CfgS
+ MOVE Selection-Char TO S-Cfg-MVS
+ WHEN OTHER
+ MOVE 'An unsupported key was pressed'
+ TO Output-Message
+ END-EVALUATE
+ ELSE
+ SET 88-No-Switch-Changes TO TRUE
+ END-IF
+ END-PERFORM
+ .
+
+ 209-Done.
+ EXIT.
+ /
+ 210-Run-Compiler SECTION.
+ *****************************************************************
+ ** Run the compiler using the switch settings we've prepared. **
+ *****************************************************************
+
+ 211-Init.
+ MOVE SPACES TO Cmd
+ Cobc-Cmd
+ Output-Message
+ DISPLAY
+ Switches-Screen
+ END-DISPLAY
+ MOVE 1 TO I
+ EVALUATE TRUE
+ WHEN S-Cfg-BS2000 NOT = SPACES
+ MOVE 'bs2000' TO Config-File
+ WHEN S-Cfg-COBOL85 NOT = SPACES
+ MOVE 'cobol85' TO Config-File
+ WHEN S-Cfg-COBOL2002 NOT = SPACES
+ MOVE 'cobol2002' TO Config-File
+ WHEN S-Cfg-IBM NOT = SPACES
+ MOVE 'ibm' TO Config-File
+ WHEN S-Cfg-MF NOT = SPACES
+ MOVE 'mf' TO Config-File
+ WHEN S-Cfg-MVS NOT = SPACES
+ MOVE 'mvs' TO Config-File
+ WHEN OTHER
+ MOVE 'default' TO Config-File
+ END-EVALUATE
+ .
+
+ 212-Build-Compile-Command.
+GC0909 MOVE SPACES TO Cobc-Cmd
+GC0909 STRING 'cobc -std='
+GC0909 TRIM(Config-File,TRAILING)
+GC0909 ' '
+GC0909 INTO Cobc-Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+ IF S-SUBROUTINE NOT = ' '
+ STRING '-m '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ ELSE
+ STRING '-x '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF S-DEBUG NOT = ' '
+ STRING '-fdebugging-line '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF S-NOTRUNC NOT = ' '
+ STRING '-fnotrunc '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF S-TRACEALL NOT = ' '
+GC0809 STRING '-ftraceall '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF S-TRACE NOT = ' '
+ STRING '-ftrace '
+ DELIMITED SIZE INTO Cobc-Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+
+GC0709 IF S-EXTRA > SPACES
+GC0709 STRING ' '
+GC0709 TRIM(S-Extra,TRAILING)
+GC0709 ' '
+GC0709 DELIMITED SIZE INTO Cobc-Cmd
+GC0709 WITH POINTER I
+GC0709 END-STRING
+GC0709 END-IF
+GC0909 STRING TRIM(Prog-File-Name,TRAILING)
+GC0909 DELIMITED SIZE INTO Cobc-Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+ .
+
+ 213-Run-Compiler.
+GC0410 MOVE ' Compiling...' TO Output-Message
+GC0410 DISPLAY
+GC0410 Switches-Screen
+GC0410 END-DISPLAY
+GC0609 SET 88-Output-File-Avail TO TRUE
+ MOVE SPACES TO Cmd
+ STRING TRIM(Cobc-Cmd,TRAILING)
+ ' 2>'
+ TRIM(Cobc-Output-File,TRAILING)
+ DELIMITED SIZE
+ INTO Cmd
+ END-STRING
+ CALL 'SYSTEM'
+ USING TRIM(Cmd,TRAILING)
+ END-CALL
+GC0909 IF RETURN-CODE = 0
+GC0909 SET 88-Compile-OK TO TRUE
+GC0909 ELSE
+GC0909 SET 88-Compile-Failed TO TRUE
+GC0909 END-IF
+GC0909 IF 88-Compile-OK
+GC0909 OPEN INPUT Cobc-Output
+GC0909 READ Cobc-Output
+GC0909 AT END
+GC0909 CONTINUE
+GC0909 NOT AT END
+GC0909 SET 88-Compile-OK-Warn TO TRUE
+GC0909 END-READ
+GC0909 CLOSE Cobc-Output
+GC0909 END-IF
+GC0909 MOVE SPACES TO Output-Message
+ IF 88-Compile-OK
+GC0909 MOVE ' Compilation Was Successful' TO Output-Message
+GC0909 DISPLAY
+GC0909 Switches-Screen
+GC0909 END-DISPLAY
+GC0909 CALL 'C$SLEEP'
+GC0909 USING 2
+GC0909 END-CALL
+GC0909 MOVE SPACES TO Output-Message
+GC0609 SET 88-Complete TO TRUE
+ ELSE
+GC0909 DISPLAY
+GC0909 Blank-Screen
+GC0909 END-DISPLAY
+GC0909 IF 88-Compile-OK-Warn
+GC0909 DISPLAY ' Compilation was successful, but ' &
+GC0909 'warnings were generated:'
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 1 LINE
+GC0909 END-DISPLAY
+GC0909 ELSE
+GC0909 DISPLAY 'Compilation Failed:'
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 1 LINE
+GC0909 END-DISPLAY
+GC0909 END-IF
+GC0609 SET 88-Compile-Failed TO TRUE
+GC0609 SET 88-Complete TO TRUE
+GC0909 DISPLAY ' '
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 1 LINE
+GC0909 END-DISPLAY
+GC0909 OPEN INPUT Cobc-Output
+GC0909 PERFORM FOREVER
+GC0909 READ Cobc-Output AT END
+GC0909 EXIT PERFORM
+GC0909 END-READ
+GC0909 DISPLAY TRIM(Cobc-Output-Rec,TRAILING)
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 1 LINE
+GC0909 END-DISPLAY
+GC0909 END-PERFORM
+GC0909 CLOSE Cobc-Output
+GC0909 DISPLAY ' '
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 2 LINES
+GC0909 END-DISPLAY
+GC0909 DISPLAY 'Press ENTER to close:'
+SCROLL* AT LINE 24 COLUMN 1
+SCROLL* WITH SCROLL UP 1 LINE
+GC0909 END-DISPLAY
+GC0909 ACCEPT Dummy
+GC0909 FROM CONSOLE
+GC0909 END-ACCEPT
+GC0909 DISPLAY
+GC0909 Blank-Screen
+GC0909 END-DISPLAY
+ END-IF
+ .
+
+ 219-Done.
+ IF 88-Compile-Failed
+ PERFORM 900-Terminate
+ END-IF
+ .
+ /
+GC0410 220-Make-Listing SECTION.
+GC0410*****************************************************************
+GC0410** Generate a source and/or xref listing using XREF **
+GC0410*****************************************************************
+GC0410
+GC0410 221-Init.
+GC0410 MOVE ' Generating cross-reference listing...'
+GC0410 TO Output-Message
+GC0410 DISPLAY
+GC0410 Switches-Screen
+GC0410 END-DISPLAY
+GC0410 CALL "CBL_DELETE_FILE"
+GC0410 USING CONCATENATE(TRIM(Prog-Name,Trailing),".lst")
+GC0410 END-CALL
+GC0410 MOVE 0 TO RETURN-CODE
+GC0410 .
+GC0410
+GC0410 213-Run-OCXref.
+GC0410 MOVE SPACES TO Output-Message
+GC0410 CALL 'LISTING'
+GC0410 USING S-SOURCE
+GC0410 S-XREF
+GC0410 File-Name
+GC0410 ON EXCEPTION
+GC0410 MOVE ' LISTING module is not available'
+GC0410 TO Output-Message
+GC0410 MOVE 1 TO RETURN-CODE
+GC0410 END-CALL
+GC0410 IF RETURN-CODE = 0
+GC0410 MOVE ' Listing generated'
+GC0410 TO Output-Message
+GC0410 IF OS-Windows OR OS-Cygwin
+GC0410 MOVE SPACES TO Cmd
+GC0410 STRING
+GC0410 'cmd /c '
+GC0410 TRIM(Prog-Name,TRAILING)
+GC0410 '.lst'
+GC0410 DELIMITED SIZE INTO Cmd
+GC0410 END-STRING
+GC0410 CALL 'SYSTEM'
+GC0410 USING TRIM(Cmd,TRAILING)
+GC0410 END-CALL
+GC0410 END-IF
+GC0410 ELSE
+GC0410 IF Output-Message = SPACES
+GC0410 MOVE ' Listing generation failed'
+GC0410 TO Output-Message
+GC0410 END-IF
+GC0410 END-IF
+GC0410 DISPLAY
+GC0410 Switches-Screen
+GC0410 END-DISPLAY
+GC0410 CALL 'C$SLEEP'
+GC0410 USING 2
+GC0410 END-CALL
+GC0410 .
+ /
+ 230-Run-Program SECTION.
+ *****************************************************************
+ ** Run the compiled program **
+ *****************************************************************
+
+ 232-Build-Command.
+GC0909 MOVE SPACES TO Cmd
+GC0909 MOVE 1 TO I
+ IF S-SUBROUTINE NOT = ' '
+ OR S-DLL NOT = ' '
+ STRING 'cobcrun ' DELIMITED SIZE
+ INTO Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF Prog-Folder NOT = SPACES
+GC0909 IF OS-Cygwin AND Prog-Folder (2:1) = ':'
+GC0909 STRING '/cygdrive/'
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 STRING LOWER-CASE(Prog-Folder (1:1))
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 PERFORM VARYING J FROM 3 BY 1
+GC0909 UNTIL J > LENGTH(TRIM(Prog-Folder))
+GC0909 IF Prog-Folder (J:1) = '\'
+GC0909 STRING '/'
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 ELSE
+GC0909 STRING Prog-Folder (J:1)
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 END-IF
+GC0909 END-PERFORM
+GC0909 ELSE
+GC0410 STRING '"' TRIM(Prog-Folder,TRAILING)
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 END-IF
+GC0909 STRING Dir-Char
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 ELSE
+GC0909 IF OS-Cygwin OR OS-UNIX
+GC0909 STRING './'
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 END-IF
+ END-IF
+GC0909 STRING TRIM(Prog-Name,TRAILING)
+GC0909 INTO Cmd
+GC0909 WITH POINTER I
+GC0909 END-STRING
+GC0909 IF S-SUBROUTINE = ' '
+GC0909 AND S-DLL NOT = ' '
+GC0909 STRING '.exe' DELIMITED SIZE
+ INTO Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF S-ARGS NOT = SPACES
+GC0809 STRING ' ' TRIM(S-ARGS,TRAILING)
+ INTO Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ IF OS-Unknown OR OS-Windows
+GC0410 STRING '"&&pause'
+ INTO Cmd
+ WITH POINTER I
+ END-STRING
+ ELSE
+ STRING ';echo "Press ENTER to close...";read'
+ INTO Cmd
+ WITH POINTER I
+ END-STRING
+ END-IF
+ .
+
+ 233-Run-Program.
+GC0909 DISPLAY
+GC0909 Blank-Screen
+GC0909 END-DISPLAY
+
+ CALL 'SYSTEM'
+ USING TRIM(Cmd,TRAILING)
+ END-CALL
+ PERFORM 900-Terminate
+ .
+
+ 239-Done.
+ EXIT.
+ /
+ 900-Terminate SECTION.
+ *****************************************************************
+ ** Display a message and halt the program **
+ *****************************************************************
+
+ 901-Display-Message.
+GC0909 IF Output-Message > SPACES
+GC0909 DISPLAY
+GC0909 Switches-Screen
+GC0909 END-DISPLAY
+GC0909 CALL 'C$SLEEP'
+GC0909 USING 2
+GC0909 END-CALL
+GC0909 END-IF
+ DISPLAY
+ Blank-Screen
+ END-DISPLAY
+ .
+
+ 909-Done.
+ GOBACK
+ .
+
+ END PROGRAM OCic.
+
+ IDENTIFICATION DIVISION.
+ PROGRAM-ID. GETOSTYPE.
+ *****************************************************************
+ ** This subprogram determine the OS type the program is run- **
+ ** ning under, passing that result back in RETURN-CODE as fol- **
+ ** lows: **
+ ** **
+ ** 0: Cannot be determined **
+ ** 1: Native Windows or Windows/MinGW **
+ ** 2: Cygwin **
+ ** 3: UNIX/Linux/MacOS **
+ *****************************************************************
+ ** DATE CHANGE DESCRIPTION **
+ ** ====== ==================================================== **
+ ** GC0909 Initial coding. **
+ *****************************************************************
+ ENVIRONMENT DIVISION.
+ CONFIGURATION SECTION.
+ REPOSITORY.
+ FUNCTION ALL INTRINSIC.
+ DATA DIVISION.
+ WORKING-STORAGE SECTION.
+ 01 Env-Path PIC X(1024).
+ 01 Tally USAGE BINARY-LONG.
+ PROCEDURE DIVISION.
+ 000-Main SECTION.
+ 010-Get-TEMP-Var.
+ MOVE SPACES TO Env-Path
+ ACCEPT Env-Path
+ FROM ENVIRONMENT "PATH"
+ ON EXCEPTION
+ MOVE 0 TO RETURN-CODE
+ GOBACK
+ END-ACCEPT
+ IF Env-Path = SPACES
+ MOVE 0 TO RETURN-CODE
+ ELSE
+ MOVE 0 TO Tally
+ INSPECT Env-Path
+ TALLYING Tally FOR ALL ";"
+ IF Tally = 0 *> Must be some form of UNIX
+ MOVE 0 TO Tally
+ INSPECT Env-Path
+ TALLYING TALLY FOR ALL "/cygdrive/"
+ IF Tally = 0 *> UNIX/MacOS
+ MOVE 3 TO RETURN-CODE
+ ELSE *> Cygwin
+ MOVE 2 TO RETURN-CODE
+ END-IF
+ ELSE *> Assume Windows[/MinGW]
+ MOVE 1 TO RETURN-CODE
+ END-IF
+ END-IF
+ GOBACK
+ .
+ END PROGRAM GETOSTYPE.
+
+ IDENTIFICATION DIVISION.
+ PROGRAM-ID. CHECKSOURCE.
+ *****************************************************************
+ ** This subprogram will scan a line of source code it is given **
+ ** looking for "LINKAGE SECTION" or "IDENTIFICATION DIVISION". **
+ ** **
+ ** ****NOTE**** ****NOTE**** ****NOTE**** ****NOTE*** **
+ ** **
+ ** These two strings must be found IN THEIR ENTIRETY within **
+ ** the 1st 80 columns of program source records, and cannot **
+ ** follow either a "*>" sequence OR a "*" in col 7. **
+ *****************************************************************
+ ** DATE CHANGE DESCRIPTION **
+ ** ====== ==================================================== **
+ ** GC0809 Initial coding. **
+ *****************************************************************
+ ENVIRONMENT DIVISION.
+ CONFIGURATION SECTION.
+ REPOSITORY.
+ FUNCTION ALL INTRINSIC.
+ DATA DIVISION.
+ WORKING-STORAGE SECTION.
+ 01 Compressed-Src.
+ 05 CS-Char OCCURS 80 TIMES PIC X(1).
+
+ 01 Flags.
+ 05 F-Found-SPACE PIC X(1).
+ 88 88-Skipping-SPACE VALUE 'Y'.
+ 88 88-Not-Skipping-SPACE VALUE 'N'.
+
+ 01 I USAGE BINARY-CHAR.
+
+ 01 J USAGE BINARY-CHAR.
+ LINKAGE SECTION.
+ 01 Argument-1.
+ 02 A1-Char OCCURS 80 TIMES PIC X(1).
+
+ 01 Argument-2 PIC X(1).
+ 88 88-A2-LINKAGE-SECTION VALUE 'L'.
+ 88 88-A2-IDENTIFICATION-DIVISION VALUE 'I'.
+ 88 88-A2-Nothing-Special VALUE ' '.
+ PROCEDURE DIVISION USING Argument-1, Argument-2.
+ 000-Main SECTION.
+
+ 010-Initialize.
+ SET 88-A2-Nothing-Special TO TRUE
+ IF A1-Char (7) = '*'
+ GOBACK
+ END-IF
+ .
+
+ 020-Compress-Multiple-SPACES.
+ SET 88-Not-Skipping-SPACE TO TRUE
+ MOVE 0 TO J
+ MOVE SPACES TO Compressed-Src
+ PERFORM VARYING I FROM 1 BY 1
+ UNTIL I > 80
+ IF A1-Char (I) = SPACE
+ IF 88-Not-Skipping-SPACE
+ ADD 1 TO J
+ MOVE UPPER-CASE(A1-Char (I)) TO CS-Char (J)
+ SET 88-Skipping-SPACE TO TRUE
+ END-IF
+ ELSE
+ SET 88-Not-Skipping-SPACE TO TRUE
+ ADD 1 TO J
+ MOVE A1-Char (I) TO CS-Char (J)
+ END-IF
+ END-PERFORM
+ .
+
+ 030-Scan-Compressed-Src.
+ PERFORM VARYING I FROM 1 BY 1
+ UNTIL I > 66
+ EVALUATE TRUE
+ WHEN CS-Char (I) = '*'
+ IF Compressed-Src (I : 2) = '*>'
+ GOBACK
+ END-IF
+ WHEN (CS-Char (I) = 'L') AND (I < 66)
+ IF Compressed-Src (I : 15) = 'LINKAGE SECTION'
+ SET 88-A2-LINKAGE-SECTION TO TRUE
+ GOBACK
+ END-IF
+ WHEN (CS-Char (I) = 'I') AND (I < 58)
+ IF Compressed-Src (I : 23) = 'IDENTIFICATION ' &
+ 'DIVISION'
+ SET 88-A2-IDENTIFICATION-DIVISION TO TRUE
+ GOBACK
+ END-IF
+ END-EVALUATE
+ END-PERFORM
+ .
+
+ 099-Never-Found-Either-One.
+ GOBACK
+ .
+ END PROGRAM CHECKSOURCE.
+
+ IDENTIFICATION DIVISION.
+ PROGRAM-ID. LISTING.
+ *****************************************************************
+ ** This subprogram generates a cross-reference listing of an **
+ ** OpenCOBOL program. **
+ ** **
+ ** Linkage: CALL "LISTING" USING <source> **
+ ** <xref> **
+ ** <filename> **
+ ** **
+ ** Where: **
+ ** <source> is a PIC X(1) flag indicating **
+ ** whether or not a source listing **
+ ** should be produced (space=NO, **
+ ** non-space=yes) **
+ ** <xref> is a PIC X(1) flag indicating **
+ ** whether or not an xref listing **
+ ** should be produced (space=NO, **
+ ** non-space=yes) **
+ ** <filename> is the [path]filename of the **
+ ** program being listed and/or **
+ ** xreffed in a PIC X(256) form. **
+ *****************************************************************
+ ** **
+ ** AUTHOR: GARY L. CUTLER **
+ ** CutlerGL@gmail.com **
+ ** Copyright (C) 2010, Gary L. Cutler, GPL **
+ ** **
+ ** DATE-WRITTEN: April 1, 2010 **
+ ** **
+ *****************************************************************
+ ** DATE CHANGE DESCRIPTION **
+ ** ====== ==================================================== **
+ ** GC0410 Initial coding **
+ ** GC0710 Handle duplicate data names (i.e. "CORRESPONDING" or **
+ ** qualified items) better; ignore "END PROGRAM" recs **
+ ** so program name doesn't appear in listing. **
+ *****************************************************************
+ ENVIRONMENT DIVISION.
+ CONFIGURATION SECTION.
+ REPOSITORY.
+ FUNCTION ALL INTRINSIC.
+ INPUT-OUTPUT SECTION.
+ FILE-CONTROL.
+ SELECT Expand-Code ASSIGN TO Expanded-Src-Filename
+ ORGANIZATION IS LINE SEQUENTIAL.
+ SELECT Report-File ASSIGN TO Report-Filename
+ ORGANIZATION IS LINE SEQUENTIAL.
+ SELECT Sort-File ASSIGN TO DISK.
+ SELECT Source-Code ASSIGN TO Src-Filename
+ ORGANIZATION IS LINE SEQUENTIAL.
+ DATA DIVISION.
+ FILE SECTION.
+ FD Expand-Code.
+ 01 Expand-Code-Rec.
+ 05 ECR-1 PIC X.
+ 05 ECR-2-256 PIC X(256).
+ 01 Expand-Code-Rec-Alt.
+ 05 ECR-1-128 PIC X(128).
+ 05 ECR-129-256 PIC X(128).
+
+ FD Report-File.
+ 01 Report-Rec PIC X(135).
+
+ SD Sort-File.
+ 01 Sort-Rec.
+ 05 SR-Prog-ID PIC X(15).
+ 05 SR-Token-UC PIC X(32).
+ 05 SR-Token PIC X(32).
+ 05 SR-Section PIC X(15).
+ 05 SR-Line-No-Def PIC 9(6).
+ 05 SR-Reference.
+ 10 SR-Line-No-Ref PIC 9(6).
+ 10 SR-Ref-Flag PIC X(1).
+
+ FD Source-Code.
+ 01 Source-Code-Rec.
+GC0410 05 SCR-1-128.
+GC0410 10 FILLER PIC X(6).
+GC0410 10 SCR-7 PIC X(1).
+GC0410 10 FILLER PIC X(121).
+ 05 SCR-129-256 PIC X(128).
+
+ WORKING-STORAGE SECTION.
+ 78 Line-Nos-Per-Rec VALUE 8.
+
+ 01 Cmd PIC X(256).
+
+ 01 Delim PIC X(2).
+
+ 01 Detail-Line-S.
+ 05 DLS-Line-No PIC ZZZZZ9.
+ 05 FILLER PIC X(1).
+ 05 DLS-Statement PIC X(128).
+
+ 01 Detail-Line-X.
+ 05 DLX-Prog-ID PIC X(15).
+ 05 FILLER PIC X(1).
+ 05 DLX-Token PIC X(32).
+ 05 FILLER PIC X(1).
+ 05 DLX-Line-No-Def PIC ZZZZZ9.
+ 05 FILLER PIC X(1).
+ 05 DLX-Section PIC X(15).
+ 05 FILLER PIC X(1).
+ 05 DLX-Reference OCCURS Line-Nos-Per-Rec TIMES.
+ 10 DLX-Line-No-Ref PIC ZZZZZ9.
+ 10 DLX-Ref-Flag PIC X(1).
+ 10 FILLER PIC X(1).
+
+ 01 Dummy PIC X(1).
+
+ 01 Env-TEMP PIC X(256).
+
+ 01 Expanded-Src-Filename PIC X(256).
+
+ 01 Filename PIC X(256).
+
+ 01 Flags.
+GC0710 05 F-Duplicate PIC X(1).
+ 05 F-First-Record PIC X(1).
+ 05 F-In-Which-Pgm PIC X(1).
+ 88 In-Main-Module VALUE 'M'.
+ 88 In-Copybook VALUE 'C'.
+ 05 F-Last-Token-Ended-Sent PIC X(1).
+ 05 F-Processing-PICTURE PIC X(1).
+ 05 F-Token-Ended-Sentence PIC X(1).
+GC0710 05 F-Verb-Has-Been-Found PIC X(1).
+
+ 01 Group-Indicators.
+ 05 GI-Prog-ID PIC X(15).
+ 05 GI-Token PIC X(32).
+
+ 01 Heading-1S.
+ 05 FILLER PIC X(125) VALUE
+ "OpenCOBOL 1.1 06FEB2009 Source Listing - " &
+ "OCic Copyright (C) 2009-2010, Gary L. Cutler, GPL".
+ 05 H1S-Date PIC 9999/99/99.
+
+ 01 Heading-1X.
+ 05 FILLER PIC X(125) VALUE
+ "OpenCOBOL 1.1 06FEB2009 Cross-Reference Listing - " &
+ "OCic Copyright (C) 2009-2010, Gary L. Cutler, GPL".
+ 05 H1X-Date PIC 9999/99/99.
+
+ 01 Heading-2 PIC X(135).
+
+ 01 Heading-4S PIC X(16) VALUE
+ "Line Statement".
+
+ 01 Heading-4X PIC X(96) VALUE
+ "PROGRAM-ID Identifier/Register/Function Defn Wher
+ - "e Defined References (* = Updated)".
+
+ 01 Heading-5S PIC X(135) VALUE
+ "====== =====================================================
+ - "============================================================
+ - "===============".
+
+ 01 Heading-5X PIC X(135) VALUE
+ "=============== ================================ ====== ====
+ - "=========== ================================================
+ - "===============".
+
+ 01 Held-Reference PIC X(100).
+
+ 01 I USAGE BINARY-LONG.
+
+ 01 J USAGE BINARY-LONG.
+
+ 01 Lines-Left USAGE BINARY-LONG.
+
+ 01 Lines-Per-Page USAGE BINARY-LONG.
+
+ 01 Lines-Per-Page-ENV PIC X(256).
+
+ 01 Num-UserNames USAGE BINARY-LONG.
+
+ 01 PIC-X10 PIC X(10).
+
+ 01 PIC-X32 PIC X(32).
+
+ 01 PIC-X256 PIC X(256).
+
+ 01 Program-Path PIC X(256).
+
+ 01 Report-Filename PIC X(256).
+
+ 01 Reserved-Words.
+ 05 FILLER PIC X(33) VALUE "IABS".
+ 05 FILLER PIC X(33) VALUE "VACCEPT".
+ 05 FILLER PIC X(33) VALUE " ACCESS".
+ 05 FILLER PIC X(33) VALUE "IACOS".
+ 05 FILLER PIC X(33) VALUE " ACTIVE-CLASS".
+ 05 FILLER PIC X(33) VALUE "VADD".
+ 05 FILLER PIC X(33) VALUE " ADDRESS".
+ 05 FILLER PIC X(33) VALUE " ADVANCING".
+ 05 FILLER PIC X(33) VALUE "KAFTER".
+ 05 FILLER PIC X(33) VALUE " ALIGNED".
+ 05 FILLER PIC X(33) VALUE " ALL".
+ 05 FILLER PIC X(33) VALUE "VALLOCATE".
+ 05 FILLER PIC X(33) VALUE " ALPHABET".
+ 05 FILLER PIC X(33) VALUE " ALPHABETIC".
+ 05 FILLER PIC X(33) VALUE " ALPHABETIC-LOWER".
+ 05 FILLER PIC X(33) VALUE " ALPHABETIC-UPPER".
+ 05 FILLER PIC X(33) VALUE " ALPHANUMERIC".
+ 05 FILLER PIC X(33) VALUE " ALPHANUMERIC-EDITED".
+ 05 FILLER PIC X(33) VALUE " ALSO".
+ 05 FILLER PIC X(33) VALUE "VALTER".
+ 05 FILLER PIC X(33) VALUE " ALTERNATE".
+ 05 FILLER PIC X(33) VALUE " AND".
+ 05 FILLER PIC X(33) VALUE "IANNUITY".
+ 05 FILLER PIC X(33) VALUE " ANY".
+ 05 FILLER PIC X(33) VALUE " ANYCASE".
+ 05 FILLER PIC X(33) VALUE " ARE".
+ 05 FILLER PIC X(33) VALUE " AREA".
+ 05 FILLER PIC X(33) VALUE " AREAS".
+ 05 FILLER PIC X(33) VALUE " ARGUMENT-NUMBER".
+ 05 FILLER PIC X(33) VALUE " ARGUMENT-VALUE".
+ 05 FILLER PIC X(33) VALUE " AS".
+ 05 FILLER PIC X(33) VALUE " ASCENDING".
+ 05 FILLER PIC X(33) VALUE "IASIN".
+ 05 FILLER PIC X(33) VALUE " ASSIGN".
+ 05 FILLER PIC X(33) VALUE " AT".
+ 05 FILLER PIC X(33) VALUE "IATAN".
+ 05 FILLER PIC X(33) VALUE " AUTHOR".
+ 05 FILLER PIC X(33) VALUE " AUTO".
+ 05 FILLER PIC X(33) VALUE " AUTO-SKIP".
+ 05 FILLER PIC X(33) VALUE " AUTOMATIC".
+ 05 FILLER PIC X(33) VALUE " AUTOTERMINATE".
+ 05 FILLER PIC X(33) VALUE " BACKGROUND-COLOR".
+ 05 FILLER PIC X(33) VALUE " BASED".
+ 05 FILLER PIC X(33) VALUE " BEEP".
+ 05 FILLER PIC X(33) VALUE " BEFORE".
+ 05 FILLER PIC X(33) VALUE " BELL".
+ 05 FILLER PIC X(33) VALUE " BINARY".
+ 05 FILLER PIC X(33) VALUE " BINARY-C-LONG".
+ 05 FILLER PIC X(33) VALUE " BINARY-CHAR".
+ 05 FILLER PIC X(33) VALUE " BINARY-DOUBLE".
+ 05 FILLER PIC X(33) VALUE " BINARY-LONG".
+ 05 FILLER PIC X(33) VALUE " BINARY-SHORT".
+ 05 FILLER PIC X(33) VALUE " BIT".
+ 05 FILLER PIC X(33) VALUE " BLANK".
+ 05 FILLER PIC X(33) VALUE " BLINK".
+ 05 FILLER PIC X(33) VALUE " BLOCK".
+ 05 FILLER PIC X(33) VALUE " BOOLEAN".
+ 05 FILLER PIC X(33) VALUE " BOTTOM".
+ 05 FILLER PIC X(33) VALUE "YBY".
+ 05 FILLER PIC X(33) VALUE "IBYTE-LENGTH".
+ 05 FILLER PIC X(33) VALUE "MC01".
+ 05 FILLER PIC X(33) VALUE "MC02".
+ 05 FILLER PIC X(33) VALUE "MC03".
+ 05 FILLER PIC X(33) VALUE "MC04".
+ 05 FILLER PIC X(33) VALUE "MC05".
+ 05 FILLER PIC X(33) VALUE "MC06".
+ 05 FILLER PIC X(33) VALUE "MC07".
+ 05 FILLER PIC X(33) VALUE "MC08".
+ 05 FILLER PIC X(33) VALUE "MC09".
+ 05 FILLER PIC X(33) VALUE "MC10".
+ 05 FILLER PIC X(33) VALUE "MC11".
+ 05 FILLER PIC X(33) VALUE "MC12".
+ 05 FILLER PIC X(33) VALUE "VCALL".
+ 05 FILLER PIC X(33) VALUE "VCANCEL".
+ 05 FILLER PIC X(33) VALUE " CF".
+ 05 FILLER PIC X(33) VALUE " CH".
+ 05 FILLER PIC X(33) VALUE " CHAINING".
+ 05 FILLER PIC X(33) VALUE "ICHAR".
+ 05 FILLER PIC X(33) VALUE " CHARACTER".
+ 05 FILLER PIC X(33) VALUE " CHARACTERS".
+ 05 FILLER PIC X(33) VALUE " CLASS".
+ 05 FILLER PIC X(33) VALUE " CLASS-ID".
+ 05 FILLER PIC X(33) VALUE "VCLOSE".
+ 05 FILLER PIC X(33) VALUE "ICOB-CRT-STATUS".
+ 05 FILLER PIC X(33) VALUE " CODE".
+ 05 FILLER PIC X(33) VALUE " CODE-SET".
+ 05 FILLER PIC X(33) VALUE " COL".
+ 05 FILLER PIC X(33) VALUE " COLLATING".
+ 05 FILLER PIC X(33) VALUE " COLS".
+ 05 FILLER PIC X(33) VALUE " COLUMN".
+ 05 FILLER PIC X(33) VALUE " COLUMNS".
+ 05 FILLER PIC X(33) VALUE "ICOMBINED-DATETIME".
+ 05 FILLER PIC X(33) VALUE " COMMA".
+ 05 FILLER PIC X(33) VALUE " COMMAND-LINE".
+ 05 FILLER PIC X(33) VALUE "VCOMMIT".
+ 05 FILLER PIC X(33) VALUE " COMMON".
+ 05 FILLER PIC X(33) VALUE " COMP".
+ 05 FILLER PIC X(33) VALUE " COMP-1".
+ 05 FILLER PIC X(33) VALUE " COMP-2".
+ 05 FILLER PIC X(33) VALUE " COMP-3".
+ 05 FILLER PIC X(33) VALUE " COMP-4".
+ 05 FILLER PIC X(33) VALUE " COMP-5".
+ 05 FILLER PIC X(33) VALUE " COMP-X".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-1".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-2".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-3".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-4".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-5".
+ 05 FILLER PIC X(33) VALUE " COMPUTATIONAL-X".
+ 05 FILLER PIC X(33) VALUE "VCOMPUTE".
+ 05 FILLER PIC X(33) VALUE "ICONCATENATE".
+ 05 FILLER PIC X(33) VALUE " CONDITION".
+ 05 FILLER PIC X(33) VALUE "KCONFIGURATION".
+ 05 FILLER PIC X(33) VALUE "MCONSOLE".
+ 05 FILLER PIC X(33) VALUE " CONSTANT".
+ 05 FILLER PIC X(33) VALUE " CONTAINS".
+ 05 FILLER PIC X(33) VALUE " CONTENT".
+ 05 FILLER PIC X(33) VALUE "VCONTINUE".
+ 05 FILLER PIC X(33) VALUE " CONTROL".
+ 05 FILLER PIC X(33) VALUE " CONTROLS".
+ 05 FILLER PIC X(33) VALUE "KCONVERTING".
+ 05 FILLER PIC X(33) VALUE " COPY".
+ 05 FILLER PIC X(33) VALUE " CORR".
+ 05 FILLER PIC X(33) VALUE " CORRESPONDING".
+ 05 FILLER PIC X(33) VALUE "ICOS".
+ 05 FILLER PIC X(33) VALUE "KCOUNT".
+ 05 FILLER PIC X(33) VALUE " CRT".
+ 05 FILLER PIC X(33) VALUE " CURRENCY".
+ 05 FILLER PIC X(33) VALUE "ICURRENT-DATE".
+ 05 FILLER PIC X(33) VALUE " CURSOR".
+ 05 FILLER PIC X(33) VALUE " CYCLE".
+ 05 FILLER PIC X(33) VALUE "KDATA".
+ 05 FILLER PIC X(33) VALUE " DATA-POINTER".
+ 05 FILLER PIC X(33) VALUE " DATE".
+ 05 FILLER PIC X(33) VALUE " DATE-COMPILED".
+ 05 FILLER PIC X(33) VALUE " DATE-MODIFIED".
+ 05 FILLER PIC X(33) VALUE "IDATE-OF-INTEGER".
+ 05 FILLER PIC X(33) VALUE "IDATE-TO-YYYYMMDD".
+ 05 FILLER PIC X(33) VALUE " DATE-WRITTEN".
+ 05 FILLER PIC X(33) VALUE " DAY".
+ 05 FILLER PIC X(33) VALUE "IDAY-OF-INTEGER".
+ 05 FILLER PIC X(33) VALUE " DAY-OF-WEEK".
+ 05 FILLER PIC X(33) VALUE "IDAY-TO-YYYYDDD".
+ 05 FILLER PIC X(33) VALUE " DE".
+ 05 FILLER PIC X(33) VALUE " DEBUGGING".
+ 05 FILLER PIC X(33) VALUE " DECIMAL-POINT".
+ 05 FILLER PIC X(33) VALUE " DECLARATIVES".
+ 05 FILLER PIC X(33) VALUE " DEFAULT".
+ 05 FILLER PIC X(33) VALUE "VDELETE".
+ 05 FILLER PIC X(33) VALUE " DELIMITED".
+ 05 FILLER PIC X(33) VALUE "KDELIMITER".
+ 05 FILLER PIC X(33) VALUE " DEPENDING".
+ 05 FILLER PIC X(33) VALUE " DESCENDING".
+ 05 FILLER PIC X(33) VALUE " DESTINATION".
+ 05 FILLER PIC X(33) VALUE " DETAIL".
+ 05 FILLER PIC X(33) VALUE " DISABLE".
+ 05 FILLER PIC X(33) VALUE " DISK".
+ 05 FILLER PIC X(33) VALUE "VDISPLAY".
+ 05 FILLER PIC X(33) VALUE "VDIVIDE".
+ 05 FILLER PIC X(33) VALUE "KDIVISION".
+ 05 FILLER PIC X(33) VALUE "KDOWN".
+ 05 FILLER PIC X(33) VALUE " DUPLICATES".
+ 05 FILLER PIC X(33) VALUE " DYNAMIC".
+ 05 FILLER PIC X(33) VALUE "IE".
+ 05 FILLER PIC X(33) VALUE " EBCDIC".
+ 05 FILLER PIC X(33) VALUE " EC".
+ 05 FILLER PIC X(33) VALUE "VELSE".
+GC0710 05 FILLER PIC X(33) VALUE "KEND".
+ 05 FILLER PIC X(33) VALUE " END-ACCEPT".
+ 05 FILLER PIC X(33) VALUE " END-ADD".
+ 05 FILLER PIC X(33) VALUE " END-CALL".
+ 05 FILLER PIC X(33) VALUE " END-COMPUTE".
+ 05 FILLER PIC X(33) VALUE " END-DELETE".
+ 05 FILLER PIC X(33) VALUE " END-DISPLAY".
+ 05 FILLER PIC X(33) VALUE " END-DIVIDE".
+ 05 FILLER PIC X(33) VALUE " END-EVALUATE".
+ 05 FILLER PIC X(33) VALUE " END-IF".
+ 05 FILLER PIC X(33) VALUE " END-MULTIPLY".
+ 05 FILLER PIC X(33) VALUE " END-OF-PAGE".
+ 05 FILLER PIC X(33) VALUE " END-PERFORM".
+ 05 FILLER PIC X(33) VALUE " END-READ".
+ 05 FILLER PIC X(33) VALUE " END-RETURN".
+ 05 FILLER PIC X(33) VALUE " END-REWRITE".
+ 05 FILLER PIC X(33) VALUE " END-SEARCH".
+ 05 FILLER PIC X(33) VALUE " END-START".
+ 05 FILLER PIC X(33) VALUE " END-STRING".
+ 05 FILLER PIC X(33) VALUE " END-SUBTRACT".
+ 05 FILLER PIC X(33) VALUE " END-UNSTRING".
+ 05 FILLER PIC X(33) VALUE " END-WRITE".
+ 05 FILLER PIC X(33) VALUE "VENTRY".
+ 05 FILLER PIC X(33) VALUE "KENVIRONMENT".
+ 05 FILLER PIC X(33) VALUE " ENVIRONMENT-NAME".
+ 05 FILLER PIC X(33) VALUE " ENVIRONMENT-VALUE".
+ 05 FILLER PIC X(33) VALUE " EO".
+ 05 FILLER PIC X(33) VALUE " EOL".
+ 05 FILLER PIC X(33) VALUE " EOP".
+ 05 FILLER PIC X(33) VALUE " EOS".
+ 05 FILLER PIC X(33) VALUE " EQUAL".
+ 05 FILLER PIC X(33) VALUE "KEQUALS".
+ 05 FILLER PIC X(33) VALUE " ERASE".
+ 05 FILLER PIC X(33) VALUE " ERROR".
+ 05 FILLER PIC X(33) VALUE " ESCAPE".
+ 05 FILLER PIC X(33) VALUE "VEVALUATE".
+ 05 FILLER PIC X(33) VALUE " EXCEPTION".
+ 05 FILLER PIC X(33) VALUE "IEXCEPTION-FILE".
+ 05 FILLER PIC X(33) VALUE "IEXCEPTION-LOCATION".
+ 05 FILLER PIC X(33) VALUE " EXCEPTION-OBJECT".
+ 05 FILLER PIC X(33) VALUE "IEXCEPTION-STATEMENT".
+ 05 FILLER PIC X(33) VALUE "IEXCEPTION-STATUS".
+ 05 FILLER PIC X(33) VALUE " EXCLUSIVE".
+ 05 FILLER PIC X(33) VALUE "VEXIT".
+ 05 FILLER PIC X(33) VALUE "IEXP".
+ 05 FILLER PIC X(33) VALUE "IEXP10".
+ 05 FILLER PIC X(33) VALUE " EXTEND".
+ 05 FILLER PIC X(33) VALUE " EXTERNAL".
+ 05 FILLER PIC X(33) VALUE "IFACTORIAL".
+ 05 FILLER PIC X(33) VALUE " FACTORY".
+ 05 FILLER PIC X(33) VALUE " FALSE".
+ 05 FILLER PIC X(33) VALUE "KFD".
+ 05 FILLER PIC X(33) VALUE "KFILE".
+ 05 FILLER PIC X(33) VALUE " FILE-CONTROL".
+ 05 FILLER PIC X(33) VALUE " FILE-ID".
+ 05 FILLER PIC X(33) VALUE " FILLER".
+ 05 FILLER PIC X(33) VALUE " FINAL".
+ 05 FILLER PIC X(33) VALUE " FIRST".
+ 05 FILLER PIC X(33) VALUE " FLOAT-BINARY-16".
+ 05 FILLER PIC X(33) VALUE " FLOAT-BINARY-34".
+ 05 FILLER PIC X(33) VALUE " FLOAT-BINARY-7".
+ 05 FILLER PIC X(33) VALUE " FLOAT-DECIMAL-16".
+ 05 FILLER PIC X(33) VALUE " FLOAT-DECIMAL-34".
+ 05 FILLER PIC X(33) VALUE " FLOAT-EXTENDED".
+ 05 FILLER PIC X(33) VALUE " FLOAT-LONG".
+ 05 FILLER PIC X(33) VALUE " FLOAT-SHORT".
+ 05 FILLER PIC X(33) VALUE " FOOTING".
+ 05 FILLER PIC X(33) VALUE " FOR".
+ 05 FILLER PIC X(33) VALUE " FOREGROUND-COLOR".
+ 05 FILLER PIC X(33) VALUE " FOREVER".
+ 05 FILLER PIC X(33) VALUE " FORMAT".
+ 05 FILLER PIC X(33) VALUE "MFORMFEED".
+ 05 FILLER PIC X(33) VALUE "IFRACTION-PART".
+ 05 FILLER PIC X(33) VALUE "VFREE".
+ 05 FILLER PIC X(33) VALUE " FROM".
+ 05 FILLER PIC X(33) VALUE " FULL".
+ 05 FILLER PIC X(33) VALUE " FUNCTION".
+ 05 FILLER PIC X(33) VALUE " FUNCTION-ID".
+ 05 FILLER PIC X(33) VALUE " FUNCTION-POINTER".
+ 05 FILLER PIC X(33) VALUE "VGENERATE".
+ 05 FILLER PIC X(33) VALUE " GET".
+ 05 FILLER PIC X(33) VALUE "KGIVING".
+ 05 FILLER PIC X(33) VALUE " GLOBAL".
+ 05 FILLER PIC X(33) VALUE "VGO".
+ 05 FILLER PIC X(33) VALUE "VGOBACK".
+ 05 FILLER PIC X(33) VALUE " GREATER".
+ 05 FILLER PIC X(33) VALUE " GROUP".
+ 05 FILLER PIC X(33) VALUE " GROUP-USAGE".
+ 05 FILLER PIC X(33) VALUE " HEADING".
+ 05 FILLER PIC X(33) VALUE " HIGH-VALUE".
+ 05 FILLER PIC X(33) VALUE " HIGH-VALUES".
+ 05 FILLER PIC X(33) VALUE " HIGHLIGHT".
+ 05 FILLER PIC X(33) VALUE " I-O".
+ 05 FILLER PIC X(33) VALUE " I-O-CONTROL".
+ 05 FILLER PIC X(33) VALUE "KID".
+ 05 FILLER PIC X(33) VALUE "KIDENTIFICATION".
+ 05 FILLER PIC X(33) VALUE "VIF".
+ 05 FILLER PIC X(33) VALUE " IGNORE".
+ 05 FILLER PIC X(33) VALUE " IGNORING".
+ 05 FILLER PIC X(33) VALUE " IN".
+ 05 FILLER PIC X(33) VALUE " INDEX".
+ 05 FILLER PIC X(33) VALUE "KINDEXED".
+ 05 FILLER PIC X(33) VALUE " INDICATE".
+ 05 FILLER PIC X(33) VALUE " INFINITY".
+ 05 FILLER PIC X(33) VALUE " INHERITS".
+ 05 FILLER PIC X(33) VALUE " INITIAL".
+ 05 FILLER PIC X(33) VALUE " INITIALISED".
+ 05 FILLER PIC X(33) VALUE "VINITIALIZE".
+ 05 FILLER PIC X(33) VALUE " INITIALIZED".
+ 05 FILLER PIC X(33) VALUE "VINITIATE".
+ 05 FILLER PIC X(33) VALUE " INPUT".
+ 05 FILLER PIC X(33) VALUE "KINPUT-OUTPUT".
+ 05 FILLER PIC X(33) VALUE "VINSPECT".
+ 05 FILLER PIC X(33) VALUE " INSTALLATION".
+ 05 FILLER PIC X(33) VALUE "IINTEGER".
+ 05 FILLER PIC X(33) VALUE "IINTEGER-OF-DATE".
+ 05 FILLER PIC X(33) VALUE "IINTEGER-OF-DAY".
+ 05 FILLER PIC X(33) VALUE "IINTEGER-PART".
+ 05 FILLER PIC X(33) VALUE " INTERFACE".
+ 05 FILLER PIC X(33) VALUE " INTERFACE-ID".
+ 05 FILLER PIC X(33) VALUE "KINTO".
+ 05 FILLER PIC X(33) VALUE " INTRINSIC".
+ 05 FILLER PIC X(33) VALUE " INVALID".
+ 05 FILLER PIC X(33) VALUE " INVOKE".
+ 05 FILLER PIC X(33) VALUE " IS".
+ 05 FILLER PIC X(33) VALUE " JUST".
+ 05 FILLER PIC X(33) VALUE " JUSTIFIED".
+ 05 FILLER PIC X(33) VALUE " KEY".
+ 05 FILLER PIC X(33) VALUE " LABEL".
+ 05 FILLER PIC X(33) VALUE " LAST".
+ 05 FILLER PIC X(33) VALUE " LEADING".
+ 05 FILLER PIC X(33) VALUE " LEFT".
+ 05 FILLER PIC X(33) VALUE " LEFT-JUSTIFY".
+ 05 FILLER PIC X(33) VALUE "ILENGTH".
+ 05 FILLER PIC X(33) VALUE " LESS".
+ 05 FILLER PIC X(33) VALUE " LIMIT".
+ 05 FILLER PIC X(33) VALUE " LIMITS".
+ 05 FILLER PIC X(33) VALUE " LINAGE".
+ 05 FILLER PIC X(33) VALUE "ILINAGE-COUNTER".
+ 05 FILLER PIC X(33) VALUE " LINE".
+ 05 FILLER PIC X(33) VALUE " LINE-COUNTER".
+ 05 FILLER PIC X(33) VALUE " LINES".
+ 05 FILLER PIC X(33) VALUE "KLINKAGE".
+ 05 FILLER PIC X(33) VALUE "KLOCAL-STORAGE".
+ 05 FILLER PIC X(33) VALUE " LOCALE".
+ 05 FILLER PIC X(33) VALUE "ILOCALE-DATE".
+ 05 FILLER PIC X(33) VALUE "ILOCALE-TIME".
+ 05 FILLER PIC X(33) VALUE "ILOCALE-TIME-FROM-SECONDS".
+ 05 FILLER PIC X(33) VALUE " LOCK".
+ 05 FILLER PIC X(33) VALUE "ILOG".
+ 05 FILLER PIC X(33) VALUE "ILOG10".
+ 05 FILLER PIC X(33) VALUE " LOW-VALUE".
+ 05 FILLER PIC X(33) VALUE " LOW-VALUES".
+ 05 FILLER PIC X(33) VALUE " LOWER".
+ 05 FILLER PIC X(33) VALUE "ILOWER-CASE".
+ 05 FILLER PIC X(33) VALUE " LOWLIGHT".
+ 05 FILLER PIC X(33) VALUE " MANUAL".
+ 05 FILLER PIC X(33) VALUE "IMAX".
+ 05 FILLER PIC X(33) VALUE "IMEAN".
+ 05 FILLER PIC X(33) VALUE "IMEDIAN".
+ 05 FILLER PIC X(33) VALUE " MEMORY".
+ 05 FILLER PIC X(33) VALUE "VMERGE".
+ 05 FILLER PIC X(33) VALUE " METHOD".
+ 05 FILLER PIC X(33) VALUE " METHOD-ID".
+ 05 FILLER PIC X(33) VALUE "IMIDRANGE".
+ 05 FILLER PIC X(33) VALUE "IMIN".
+ 05 FILLER PIC X(33) VALUE " MINUS".
+ 05 FILLER PIC X(33) VALUE "IMOD".
+ 05 FILLER PIC X(33) VALUE " MODE".
+ 05 FILLER PIC X(33) VALUE "VMOVE".
+ 05 FILLER PIC X(33) VALUE " MULTIPLE".
+ 05 FILLER PIC X(33) VALUE "VMULTIPLY".
+ 05 FILLER PIC X(33) VALUE " NATIONAL".
+ 05 FILLER PIC X(33) VALUE " NATIONAL-EDITED".
+ 05 FILLER PIC X(33) VALUE " NATIVE".
+ 05 FILLER PIC X(33) VALUE " NEGATIVE".
+ 05 FILLER PIC X(33) VALUE " NESTED".
+ 05 FILLER PIC X(33) VALUE "VNEXT".
+ 05 FILLER PIC X(33) VALUE " NO".
+ 05 FILLER PIC X(33) VALUE " NOT".
+ 05 FILLER PIC X(33) VALUE " NULL".
+ 05 FILLER PIC X(33) VALUE " NULLS".
+ 05 FILLER PIC X(33) VALUE " NUMBER".
+ 05 FILLER PIC X(33) VALUE "INUMBER-OF-CALL-PARAMETERS".
+ 05 FILLER PIC X(33) VALUE " NUMBERS".
+ 05 FILLER PIC X(33) VALUE " NUMERIC".
+ 05 FILLER PIC X(33) VALUE " NUMERIC-EDITED".
+ 05 FILLER PIC X(33) VALUE "INUMVAL".
+ 05 FILLER PIC X(33) VALUE "INUMVAL-C".
+ 05 FILLER PIC X(33) VALUE " OBJECT".
+ 05 FILLER PIC X(33) VALUE " OBJECT-COMPUTER".
+ 05 FILLER PIC X(33) VALUE " OBJECT-REFERENCE".
+ 05 FILLER PIC X(33) VALUE " OCCURS".
+ 05 FILLER PIC X(33) VALUE " OF".
+ 05 FILLER PIC X(33) VALUE " OFF".
+ 05 FILLER PIC X(33) VALUE " OMITTED".
+ 05 FILLER PIC X(33) VALUE " ON".
+ 05 FILLER PIC X(33) VALUE " ONLY".
+ 05 FILLER PIC X(33) VALUE "VOPEN".
+ 05 FILLER PIC X(33) VALUE " OPTIONAL".
+ 05 FILLER PIC X(33) VALUE " OPTIONS".
+ 05 FILLER PIC X(33) VALUE " OR".
+ 05 FILLER PIC X(33) VALUE "IORD".
+ 05 FILLER PIC X(33) VALUE "IORD-MAX".
+ 05 FILLER PIC X(33) VALUE "IORD-MIN".
+ 05 FILLER PIC X(33) VALUE " ORDER".
+ 05 FILLER PIC X(33) VALUE " ORGANIZATION".
+ 05 FILLER PIC X(33) VALUE " OTHER".
+ 05 FILLER PIC X(33) VALUE " OUTPUT".
+ 05 FILLER PIC X(33) VALUE " OVERFLOW".
+ 05 FILLER PIC X(33) VALUE " OVERLINE".
+ 05 FILLER PIC X(33) VALUE " OVERRIDE".
+ 05 FILLER PIC X(33) VALUE " PACKED-DECIMAL".
+ 05 FILLER PIC X(33) VALUE " PADDING".
+ 05 FILLER PIC X(33) VALUE " PAGE".
+ 05 FILLER PIC X(33) VALUE " PAGE-COUNTER".
+ 05 FILLER PIC X(33) VALUE " PARAGRAPH".
+ 05 FILLER PIC X(33) VALUE "VPERFORM".
+ 05 FILLER PIC X(33) VALUE " PF".
+ 05 FILLER PIC X(33) VALUE " PH".
+ 05 FILLER PIC X(33) VALUE "IPI".
+ 05 FILLER PIC X(33) VALUE "KPIC".
+ 05 FILLER PIC X(33) VALUE "KPICTURE".
+ 05 FILLER PIC X(33) VALUE " PLUS".
+ 05 FILLER PIC X(33) VALUE "KPOINTER".
+ 05 FILLER PIC X(33) VALUE " POSITION".
+ 05 FILLER PIC X(33) VALUE " POSITIVE".
+ 05 FILLER PIC X(33) VALUE " PRESENT".
+ 05 FILLER PIC X(33) VALUE "IPRESENT-VALUE".
+ 05 FILLER PIC X(33) VALUE " PREVIOUS".
+ 05 FILLER PIC X(33) VALUE "MPRINTER".
+ 05 FILLER PIC X(33) VALUE " PRINTING".
+ 05 FILLER PIC X(33) VALUE "KPROCEDURE".
+ 05 FILLER PIC X(33) VALUE " PROCEDURE-POINTER".
+ 05 FILLER PIC X(33) VALUE " PROCEDURES".
+ 05 FILLER PIC X(33) VALUE " PROCEED".
+ 05 FILLER PIC X(33) VALUE " PROGRAM".
+ 05 FILLER PIC X(33) VALUE "KPROGRAM-ID".
+ 05 FILLER PIC X(33) VALUE " PROGRAM-POINTER".
+ 05 FILLER PIC X(33) VALUE " PROMPT".
+ 05 FILLER PIC X(33) VALUE " PROPERTY".
+ 05 FILLER PIC X(33) VALUE " PROTOTYPE".
+ 05 FILLER PIC X(33) VALUE " QUOTE".
+ 05 FILLER PIC X(33) VALUE " QUOTES".
+ 05 FILLER PIC X(33) VALUE " RAISE".
+ 05 FILLER PIC X(33) VALUE " RAISING".
+ 05 FILLER PIC X(33) VALUE "IRANDOM".
+ 05 FILLER PIC X(33) VALUE "IRANGE".
+ 05 FILLER PIC X(33) VALUE " RD".
+ 05 FILLER PIC X(33) VALUE "VREAD".
+ 05 FILLER PIC X(33) VALUE "VREADY".
+ 05 FILLER PIC X(33) VALUE " RECORD".
+ 05 FILLER PIC X(33) VALUE " RECORDING".
+ 05 FILLER PIC X(33) VALUE " RECORDS".
+ 05 FILLER PIC X(33) VALUE " RECURSIVE".
+ 05 FILLER PIC X(33) VALUE "KREDEFINES".
+ 05 FILLER PIC X(33) VALUE " REEL".
+ 05 FILLER PIC X(33) VALUE " REFERENCE".
+ 05 FILLER PIC X(33) VALUE " RELATIVE".
+ 05 FILLER PIC X(33) VALUE "VRELEASE".
+ 05 FILLER PIC X(33) VALUE "IREM".
+ 05 FILLER PIC X(33) VALUE " REMAINDER".
+ 05 FILLER PIC X(33) VALUE " REMARKS".
+ 05 FILLER PIC X(33) VALUE " REMOVAL".
+ 05 FILLER PIC X(33) VALUE "KRENAMES".
+ 05 FILLER PIC X(33) VALUE "KREPLACING".
+ 05 FILLER PIC X(33) VALUE "KREPORT".
+ 05 FILLER PIC X(33) VALUE " REPORTING".
+ 05 FILLER PIC X(33) VALUE " REPORTS".
+ 05 FILLER PIC X(33) VALUE " REPOSITORY".
+ 05 FILLER PIC X(33) VALUE " REPRESENTS-NOT-A-NUMBER".
+ 05 FILLER PIC X(33) VALUE " REQUIRED".
+ 05 FILLER PIC X(33) VALUE " RESERVE".
+ 05 FILLER PIC X(33) VALUE " RESUME".
+ 05 FILLER PIC X(33) VALUE " RETRY".
+ 05 FILLER PIC X(33) VALUE "VRETURN".
+ 05 FILLER PIC X(33) VALUE "IRETURN-CODE".
+ 05 FILLER PIC X(33) VALUE "KRETURNING".
+ 05 FILLER PIC X(33) VALUE "IREVERSE".
+ 05 FILLER PIC X(33) VALUE " REVERSE-VIDEO".
+ 05 FILLER PIC X(33) VALUE " REWIND".
+ 05 FILLER PIC X(33) VALUE "VREWRITE".
+ 05 FILLER PIC X(33) VALUE " RF".
+ 05 FILLER PIC X(33) VALUE " RH".
+ 05 FILLER PIC X(33) VALUE " RIGHT".
+ 05 FILLER PIC X(33) VALUE " RIGHT-JUSTIFY".
+ 05 FILLER PIC X(33) VALUE "VROLLBACK".
+ 05 FILLER PIC X(33) VALUE " ROUNDED".
+ 05 FILLER PIC X(33) VALUE " RUN".
+ 05 FILLER PIC X(33) VALUE " SAME".
+ 05 FILLER PIC X(33) VALUE "KSCREEN".
+ 05 FILLER PIC X(33) VALUE " SCROLL".
+ 05 FILLER PIC X(33) VALUE "KSD".
+ 05 FILLER PIC X(33) VALUE "VSEARCH".
+ 05 FILLER PIC X(33) VALUE "ISECONDS-FROM-FORMATTED-TIME".
+ 05 FILLER PIC X(33) VALUE "ISECONDS-PAST-MIDNIGHT".
+ 05 FILLER PIC X(33) VALUE "KSECTION".
+ 05 FILLER PIC X(33) VALUE " SECURE".
+ 05 FILLER PIC X(33) VALUE " SECURITY".
+ 05 FILLER PIC X(33) VALUE " SEGMENT-LIMIT".
+ 05 FILLER PIC X(33) VALUE " SELECT".
+ 05 FILLER PIC X(33) VALUE " SELF".
+ 05 FILLER PIC X(33) VALUE " SENTENCE".
+ 05 FILLER PIC X(33) VALUE " SEPARATE".
+ 05 FILLER PIC X(33) VALUE " SEQUENCE".
+ 05 FILLER PIC X(33) VALUE " SEQUENTIAL".
+ 05 FILLER PIC X(33) VALUE "VSET".
+ 05 FILLER PIC X(33) VALUE " SHARING".
+ 05 FILLER PIC X(33) VALUE "ISIGN".
+ 05 FILLER PIC X(33) VALUE " SIGNED".
+ 05 FILLER PIC X(33) VALUE " SIGNED-INT".
+ 05 FILLER PIC X(33) VALUE " SIGNED-LONG".
+ 05 FILLER PIC X(33) VALUE " SIGNED-SHORT".
+ 05 FILLER PIC X(33) VALUE "ISIN".
+ 05 FILLER PIC X(33) VALUE " SIZE".
+ 05 FILLER PIC X(33) VALUE "VSORT".
+ 05 FILLER PIC X(33) VALUE " SORT-MERGE".
+ 05 FILLER PIC X(33) VALUE "ISORT-RETURN".
+ 05 FILLER PIC X(33) VALUE " SOURCE".
+ 05 FILLER PIC X(33) VALUE " SOURCE-COMPUTER".
+ 05 FILLER PIC X(33) VALUE " SOURCES".
+ 05 FILLER PIC X(33) VALUE " SPACE".
+ 05 FILLER PIC X(33) VALUE " SPACE-FILL".
+ 05 FILLER PIC X(33) VALUE " SPACES".
+ 05 FILLER PIC X(33) VALUE " SPECIAL-NAMES".
+ 05 FILLER PIC X(33) VALUE "ISQRT".
+ 05 FILLER PIC X(33) VALUE " STANDARD".
+ 05 FILLER PIC X(33) VALUE " STANDARD-1".
+ 05 FILLER PIC X(33) VALUE " STANDARD-2".
+ 05 FILLER PIC X(33) VALUE "ISTANDARD-DEVIATION".
+ 05 FILLER PIC X(33) VALUE "VSTART".
+ 05 FILLER PIC X(33) VALUE " STATUS".
+ 05 FILLER PIC X(33) VALUE "VSTOP".
+ 05 FILLER PIC X(33) VALUE "ISTORED-CHAR-LENGTH".
+ 05 FILLER PIC X(33) VALUE "VSTRING".
+ 05 FILLER PIC X(33) VALUE "ISUBSTITUTE".
+ 05 FILLER PIC X(33) VALUE "ISUBSTITUTE-CASE".
+ 05 FILLER PIC X(33) VALUE "VSUBTRACT".
+ 05 FILLER PIC X(33) VALUE "ISUM".
+ 05 FILLER PIC X(33) VALUE " SUPER".
+ 05 FILLER PIC X(33) VALUE "VSUPPRESS".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-1".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-2".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-3".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-4".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-5".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-6".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-7".
+ 05 FILLER PIC X(33) VALUE "MSWITCH-8".
+ 05 FILLER PIC X(33) VALUE " SYMBOLIC".
+ 05 FILLER PIC X(33) VALUE " SYNC".
+ 05 FILLER PIC X(33) VALUE " SYNCHRONIZED".
+ 05 FILLER PIC X(33) VALUE "MSYSERR".
+ 05 FILLER PIC X(33) VALUE "MSYSIN".
+ 05 FILLER PIC X(33) VALUE "MSYSIPT".
+ 05 FILLER PIC X(33) VALUE "MSYSLIST".
+ 05 FILLER PIC X(33) VALUE "MSYSLST".
+ 05 FILLER PIC X(33) VALUE "MSYSOUT".
+ 05 FILLER PIC X(33) VALUE " SYSTEM-DEFAULT".
+ 05 FILLER PIC X(33) VALUE " TABLE".
+ 05 FILLER PIC X(33) VALUE "KTALLYING".
+ 05 FILLER PIC X(33) VALUE "ITAN".
+ 05 FILLER PIC X(33) VALUE " TAPE".
+ 05 FILLER PIC X(33) VALUE "VTERMINATE".
+ 05 FILLER PIC X(33) VALUE " TEST".
+ 05 FILLER PIC X(33) VALUE "ITEST-DATE-YYYYMMDD".
+ 05 FILLER PIC X(33) VALUE "ITEST-DAY-YYYYDDD".
+ 05 FILLER PIC X(33) VALUE " THAN".
+ 05 FILLER PIC X(33) VALUE " THEN".
+ 05 FILLER PIC X(33) VALUE " THROUGH".
+ 05 FILLER PIC X(33) VALUE " THRU".
+ 05 FILLER PIC X(33) VALUE " TIME".
+ 05 FILLER PIC X(33) VALUE " TIMES".
+ 05 FILLER PIC X(33) VALUE "KTO".
+ 05 FILLER PIC X(33) VALUE " TOP".
+ 05 FILLER PIC X(33) VALUE " TRAILING".
+ 05 FILLER PIC X(33) VALUE " TRAILING-SIGN".
+ 05 FILLER PIC X(33) VALUE "VTRANSFORM".
+ 05 FILLER PIC X(33) VALUE "ITRIM".
+ 05 FILLER PIC X(33) VALUE " TRUE".
+ 05 FILLER PIC X(33) VALUE " TYPE".
+ 05 FILLER PIC X(33) VALUE " TYPEDEF".
+ 05 FILLER PIC X(33) VALUE " UNDERLINE".
+ 05 FILLER PIC X(33) VALUE " UNIT".
+ 05 FILLER PIC X(33) VALUE " UNIVERSAL".
+ 05 FILLER PIC X(33) VALUE "VUNLOCK".
+ 05 FILLER PIC X(33) VALUE " UNSIGNED".
+ 05 FILLER PIC X(33) VALUE " UNSIGNED-INT".
+ 05 FILLER PIC X(33) VALUE " UNSIGNED-LONG".
+ 05 FILLER PIC X(33) VALUE " UNSIGNED-SHORT".
+ 05 FILLER PIC X(33) VALUE "VUNSTRING".
+ 05 FILLER PIC X(33) VALUE " UNTIL".
+ 05 FILLER PIC X(33) VALUE "KUP".
+ 05 FILLER PIC X(33) VALUE " UPDATE".
+ 05 FILLER PIC X(33) VALUE " UPON".
+ 05 FILLER PIC X(33) VALUE " UPPER".
+ 05 FILLER PIC X(33) VALUE "IUPPER-CASE".
+ 05 FILLER PIC X(33) VALUE " USAGE".
+ 05 FILLER PIC X(33) VALUE "VUSE".
+ 05 FILLER PIC X(33) VALUE " USER-DEFAULT".
+ 05 FILLER PIC X(33) VALUE "KUSING".
+ 05 FILLER PIC X(33) VALUE " VAL-STATUS".
+ 05 FILLER PIC X(33) VALUE " VALID".
+ 05 FILLER PIC X(33) VALUE " VALIDATE".
+ 05 FILLER PIC X(33) VALUE " VALIDATE-STATUS".
+ 05 FILLER PIC X(33) VALUE " VALUE".
+ 05 FILLER PIC X(33) VALUE " VALUES".
+ 05 FILLER PIC X(33) VALUE "IVARIANCE".
+ 05 FILLER PIC X(33) VALUE "KVARYING".
+ 05 FILLER PIC X(33) VALUE " WAIT".
+ 05 FILLER PIC X(33) VALUE "VWHEN".
+ 05 FILLER PIC X(33) VALUE "IWHEN-COMPILED".
+ 05 FILLER PIC X(33) VALUE " WITH".
+ 05 FILLER PIC X(33) VALUE " WORDS".
+ 05 FILLER PIC X(33) VALUE "KWORKING-STORAGE".
+ 05 FILLER PIC X(33) VALUE "VWRITE".
+ 05 FILLER PIC X(33) VALUE "IYEAR-TO-YYYY".
+ 05 FILLER PIC X(33) VALUE " YYYYDDD".
+ 05 FILLER PIC X(33) VALUE " YYYYMMDD".
+ 05 FILLER PIC X(33) VALUE " ZERO".
+ 05 FILLER PIC X(33) VALUE " ZERO-FILL".
+ 05 FILLER PIC X(33) VALUE " ZEROES".
+ 05 FILLER PIC X(33) VALUE " ZEROS".
+ 01 Reserved-Word-Table REDEFINES Reserved-Words.
+ 05 Reserved-Word OCCURS 591 TIMES
+ ASCENDING KEY RW-Word
+ INDEXED RW-Idx.
+ 10 RW-Type PIC X(1).
+ 10 RW-Word PIC X(32).
+
+ 01 Saved-Section PIC X(15).
+
+ 01 Search-Token PIC X(32).
+
+ 01 Source-Line-No PIC 9(6).
+
+ 01 Src-Ptr USAGE BINARY-LONG.
+
+ 01 Syntax-Parsing-Items.
+ 05 SPI-Current-Char PIC X(1).
+ 88 Current-Char-Is-Punct VALUE "=", "(", ")", "*", "/",
+ "&", ";", ",", "<", ">",
+ ":".
+ 88 Current-Char-Is-Quote VALUE '"', "'".
+ 88 Current-Char-Is-X VALUE "x", "X".
+ 88 Current-Char-Is-Z VALUE "z", "Z".
+ 05 SPI-Current-Division PIC X(1).
+ 88 In-IDENTIFICATION-DIVISION VALUE "I", "?".
+ 88 In-ENVIRONMENT-DIVISION VALUE "E".
+ 88 In-DATA-DIVISION VALUE "D".
+ 88 In-PROCEDURE-DIVISION VALUE "P".
+ 05 SPI-Current-Line-No PIC 9(6).
+ 05 SPI-Current-Program-ID.
+ 10 FILLER PIC X(12).
+ 10 SPI-CP-13-15 PIC X(3).
+ 05 SPI-Current-Section.
+ 10 SPI-CS-1 PIC X(1).
+ 10 SPI-CS-2-14.
+ 15 FILLER PIC X(10).
+ 15 SPI-CS-11-14 PIC X(3).
+ 10 SPI-CS-15 PIC X(1).
+ 05 SPI-Current-Token PIC X(32).
+ 05 SPI-Current-Token-UC PIC X(32).
+ 05 SPI-Current-Verb PIC X(12).
+ 05 SPI-Next-Char PIC X(1).
+ 88 Next-Char-Is-Quote VALUE '"', "'".
+ 05 SPI-Prior-Token PIC X(32).
+ 05 SPI-Token-Type PIC X(1).
+ 88 Token-Is-EOF VALUE HIGH-VALUES.
+ 88 Token-Is-Identifier VALUE "I".
+ 88 Token-Is-Key-Word VALUE "K", "V".
+ 88 Token-Is-Literal-Alpha VALUE "L".
+ 88 Token-Is-Literal-Number VALUE "N".
+ 88 Token-Is-Verb VALUE "V".
+GC0710 88 Token-Is-Reserved-Word VALUE " ".
+
+ 01 Tally USAGE BINARY-LONG.
+
+ 01 Todays-Date PIC 9(8).
+
+ LINKAGE SECTION.
+ 01 Produce-Source-Listing PIC X(1).
+ 01 Produce-Xref-Listing PIC X(1).
+ 01 Src-Filename PIC X(256).
+ /
+ PROCEDURE DIVISION USING Produce-Source-Listing
+ Produce-Xref-Listing
+ Src-Filename.
+ 000-Main SECTION.
+ 001-Init.
+ PERFORM 100-Initialization
+ PERFORM 200-Execute-cobc
+ OPEN OUTPUT Report-File
+ IF Produce-Source-Listing NOT = SPACE
+ PERFORM 500-Produce-Source-Listing
+ END-IF
+ IF Produce-Xref-Listing NOT = SPACE
+ SORT Sort-File
+ ASCENDING KEY SR-Prog-ID
+ SR-Token-UC
+ SR-Line-No-Ref
+ INPUT PROCEDURE 300-Tokenize-Source
+ OUTPUT PROCEDURE 400-Produce-Xref-Listing
+ END-IF
+ CLOSE Report-File
+ GOBACK
+ .
+ /
+ 100-Initialization SECTION.
+ *****************************************************************
+ ** Perform all program-wide initialization operations **
+ *****************************************************************
+ 101-Establish-Working-Env.
+ MOVE TRIM(Src-Filename,Leading) TO Src-Filename
+ ACCEPT Env-TEMP
+ FROM ENVIRONMENT "TEMP"
+ END-ACCEPT
+ ACCEPT Lines-Per-Page-ENV
+ FROM ENVIRONMENT "OCXREF_LINES"
+ END-ACCEPT
+ INSPECT Src-Filename REPLACING ALL "\" BY "/"
+ INSPECT Env-TEMP REPLACING ALL "\" BY "/"
+ MOVE Src-Filename TO Program-Path
+ MOVE Program-Path TO Heading-2
+ CALL "C$JUSTIFY"
+ USING Heading-2, "Right"
+ END-CALL
+ MOVE LENGTH(TRIM(Src-Filename,Trailing)) TO I
+ MOVE 0 TO J
+ PERFORM UNTIL Src-Filename(I:1) = '/'
+ OR I = 0
+ SUBTRACT 1 FROM I
+ ADD 1 TO J
+ END-PERFORM
+ UNSTRING Src-Filename((I + 1):J) DELIMITED BY "."
+ INTO Filename, Dummy
+ END-UNSTRING
+ STRING TRIM(Env-TEMP,Trailing)
+ "/"
+ TRIM(Filename,Trailing)
+ ".i"
+ DELIMITED SIZE
+ INTO Expanded-Src-Filename
+ END-STRING
+ STRING Program-Path(1:I)
+ TRIM(Filename,Trailing)
+ ".lst"
+ DELIMITED SIZE
+ INTO Report-Filename
+ END-STRING
+ IF Lines-Per-Page-ENV NOT = SPACES
+ MOVE NUMVAL(Lines-Per-Page-ENV) TO Lines-Per-Page
+ ELSE
+ MOVE 60 TO Lines-Per-Page
+ END-IF
+ ACCEPT Todays-Date
+ FROM DATE YYYYMMDD
+ END-ACCEPT
+ MOVE Todays-Date TO H1X-Date
+ H1S-Date
+ MOVE "????????????..." TO SPI-Current-Program-ID
+ MOVE SPACES TO SPI-Current-Verb
+ Held-Reference
+ MOVE "Y" TO F-First-Record
+ .
+ /
+ 200-Execute-cobc SECTION.
+ 201-Build-Cmd.
+ STRING "cobc -E "
+ TRIM(Program-Path, Trailing)
+ " > "
+ TRIM(Expanded-Src-Filename,Trailing)
+ DELIMITED SIZE
+ INTO Cmd
+ END-STRING
+ CALL "SYSTEM"
+ USING Cmd
+ END-CALL
+ IF RETURN-CODE NOT = 0
+ DISPLAY
+ "Cross-reference terminated by previous errors"
+ UPON SYSERR
+ END-DISPLAY
+ GOBACK
+ END-IF
+ .
+
+ 209-Exit.
+ EXIT
+ .
+ /
+ 300-Tokenize-Source SECTION.
+ 301-Driver.
+ OPEN INPUT Expand-Code
+ MOVE SPACES TO Expand-Code-Rec
+ MOVE 256 TO Src-Ptr
+ MOVE 0 TO Num-UserNames
+ SPI-Current-Line-No
+ MOVE "?" TO SPI-Current-Division
+GC0710 MOVE "N" TO F-Verb-Has-Been-Found.
+ PERFORM FOREVER
+ PERFORM 310-Get-Token
+ IF Token-Is-EOF
+ EXIT PERFORM
+ END-IF
+ MOVE UPPER-CASE(SPI-Current-Token)
+ TO SPI-Current-Token-UC
+ IF Token-Is-Verb
+ MOVE SPI-Current-Token-UC TO SPI-Current-Verb
+ SPI-Prior-Token
+ IF Held-Reference NOT = SPACES
+ MOVE Held-Reference TO Sort-Rec
+ MOVE SPACES TO Held-Reference
+ RELEASE Sort-Rec
+ END-IF
+ END-IF
+ EVALUATE TRUE
+ WHEN In-IDENTIFICATION-DIVISION
+ PERFORM 320-IDENTIFICATION-DIVISION
+ WHEN In-ENVIRONMENT-DIVISION
+ PERFORM 330-ENVIRONMENT-DIVISION
+ WHEN In-DATA-DIVISION
+ PERFORM 340-DATA-DIVISION
+ WHEN In-PROCEDURE-DIVISION
+ PERFORM 350-PROCEDURE-DIVISION
+ END-EVALUATE
+ IF Token-Is-Key-Word
+ MOVE SPI-Current-Token-UC TO SPI-Prior-Token
+ END-IF
+ IF F-Token-Ended-Sentence = "Y"
+ AND SPI-Current-Division NOT = "I"
+ MOVE SPACES TO SPI-Prior-Token
+ SPI-Current-Verb
+ END-IF
+
+ END-PERFORM
+ CLOSE Expand-Code
+ EXIT SECTION
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 310-Get-Token.
+ *>-- Position to 1st non-blank character
+ MOVE F-Token-Ended-Sentence TO F-Last-Token-Ended-Sent
+ MOVE "N" TO F-Token-Ended-Sentence
+ PERFORM UNTIL Expand-Code-Rec(Src-Ptr : 1) NOT = SPACE
+ IF Src-Ptr > 255
+ READ Expand-Code AT END
+ IF Held-Reference NOT = SPACES
+ MOVE Held-Reference TO Sort-Rec
+ MOVE SPACES TO Held-Reference
+ RELEASE Sort-Rec
+ END-IF
+ SET Token-Is-EOF TO TRUE
+ MOVE 0 TO SPI-Current-Line-No
+ EXIT PARAGRAPH
+ END-READ
+ IF ECR-1 = "#"
+ PERFORM 311-Control-Record
+ ELSE
+ PERFORM 312-Expand-Code-Record
+ END-IF
+ ELSE
+ ADD 1 TO Src-Ptr
+ END-IF
+ END-PERFORM
+ *>-- Extract token string
+ MOVE Expand-Code-Rec(Src-Ptr : 1) TO SPI-Current-Char
+ MOVE Expand-Code-Rec(Src-Ptr + 1: 1) TO SPI-Next-Char
+ IF SPI-Current-Char = "."
+ ADD 1 TO Src-Ptr
+ MOVE SPI-Current-Char TO SPI-Current-Token
+ MOVE SPACE TO SPI-Token-Type
+ MOVE "Y" TO F-Token-Ended-Sentence
+ EXIT PARAGRAPH
+ END-IF
+ IF Current-Char-Is-Punct
+ AND SPI-Current-Char = "="
+ AND SPI-Current-Division = "P"
+ ADD 1 TO Src-Ptr
+ MOVE "EQUALS" TO SPI-Current-Token
+ MOVE "K" TO SPI-Token-Type
+ EXIT PARAGRAPH
+ END-IF
+ IF Current-Char-Is-Punct *> So subscripts don't get flagged w/ "*"
+ AND SPI-Current-Char = "("
+ AND SPI-Current-Division = "P"
+ MOVE SPACES TO SPI-Prior-Token
+ END-IF
+ IF Current-Char-Is-Punct
+ ADD 1 TO Src-Ptr
+ MOVE SPI-Current-Char TO SPI-Current-Token
+ MOVE SPACE TO SPI-Token-Type
+ EXIT PARAGRAPH
+ END-IF
+ IF Current-Char-Is-Quote
+ ADD 1 TO Src-Ptr
+ UNSTRING Expand-Code-Rec
+ DELIMITED BY SPI-Current-Char
+ INTO SPI-Current-Token
+ WITH POINTER Src-Ptr
+ END-UNSTRING
+ IF Expand-Code-Rec(Src-Ptr : 1) = "."
+ MOVE "Y" TO F-Token-Ended-Sentence
+ ADD 1 TO Src-Ptr
+ END-IF
+ SET Token-Is-Literal-Alpha TO TRUE
+ EXIT PARAGRAPH
+ END-IF
+ IF Current-Char-Is-X AND Next-Char-Is-Quote
+ ADD 2 TO Src-Ptr
+ UNSTRING Expand-Code-Rec
+ DELIMITED BY SPI-Next-Char
+ INTO SPI-Current-Token
+ WITH POINTER Src-Ptr
+ END-UNSTRING
+ IF Expand-Code-Rec(Src-Ptr : 1) = "."
+ MOVE "Y" TO F-Token-Ended-Sentence
+ ADD 1 TO Src-Ptr
+ END-IF
+ SET Token-Is-Literal-Number TO TRUE
+ EXIT PARAGRAPH
+ END-IF
+ IF Current-Char-Is-Z AND Next-Char-Is-Quote
+ ADD 2 TO Src-Ptr
+ UNSTRING Expand-Code-Rec
+ DELIMITED BY SPI-Next-Char
+ INTO SPI-Current-Token
+ WITH POINTER Src-Ptr
+ END-UNSTRING
+ IF Expand-Code-Rec(Src-Ptr : 1) = "."
+ MOVE "Y" TO F-Token-Ended-Sentence
+ ADD 1 TO Src-Ptr
+ END-IF
+ SET Token-Is-Literal-Alpha TO TRUE
+ EXIT PARAGRAPH
+ END-IF
+ IF F-Processing-PICTURE = "Y"
+ UNSTRING Expand-Code-Rec
+ DELIMITED BY ". " OR " "
+ INTO SPI-Current-Token
+ DELIMITER IN Delim
+ WITH POINTER Src-Ptr
+ END-UNSTRING
+ IF Delim = ". "
+ MOVE "Y" TO F-Token-Ended-Sentence
+ ADD 1 TO Src-Ptr
+ END-IF
+ IF UPPER-CASE(SPI-Current-Token) = "IS"
+ MOVE SPACE TO SPI-Token-Type
+ EXIT PARAGRAPH
+ ELSE
+ MOVE "N" TO F-Processing-PICTURE
+ MOVE SPACE TO SPI-Token-Type
+ EXIT PARAGRAPH
+ END-IF
+ END-IF
+ UNSTRING Expand-Code-Rec
+ DELIMITED BY ". " OR " " OR "=" OR "(" OR ")" OR "*"
+ OR "/" OR "&" OR ";" OR "," OR "<"
+ OR ">" OR ":"
+ INTO SPI-Current-Token
+ DELIMITER IN Delim
+ WITH POINTER Src-Ptr
+ END-UNSTRING
+ IF Delim = ". "
+ MOVE "Y" TO F-Token-Ended-Sentence
+ END-IF
+ IF Delim NOT = ". " AND " "
+ SUBTRACT 1 FROM Src-Ptr
+ END-IF
+ *>-- Classify Token
+ MOVE UPPER-CASE(SPI-Current-Token) TO Search-Token
+ IF Search-Token = "EQUAL" OR "EQUALS"
+ MOVE "EQUALS" TO SPI-Current-Token
+ MOVE "K" TO SPI-Token-Type
+ EXIT PARAGRAPH
+ END-IF
+ SEARCH ALL Reserved-Word
+ WHEN RW-Word (RW-Idx) = Search-Token
+ MOVE RW-Type (RW-Idx) TO SPI-Token-Type
+GC0710 IF Token-Is-Verb
+GC0710 MOVE "Y" TO F-Verb-Has-Been-Found
+GC0710 END-IF
+ EXIT PARAGRAPH
+ END-SEARCH
+ *>-- Not a reserved word, must be a user name
+ SET Token-Is-Identifier TO TRUE *> NEEDS EXPANSION!!!!
+ PERFORM 313-Check-For-Numeric-Token
+ IF Token-Is-Literal-Number
+ IF (F-Last-Token-Ended-Sent = "Y")
+ AND (SPI-Current-Division = "D")
+ MOVE "LEVEL #" TO SPI-Current-Token
+ MOVE "K" TO SPI-Token-Type
+ EXIT PARAGRAPH
+ ELSE
+ EXIT PARAGRAPH
+ END-IF
+ END-IF
+ EXIT PARAGRAPH
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 311-Control-Record.
+ UNSTRING ECR-2-256
+ DELIMITED BY '"'
+ INTO PIC-X10, PIC-X256, Dummy
+ END-UNSTRING
+ INSPECT PIC-X10 REPLACING ALL '"' BY SPACE
+ COMPUTE I = NUMVAL(PIC-X10) - 1
+ IF TRIM(PIC-X256,Trailing) = TRIM(Program-Path,Trailing)
+ MOVE I TO SPI-Current-Line-No
+ SET In-Main-Module TO TRUE
+ IF Saved-Section NOT = SPACES
+ MOVE Saved-Section TO SPI-Current-Section
+ END-IF
+ ELSE
+ SET In-Copybook TO TRUE
+ IF Saved-Section = SPACES
+ MOVE SPI-Current-Section TO Saved-Section
+ END-IF
+ MOVE LENGTH(TRIM(PIC-X256,Trailing)) TO I
+ MOVE 0 TO J
+ PERFORM UNTIL PIC-X256(I:1) = '/'
+ OR I = 0
+ SUBTRACT 1 FROM I
+ ADD 1 TO J
+ END-PERFORM
+ UNSTRING PIC-X256((I + 1):J) DELIMITED BY "."
+ INTO Filename, Dummy
+ END-UNSTRING
+ MOVE "[" TO SPI-CS-1
+ MOVE Filename TO SPI-CS-2-14
+ IF SPI-CS-11-14 NOT = SPACES
+ MOVE "..." TO SPI-CS-11-14
+ END-IF
+ MOVE "]" TO SPI-CS-15
+ END-IF
+ MOVE SPACES TO Expand-Code-Rec *> Force another READ
+ MOVE 256 TO Src-Ptr
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 312-Expand-Code-Record.
+ MOVE 1 TO Src-Ptr
+ IF In-Main-Module
+ ADD 1 To SPI-Current-Line-No
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 313-Check-For-Numeric-Token.
+ MOVE SPI-Current-Token TO PIC-X32
+ INSPECT PIC-X32
+ REPLACING TRAILING SPACES BY "0"
+ IF PIC-X32 IS NUMERIC *> Simple Unsigned Integer
+ SET Token-Is-Literal-Number TO TRUE
+ EXIT PARAGRAPH
+ END-IF
+ IF PIC-X32(1:1) = "+" OR "-"
+ MOVE "0" TO PIC-X32(1:1)
+ END-IF
+ MOVE 0 TO Tally
+ INSPECT PIC-X32
+ TALLYING Tally FOR ALL "."
+ IF Tally = 1
+ INSPECT PIC-X32 REPLACING ALL "." BY "0"
+ END-IF
+ IF PIC-X32 IS NUMERIC
+ SET Token-Is-Literal-Number TO TRUE
+ EXIT PARAGRAPH
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 320-IDENTIFICATION-DIVISION.
+GC0710 MOVE "N" TO F-Verb-Has-Been-Found
+ IF Token-Is-Key-Word AND SPI-Current-Token = "DIVISION"
+ MOVE SPI-Prior-Token TO SPI-Current-Division
+ EXIT PARAGRAPH
+ END-IF
+ IF SPI-Prior-Token = "PROGRAM-ID"
+ MOVE SPACES TO SPI-Prior-Token
+ MOVE SPI-Current-Token TO SPI-Current-Program-ID
+ IF SPI-CP-13-15 NOT = SPACES
+ MOVE "..." TO SPI-CP-13-15
+ END-IF
+ EXIT PARAGRAPH
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 330-ENVIRONMENT-DIVISION.
+ IF Token-Is-Key-Word AND SPI-Current-Token = "DIVISION"
+ MOVE SPI-Prior-Token TO SPI-Current-Division
+ EXIT PARAGRAPH
+ END-IF
+ IF Token-Is-Key-Word AND SPI-Current-Token = "SECTION"
+ MOVE SPI-Prior-Token TO SPI-Current-Section
+ EXIT PARAGRAPH
+ END-IF
+ IF Token-Is-Identifier
+ PERFORM 361-Release-Ref
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 340-DATA-DIVISION.
+ IF Token-Is-Key-Word AND SPI-Current-Token = "DIVISION"
+ MOVE SPI-Prior-Token TO SPI-Current-Division
+ EXIT PARAGRAPH
+ END-IF
+ IF Token-Is-Key-Word AND SPI-Current-Token = "SECTION"
+ MOVE SPI-Prior-Token TO SPI-Current-Section
+ EXIT PARAGRAPH
+ END-IF
+ IF (SPI-Current-Token = "PIC" OR "PICTURE")
+ AND (Token-Is-Key-Word)
+ MOVE "Y" TO F-Processing-PICTURE
+ EXIT PARAGRAPH
+ END-IF
+GC0710 IF Token-Is-Reserved-Word
+GC0710 AND SPI-Prior-Token = "LEVEL #"
+GC0710 MOVE SPACES TO SPI-Prior-Token
+GC0710 EXIT PARAGRAPH
+GC0710 END-IF
+ IF Token-Is-Identifier
+ EVALUATE SPI-Prior-Token
+ WHEN "FD"
+ PERFORM 360-Release-Def
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "SD"
+ PERFORM 360-Release-Def
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "LEVEL #"
+ PERFORM 360-Release-Def
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "INDEXED"
+ PERFORM 360-Release-Def
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "USING"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "INTO"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ EXIT PARAGRAPH
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 350-PROCEDURE-DIVISION.
+ IF SPI-Current-Section NOT = "PROCEDURE"
+ MOVE "PROCEDURE" TO SPI-Current-Section
+ END-IF
+GC0710 IF SPI-Current-Token-UC = "PROGRAM"
+GC0710 AND SPI-Prior-Token = "END"
+GC0710 MOVE "?" TO SPI-Current-Division
+GC0710 EXIT PARAGRAPH
+GC0710 END-IF
+ IF Token-Is-Key-Word AND SPI-Current-Token = "DIVISION"
+ MOVE SPI-Prior-Token TO SPI-Current-Division
+ EXIT PARAGRAPH
+ END-IF
+ IF SPI-Current-Verb = SPACES
+GC0710 AND F-Verb-Has-Been-Found = "Y"
+ IF Token-Is-Identifier
+ PERFORM 360-Release-Def
+ MOVE SPACES TO SPI-Prior-Token
+ END-IF
+ EXIT PARAGRAPH
+ END-IF
+ IF NOT Token-Is-Identifier
+ EXIT PARAGRAPH
+ END-IF
+ EVALUATE SPI-Current-Verb
+ WHEN "ACCEPT"
+ PERFORM 351-ACCEPT
+ WHEN "ADD"
+ PERFORM 351-ADD
+ WHEN "ALLOCATE"
+ PERFORM 351-ALLOCATE
+ WHEN "CALL"
+ PERFORM 351-CALL
+ WHEN "COMPUTE"
+ PERFORM 351-COMPUTE
+ WHEN "DIVIDE"
+ PERFORM 351-DIVIDE
+ WHEN "FREE"
+ PERFORM 351-FREE
+ WHEN "INITIALIZE"
+ PERFORM 351-INITIALIZE
+ WHEN "INSPECT"
+ PERFORM 351-INSPECT
+ WHEN "MOVE"
+ PERFORM 351-MOVE
+ WHEN "MULTIPLY"
+ PERFORM 351-MULTIPLY
+ WHEN "PERFORM"
+ PERFORM 351-PERFORM
+ WHEN "SET"
+ PERFORM 351-SET
+ WHEN "STRING"
+ PERFORM 351-STRING
+ WHEN "SUBTRACT"
+ PERFORM 351-SUBTRACT
+ WHEN "TRANSFORM"
+ PERFORM 351-TRANSFORM
+ WHEN "UNSTRING"
+ PERFORM 351-UNSTRING
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-ACCEPT.
+ EVALUATE SPI-Prior-Token
+ WHEN "ACCEPT"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-ADD.
+ EVALUATE SPI-Prior-Token
+ WHEN "GIVING"
+ PERFORM 362-Release-Upd
+ WHEN "TO"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-ALLOCATE.
+ EVALUATE SPI-Prior-Token
+ WHEN "ALLOCATE"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "RETURNING"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-CALL.
+ EVALUATE SPI-Prior-Token
+ WHEN "RETURNING"
+ PERFORM 362-Release-Upd
+ WHEN "GIVING"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-COMPUTE.
+ EVALUATE SPI-Prior-Token
+ WHEN "COMPUTE"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-DIVIDE.
+ EVALUATE SPI-Prior-Token
+ WHEN "INTO"
+ PERFORM 363-Set-Upd
+ MOVE Sort-Rec TO Held-Reference
+ WHEN "GIVING"
+ IF Held-Reference NOT = SPACES
+ MOVE Held-Reference To Sort-Rec
+ MOVE SPACES To Held-Reference
+ SR-Ref-Flag
+ RELEASE Sort-Rec
+ END-IF
+ PERFORM 362-Release-Upd
+ WHEN "REMAINDER"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-FREE.
+ PERFORM 362-Release-Upd
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-INITIALIZE.
+ EVALUATE SPI-Prior-Token
+ WHEN "INITIALIZE"
+ PERFORM 362-Release-Upd
+ WHEN "REPLACING"
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-INSPECT.
+ EVALUATE SPI-Prior-Token
+ WHEN "INSPECT"
+ PERFORM 364-Set-Ref
+ MOVE SPACES TO Held-Reference
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "TALLYING"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "REPLACING"
+ IF Held-Reference NOT = SPACES
+ MOVE Held-Reference TO Sort-Rec
+ MOVE SPACES TO Held-Reference
+ MOVE "*" TO SR-Ref-Flag
+ RELEASE Sort-Rec
+ END-IF
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "CONVERTING"
+ IF Held-Reference NOT = SPACES
+ MOVE Held-Reference TO Sort-Rec
+ MOVE SPACES TO Held-Reference
+ MOVE "*" TO SR-Ref-Flag
+ RELEASE Sort-Rec
+ END-IF
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-MOVE.
+ EVALUATE SPI-Prior-Token
+ WHEN "TO"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-MULTIPLY.
+ EVALUATE SPI-Prior-Token
+ WHEN "BY"
+ PERFORM 363-Set-Upd
+ MOVE Sort-Rec TO Held-Reference
+ WHEN "GIVING"
+ MOVE Held-Reference TO Sort-Rec
+ MOVE SPACES TO Held-Reference
+ SR-Ref-Flag
+ RELEASE Sort-Rec
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-PERFORM.
+ EVALUATE SPI-Prior-Token
+ WHEN "VARYING"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN "AFTER"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-SET.
+ EVALUATE SPI-Prior-Token
+ WHEN "SET"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-STRING.
+ EVALUATE SPI-Prior-Token
+ WHEN "INTO"
+ PERFORM 362-Release-Upd
+ WHEN "POINTER"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-SUBTRACT.
+ EVALUATE SPI-Prior-Token
+ WHEN "GIVING"
+ PERFORM 362-Release-Upd
+ WHEN "FROM"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-TRANSFORM.
+ EVALUATE SPI-Prior-Token
+ WHEN "TRANSFORM"
+ PERFORM 362-Release-Upd
+ MOVE SPACES TO SPI-Prior-Token
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 351-UNSTRING.
+ EVALUATE SPI-Prior-Token
+ WHEN "INTO"
+ PERFORM 362-Release-Upd
+ WHEN "DELIMITER"
+ PERFORM 362-Release-Upd
+ WHEN "COUNT"
+ PERFORM 362-Release-Upd
+ WHEN "POINTER"
+ PERFORM 362-Release-Upd
+ WHEN "TALLYING"
+ PERFORM 362-Release-Upd
+ WHEN OTHER
+ PERFORM 361-Release-Ref
+ END-EVALUATE
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 360-Release-Def.
+ MOVE SPACES TO Sort-Rec
+ MOVE SPI-Current-Program-ID TO SR-Prog-ID
+ MOVE SPI-Current-Token-UC TO SR-Token-UC
+ MOVE SPI-Current-Token TO SR-Token
+ MOVE SPI-Current-Section TO SR-Section
+ MOVE SPI-Current-Line-No TO SR-Line-No-Def
+ MOVE 0 TO SR-Line-No-Ref
+ RELEASE Sort-Rec
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 361-Release-Ref.
+ PERFORM 364-Set-Ref
+ RELEASE Sort-Rec
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 362-Release-Upd.
+ PERFORM 363-Set-Upd
+ RELEASE Sort-Rec
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 363-Set-Upd.
+ MOVE SPACES TO Sort-Rec
+ MOVE SPI-Current-Program-ID TO SR-Prog-ID
+ MOVE SPI-Current-Token-UC TO SR-Token-UC
+ MOVE SPI-Current-Token TO SR-Token
+ MOVE SPI-Current-Section TO SR-Section
+ MOVE SPI-Current-Line-No TO SR-Line-No-Ref
+ MOVE "*" TO SR-Ref-Flag
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 364-Set-Ref.
+ MOVE SPACES TO Sort-Rec
+ MOVE SPI-Current-Program-ID TO SR-Prog-ID
+ MOVE SPI-Current-Token-UC TO SR-Token-UC
+ MOVE SPI-Current-Token TO SR-Token
+ MOVE SPI-Current-Section TO SR-Section
+ MOVE SPI-Current-Line-No TO SR-Line-No-Ref
+ .
+ /
+ 400-Produce-Xref-Listing SECTION.
+ 401-Init.
+ MOVE SPACES TO Detail-Line-X
+ Group-Indicators
+ MOVE 0 TO I
+ Lines-Left
+GC0710 MOVE 'N' TO F-Duplicate
+ .
+
+ 402-Process-Sorted-Recs.
+ PERFORM FOREVER
+ RETURN Sort-File AT END
+ EXIT PERFORM
+ END-RETURN
+ IF SR-Prog-ID NOT = GI-Prog-ID
+ OR SR-Token-UC NOT = GI-Token
+GC0710 MOVE 'N' TO F-Duplicate
+ IF Detail-Line-X NOT = SPACES
+ PERFORM 410-Generate-Report-Line
+ END-IF
+ IF SR-Prog-ID NOT = GI-Prog-ID
+ MOVE 0 TO Lines-Left
+ END-IF
+ MOVE SR-Prog-ID TO GI-Prog-ID
+ MOVE SR-Token-UC TO GI-Token
+ END-IF
+GC0710 IF SR-Token-UC = GI-Token
+GC0710 AND SR-Line-No-Def NOT = SPACES
+GC0710 AND Detail-Line-X NOT = SPACES
+GC0710 MOVE 'Y' TO F-Duplicate
+GC0710 PERFORM 410-Generate-Report-Line
+GC0710 MOVE 0 TO I
+GC0710 MOVE SR-Prog-ID TO DLX-Prog-ID
+GC0710 MOVE ' (Duplicate Definition)' TO DLX-Token
+GC0710 MOVE SR-Section TO DLX-Section
+GC0710 MOVE SR-Line-No-Def TO DLX-Line-No-Def
+GC0710 EXIT PERFORM CYCLE
+GC0710 END-IF
+GC0710 IF SR-Token-UC = GI-Token
+GC0710 AND SR-Line-No-Def = SPACES
+GC0710 AND F-Duplicate = 'Y'
+GC0710 MOVE 'N' TO F-Duplicate
+GC0710 PERFORM 410-Generate-Report-Line
+GC0710 MOVE 0 TO I
+GC0710 MOVE SR-Prog-ID TO DLX-Prog-ID
+GC0710 MOVE ' (Duplicate References)' TO DLX-Token
+GC0710 END-IF
+ IF Detail-Line-X = SPACES
+ MOVE SR-Prog-ID TO DLX-Prog-ID
+ MOVE SR-Token TO DLX-Token
+ MOVE SR-Section TO DLX-Section
+ IF SR-Line-No-Def NOT = SPACES
+ MOVE SR-Line-No-Def TO DLX-Line-No-Def
+ END-IF
+ END-IF
+ IF SR-Reference > '000000'
+ ADD 1 TO I
+ IF I > Line-Nos-Per-Rec
+ PERFORM 410-Generate-Report-Line
+ MOVE 1 TO I
+ END-IF
+ MOVE SR-Line-No-Ref TO DLX-Line-No-Ref (I)
+ MOVE SR-Ref-Flag TO DLX-Ref-Flag (I)
+ END-IF
+ END-PERFORM
+ IF Detail-Line-X NOT = SPACES
+ PERFORM 410-Generate-Report-Line
+ END-IF
+ EXIT SECTION
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 410-Generate-Report-Line.
+ IF Lines-Left < 1
+ IF F-First-Record = "Y"
+ MOVE "N" TO F-First-Record
+ WRITE Report-Rec FROM Heading-1X BEFORE 1
+ ELSE
+ MOVE SPACES TO Report-Rec
+ WRITE Report-Rec BEFORE PAGE
+ MOVE SPACES TO Report-Rec
+ WRITE Report-Rec BEFORE 1
+ WRITE Report-Rec FROM Heading-1X BEFORE 1
+ END-IF
+ WRITE Report-Rec FROM Heading-2 BEFORE 1
+ WRITE Report-Rec FROM Heading-4X BEFORE 1
+ WRITE Report-Rec FROM Heading-5X BEFORE 1
+ COMPUTE
+ Lines-Left = Lines-Per-Page - 4
+ END-COMPUTE
+ END-IF
+ WRITE Report-Rec FROM Detail-Line-X BEFORE 1
+ MOVE SPACES TO Detail-Line-X
+ MOVE 0 TO I
+ SUBTRACT 1 FROM Lines-Left
+ .
+ /
+ 500-Produce-Source-Listing SECTION.
+ 501-Generate-Source-Listing.
+ OPEN INPUT Source-Code
+ Expand-Code
+ MOVE 0 TO Source-Line-No
+ PERFORM FOREVER
+ READ Expand-Code AT END
+ EXIT PERFORM
+ END-READ
+ IF ECR-1 = "#"
+ PERFORM 510-Control-Record
+ ELSE
+ PERFORM 520-Expand-Code-Record
+ END-IF
+ END-PERFORM
+ CLOSE Source-Code
+ Expand-Code
+ EXIT SECTION
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 510-Control-Record.
+ UNSTRING ECR-2-256
+ DELIMITED BY '"'
+ INTO PIC-X10, PIC-X256, Dummy
+ END-UNSTRING
+ IF TRIM(PIC-X256,Trailing) = TRIM(Program-Path,Trailing) *> Main Pgm
+ SET In-Main-Module TO TRUE
+ IF Source-Line-No > 0
+ READ Expand-Code END-READ
+ END-IF
+ ELSE *> COPY
+ SET In-Copybook TO TRUE
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 520-Expand-Code-Record.
+ IF In-Main-Module
+ ADD 1 To SPI-Current-Line-No
+ READ Source-Code AT END NEXT SENTENCE END-READ
+ ADD 1 TO Source-Line-No
+ MOVE SPACES TO Detail-Line-S
+ MOVE Source-Line-No TO DLS-Line-No
+ MOVE SCR-1-128 TO DLS-Statement
+GC0410 IF SCR-7 = "/"
+GC0410 MOVE 0 TO Lines-Left
+GC0410 END-IF
+ PERFORM 530-Generate-Source-Line
+ IF SCR-129-256 NOT = SPACES
+ MOVE SPACES TO Detail-Line-S
+ MOVE SCR-129-256 TO DLS-Statement
+ PERFORM 530-Generate-Source-Line
+ END-IF
+ ELSE
+ IF Expand-Code-Rec NOT = SPACES
+ MOVE SPACES TO Detail-Line-S
+ MOVE ECR-1-128 TO DLS-Statement
+ PERFORM 530-Generate-Source-Line
+ IF ECR-129-256 NOT = SPACES
+ MOVE SPACES TO Detail-Line-S
+ MOVE ECR-129-256 TO DLS-Statement
+ PERFORM 530-Generate-Source-Line
+ END-IF
+ END-IF
+ END-IF
+ .
+ *>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+ 530-Generate-Source-Line.
+ IF Lines-Left < 1
+ IF F-First-Record = "Y"
+ MOVE "N" TO F-First-Record
+ WRITE Report-Rec FROM Heading-1S BEFORE 1
+ ELSE
+ MOVE SPACES TO Report-Rec
+ WRITE Report-Rec BEFORE PAGE
+ MOVE SPACES TO Report-Rec
+ WRITE Report-Rec BEFORE 1
+ WRITE Report-Rec FROM Heading-1S BEFORE 1
+ END-IF
+ WRITE Report-Rec FROM Heading-2 BEFORE 1
+ WRITE Report-Rec FROM Heading-4S BEFORE 1
+ WRITE Report-Rec FROM Heading-5S BEFORE 1
+ COMPUTE
+ Lines-Left = Lines-Per-Page - 4
+ END-COMPUTE
+ END-IF
+ WRITE Report-Rec FROM Detail-Line-S BEFORE 1
+ MOVE SPACES TO Detail-Line-S
+ SUBTRACT 1 FROM Lines-Left
+ .
+
+ END PROGRAM LISTING.
diff --git a/tests/examplefiles/example.monkey b/tests/examplefiles/example.monkey new file mode 100644 index 00000000..facd3a73 --- /dev/null +++ b/tests/examplefiles/example.monkey @@ -0,0 +1,152 @@ +Strict + +' single line comment + +#rem +multi +line +comment +#end + +#rem +nested +#rem +multi +line +#end +comment +#end + +Import mojo + +Const ONECONST:Int = 1 +Const TWOCONST := 2 +Const THREECONST := 3, FOURCONST:Int = 4 + +Global someVariable:Int = 4 + +' sample class from the documentation +Class Game Extends App + + Function New() + End + + Function DrawSpiral(clock) + Local w=DeviceWidth/2 + For Local i#=0 Until w*1.5 Step .2 + Local x#,y# + x=w+i*Sin(i*3+clock) + y=w+i*Cos(i*2+clock) + DrawRect x,y,1,1 + Next + hitbox.Collide(event.pos) + End + + Field updateCount + + Method OnCreate() + Print "spiral" + + SetUpdateRate 60 + End + + Method OnUpdate() + updateCount+=1 + End + + Method OnRender() + Cls + DrawSpiral updateCount + DrawSpiral updateCount*1.1 + End + +End + +Class Enemy + Method Die () Abstract +End + +' extending +Class Hoodlum Extends Enemy + ' field + Field testField:Bool = True + + ' naming class with modulepath + Local currentNode:list.Node<Vector2D> + + Method Die () + Print "B'oss, he-- he killed me, b'oss!" + End +End + +' extending with generics +Class VectorNode Extends Node<Vector2D> +End + +' interfaces +Interface Computer + Method Boot () + Method Process () + Method Display () +End + +Class PC Implements Computer +End + +' array syntax +Global listOfStuff:String[42] +Global lessStuff:String[5] = listOfStuff[4..8] +Global oneStuff:String = listOfStuff[23] + +'a comma separated sequence +Global scores:Int[]=[10,20,30] +'a comma separated sequence +Global text:String[]=["Hello","There","World"] +Global worstCase:worst.List<String[]> + +' string type +Global string1:String = "Hello world" +Global string2$ = "Hello world" + +' escape characers in strings +Global string3 := "Hello~zWorld" +Global string4 := "~qHello World~q" +Global string5 := "~tIndented~n" +Global string6 := "tilda is wavey... ~~" + +' string pseudofunctions +Print " Hello World ~n".Trim() ' prints "Hello World" +Print "Hello World".ToUpper() ' prints "HELLO WORLD" + +' Boolean shorttype +Global boolVariable1:Bool = True +Global boolVariable2? = False + +' number formats +Global hexNum1:Int = $3d0dead +Global hexNum2% = $CAFEBABE + +Global floatNum1:Float = 3.141516 +Global floatNum2# = 3.141516 +Global floatNum3 := .141516 + +' preprocessor keywords +#If TARGET = "android" +DoStuff() +#ElseIf TARGET = "ios" +DoOtherStuff() +#End + +' preprocessor variable +#SOMETHING = True +#Print SOMETHING +#If SOMETHING +#End + +' operators +Global a = 32 +Global b = 32 ~ 0 +b ~= 16 +b |= 16 +b &= 16 +Global c = a | b diff --git a/tests/examplefiles/example.prg b/tests/examplefiles/example.prg new file mode 100644 index 00000000..b70d9e42 --- /dev/null +++ b/tests/examplefiles/example.prg @@ -0,0 +1,161 @@ +&& This is a concatenation of all VFP examples on Wikipedia. +&& Copyright 2013 Wikimedia, under the GFDL. + +FOR i = 1 to 10 + x = x + 6.5 +ENDFOR + +IF i = 25 + i = i + 1 +ELSE + i = i + 3 +ENDIF + +x = 1 +DO WHILE x < 50 + x = x + 1 +ENDDO + +x = 1 +DO WHILE .T. + x = x + 1 + IF x < 50 + LOOP + ELSE + EXIT + ENDIF +ENDDO + +nMonth = MONTH(DATE()) +DO CASE + CASE nMonth <= 3 + MESSAGEBOX("Q1") + + CASE nMonth <= 6 + MESSAGEBOX("Q2") + + CASE nMonth <= 9 + MESSAGEBOX("Q3") + + OTHERWISE + MESSAGEBOX("Q4") +ENDCASE + +FOR EACH oControl IN THISFORM.Controls + MESSAGEBOX(oControl.Name) +ENDFOR + +f = Factorial(10) + +FUNCTION Factorial(n) + LOCAL i,r + r = 1 + FOR i = n TO 1 STEP -1 + r = r * n + ENDFOR + RETURN r +ENDFUNC + +loForm = CREATEOBJECT("HiForm") +loForm.Show(1) + +DEFINE CLASS HiForm AS Form + AutoCenter = .T. + Caption = "Hello, World" + + ADD OBJECT lblHi as Label WITH ; + Caption = "Hello, World!" +ENDDEFINE + +loMine = CREATEOBJECT("MyClass") +? loMine.cProp1 && This will work. (Double-ampersand marks an end-of-line comment) +? loMine.cProp2 && Program Error: Property CPROP2 is not found. + +? loMine.MyMethod1() && This will work. +? loMine.MyMethod2() && Program Error: Property MYMETHOD2 is not found. + +DEFINE CLASS MyClass AS Custom + cProp1 = "My Property" && This is a public property + HIDDEN cProp2 && This is a private (hidden) property + dProp3 = {} && Another public property + + PROCEDURE Init() && Class constructor + This.cProp2 = "This is a hidden property." + ENDPROC + + PROCEDURE dProp3_Access && Property Getter + RETURN DATE() + ENDPROC + PROCEDURE dProp3_Assign(vNewVal) && Property Setter + IF VARTYPE(vNewVal) = "D" + THIS.dProp3 = vNewVal + ENDIF + ENDPROC + + PROCEDURE MyMethod1() + * This is a public method, calling a hidden method that returns + * the value of a hidden property. + RETURN This.MyMethod2() + ENDPROC + + HIDDEN PROCEDURE MyMethod2() && This is a private (hidden) method + RETURN This.cProp2 + ENDPROC +ENDDEFINE + +&& Create a table +CREATE TABLE randData (iData I) + +&& Populate with random data using xBase and SQL DML commands +FOR i = 1 TO 50 + APPEND BLANK + REPLACE iData WITH (RAND() * 100) + + INSERT INTO randData (iData) VALUES (RAND() * 100) +ENDFOR + +&& Place a structural index on the data +INDEX ON iData TAG iData +CLOSE ALL + +&& Display ordered data using xBase-style commands +USE randData +SET ORDER TO iData +GO TOP +LIST NEXT 10 && First 10 +GO BOTTOM +SKIP -10 +LIST REST && Last 10 +CLOSE ALL + +&& Browse ordered data using SQL DML commands +SELECT * ; + FROM randData ; + ORDER BY iData DESCENDING + + +&& Connect to an ODBC data source +LOCAL nHnd +nHnd = SQLCONNECT ("ODBCDSN", "user", "pwd") + +&& Execute a SQL command +LOCAL nResult +nResult = SQLEXEC (nHnd, "USE master") +IF nResult < 0 + MESSAGEBOX ("MASTER database does not exist!") + RETURN +ENDIF + +&& Retrieve data from the remote server and stores it in +&& a local data cursor +nResult = SQLEXEC (nHnd, "SELECT * FROM authors", "QAUTHORS") + +&& Update a record in a remote table using parameters +PRIVATE cAuthorID, cAuthorName +cAuthorID = "1001" +cAuthorName = "New name" +nResult = SQLEXEC (nHnd, "UPDATE authors SET auth_name = ?cAuthorName WHERE auth_id = ?cAuthorID") + +&& Close the connection +SQLDISCONNECT(nHnd) + diff --git a/tests/examplefiles/example.reg b/tests/examplefiles/example.reg new file mode 100644 index 00000000..bc4e9df4 --- /dev/null +++ b/tests/examplefiles/example.reg @@ -0,0 +1,19 @@ +Windows Registry Editor Version 5.00 +; comment + +[HKEY_CURRENT_USER\SOFTWARE\Pygments] +@="Hello" +"Key With Spaces"="Something" +"Key With ="="With Quotes" +"Key With = 2"=dword:123 +"Key" = "Value" +"Hex"=hex(0):1,2,3,a,b,f +"Hex 2"=hex(5):80,00,00,ff + +[-HKEY_CURRENT_USER\SOFTWARE\Pygments\Subkey] + +[HKEY_CURRENT_USER\SOFTWARE\Pygments\Subkey2] +; comment +@=- +"Foo"=- +"Foo"="Value" diff --git a/tests/examplefiles/example.shell-session b/tests/examplefiles/example.shell-session new file mode 100644 index 00000000..66984aa5 --- /dev/null +++ b/tests/examplefiles/example.shell-session @@ -0,0 +1,45 @@ +[user@linuxbox imx-bootlets-src-10.05.02]$ make CROSS_COMPILE=arm-none-eabi- clean +rm -rf *.sb +rm -f sd_mmc_bootstream.raw +rm -f linux_prep/board/*.o +... +Files: +rm -f power_prep.o eabi.o +Build output: +make[1]: Leaving directory `/home/...' +[user@linuxbox imx-bootlets-src-10.05.02]$ make CROSS_COMPILE=arm-none-eabi- +make[1]: Entering directory `/home/...' +... +#@echo "generating U-Boot boot stream image" +#elftosb2 -z -c ./uboot_prebuilt.db -o imx23_uboot.sb +echo "generating kernel bootstream file sd_mmc_bootstream.raw" +generating kernel bootstream file sd_mmc_bootstream.raw +#Please use cfimager to burn xxx_linux.sb. The below way will no +#work at imx28 platform. +> test +$ test +rm -f sd_mmc_bootstream.raw +[user@linuxbox imx-bootlets-src-10.05.02]$ +pi@raspberrypi ~ $ sudo sh -c "echo 17 > /sys/class/gpio/export" +pi@raspberrypi ~ $ sudo sh -c "echo out > /sys/class/gpio/gpio17/direction" +pi@raspberrypi ~ $ sudo sh -c "echo 1 > /sys/class/gpio/gpio17/value" +pi@raspberrypi ~ $ sudo sh -c "echo 0 > /sys/class/gpio/gpio17/value" +pi@raspberrypi ~ $ +[user@linuxbox ~]$ # copy other stuff to the SD card +root@imx233-olinuxino-micro:~# lsmod + Not tainted +[user@linuxbox ~]$ tail -n 2 /mnt/rpi/etc/inittab +#Spawn a getty on Raspberry Pi serial line +T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100 +pi@raspberrypi:~/Adafruit-WebIDE$ mkdir tmp +pi@raspberrypi:~/Adafruit-WebIDE$ npm config set tmp tmp +pi@raspberrypi:~/Adafruit-WebIDE$ npm install +pi@raspberrypi ~/Adafruit-WebIDE $ ifconfig eth0 +eth0 Link encap:Ethernet HWaddr b5:33:ff:33:ee:aq + inet addr:10.42.0.60 Bcast:10.42.0.255 Mask:255.255.255.0 + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:21867 errors:0 dropped:0 overruns:0 frame:0 + TX packets:8684 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:1000 + RX bytes:27338495 (26.0 MiB) TX bytes:1268356 (1.2 MiB) + diff --git a/tests/examplefiles/example.ts b/tests/examplefiles/example.ts new file mode 100644 index 00000000..545c6cf5 --- /dev/null +++ b/tests/examplefiles/example.ts @@ -0,0 +1,28 @@ +class Animal { + constructor(public name) { } + move(meters) { + alert(this.name + " moved " + meters + "m."); + } +} + +class Snake extends Animal { + constructor(name) { super(name); } + move() { + alert("Slithering..."); + super.move(5); + } +} + +class Horse extends Animal { + constructor(name) { super(name); } + move() { + alert("Galloping..."); + super.move(45); + } +} + +var sam = new Snake("Sammy the Python") +var tom: Animal = new Horse("Tommy the Palomino") + +sam.move() +tom.move(34) diff --git a/tests/examplefiles/hello.smali b/tests/examplefiles/hello.smali new file mode 100644 index 00000000..e539f00e --- /dev/null +++ b/tests/examplefiles/hello.smali @@ -0,0 +1,40 @@ +# To Recreate: +# +# echo -e 'class hello {\n public static void main(String[] args) {\n +# System.out.println("hi");\n }\n}\n' > hello.java +# javac -target 1.4 -source 1.4 hello.java +# dx --dex --output=hello.dex hello.class +# baksmali hello.dex +# cat out/hello.smali + +.class Lhello; +.super Ljava/lang/Object; +.source "hello.java" + + +# direct methods +.method constructor <init>()V + .registers 1 + + .prologue + .line 1 + invoke-direct {p0}, Ljava/lang/Object;-><init>()V + + return-void +.end method + +.method public static main([Ljava/lang/String;)V + .registers 3 + .parameter + + .prologue + .line 3 + sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream; + + const-string v1, "hi" + + invoke-virtual {v0, v1}, Ljava/io/PrintStream;->println(Ljava/lang/String;)V + + .line 4 + return-void +.end method diff --git a/tests/examplefiles/inet_pton6.dg b/tests/examplefiles/inet_pton6.dg new file mode 100644 index 00000000..4104b3e7 --- /dev/null +++ b/tests/examplefiles/inet_pton6.dg @@ -0,0 +1,71 @@ +re = import! +sys = import! + + +# IPv6address = hexpart [ ":" IPv4address ] +# IPv4address = 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT +# hexpart = [ hexseq ] [ "::" [ hexseq ] ] +# hexseq = hex4 *( ":" hex4) +# hex4 = 1*4HEXDIG +hexpart = r'({0}|)(?:::({0}|)|)'.format r'(?:[\da-f]{1,4})(?::[\da-f]{1,4})*' +addrv4 = r'(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})' +addrv6 = re.compile $ r'(?i)(?:{})(?::{})?$'.format hexpart addrv4 + + +# Parse a base-N number given a list of its digits. +# +# :param q: the number of digits in that numeral system +# +# :param digits: an iterable of integers in range [0..q] +# +# :return: a decimal integer +# +base_n = (q digits) -> foldl (x y) -> (x * q + y) 0 digits + + +# Parse a sequence of hexadecimal numbers +# +# :param q: a string of colon-separated base-16 integers +# +# :return: an iterable of Python ints +# +unhex = q -> q and map p -> (int p 16) (q.split ':') + + +# Parse an IPv6 address as specified in RFC 4291. +# +# :param address: a string, obviously. +# +# :return: an integer which, written in binary form, points to the same node. +# +inet_pton6 = address -> + raise $ ValueError 'not a valid IPv6 address' if not (match = addrv6.match address) + start, end, *ipv4 = match.groups! + + is_ipv4 = not $ None in ipv4 + shift = (7 - start.count ':' - 2 * is_ipv4) * 16 + + raise $ ValueError 'not a valid IPv6 address' if (end is None and shift) or shift < 0 + hexaddr = (base_n 0x10000 (unhex start) << shift) + base_n 0x10000 (unhex $ end or '') + (hexaddr << 32) + base_n 0x100 (map int ipv4) if is_ipv4 else hexaddr + + +inet6_type = q -> switch + not q = 'unspecified' + q == 1 = 'loopback' + (q >> 32) == 0x000000000000ffff = 'IPv4-mapped' + (q >> 64) == 0xfe80000000000000 = 'link-local' + (q >> 120) != 0x00000000000000ff = 'general unicast' + (q >> 112) % (1 << 4) == 0x0000000000000000 = 'multicast w/ reserved scope value' + (q >> 112) % (1 << 4) == 0x000000000000000f = 'multicast w/ reserved scope value' + (q >> 112) % (1 << 4) == 0x0000000000000001 = 'interface-local multicast' + (q >> 112) % (1 << 4) == 0x0000000000000004 = 'admin-local multicast' + (q >> 112) % (1 << 4) == 0x0000000000000005 = 'site-local multicast' + (q >> 112) % (1 << 4) == 0x0000000000000008 = 'organization-local multicast' + (q >> 112) % (1 << 4) == 0x000000000000000e = 'global multicast' + (q >> 112) % (1 << 4) != 0x0000000000000002 = 'multicast w/ unknown scope value' + (q >> 24) % (1 << 112) == 0x00000000000001ff = 'solicited-node multicast' + True = 'link-local multicast' + + +print $ (x -> (inet6_type x, hex x)) $ inet_pton6 $ sys.stdin.read!.strip! diff --git a/tests/examplefiles/logos_example.xm b/tests/examplefiles/logos_example.xm new file mode 100644 index 00000000..39753e23 --- /dev/null +++ b/tests/examplefiles/logos_example.xm @@ -0,0 +1,28 @@ +%hook ABC +- (id)a:(B)b { + %log; + return %orig(nil); +} +%end + +%subclass DEF: NSObject +- (id)init { + [%c(RuntimeAccessibleClass) alloc]; + return nil; +} +%end + +%group OptionalHooks +%hook ABC +- (void)release { + [self retain]; + %orig; +} +%end +%end + +%ctor { + %init; + if(OptionalCondition) + %init(OptionalHooks); +} diff --git a/tests/examplefiles/metagrammar.treetop b/tests/examplefiles/metagrammar.treetop new file mode 100644 index 00000000..acd6af63 --- /dev/null +++ b/tests/examplefiles/metagrammar.treetop @@ -0,0 +1,455 @@ +module Treetop + module Compiler + grammar Metagrammar + rule treetop_file + requires:(space? require_statement)* prefix:space? module_or_grammar suffix:space? { + def compile + requires.text_value + prefix.text_value + module_or_grammar.compile + suffix.text_value + end + } + end + + rule require_statement + prefix:space? "require" [ \t]+ [^\n\r]+ [\n\r] + end + + rule module_or_grammar + module_declaration / grammar + end + + rule module_declaration + prefix:('module' space name:([A-Z] alphanumeric_char* ('::' [A-Z] alphanumeric_char*)*) space) module_contents:(module_declaration / grammar) suffix:(space 'end') { + def compile + prefix.text_value + module_contents.compile + suffix.text_value + end + + def parser_name + prefix.name.text_value+'::'+module_contents.parser_name + end + } + end + + rule grammar + 'grammar' space grammar_name space ('do' space)? declaration_sequence space? 'end' <Grammar> + end + + rule grammar_name + ([A-Z] alphanumeric_char*) + end + + rule declaration_sequence + head:declaration tail:(space declaration)* <DeclarationSequence> { + def declarations + [head] + tail + end + + def tail + super.elements.map { |elt| elt.declaration } + end + } + / + '' { + def compile(builder) + end + } + end + + rule declaration + parsing_rule / include_declaration + end + + rule include_declaration + 'include' space [A-Z] (alphanumeric_char / '::')* { + def compile(builder) + builder << text_value + end + } + end + + rule parsing_rule + 'rule' space nonterminal space ('do' space)? parsing_expression space 'end' <ParsingRule> + end + + rule parsing_expression + choice / sequence / primary + end + + rule choice + head:alternative tail:(space? '/' space? alternative)+ <Choice> { + def alternatives + [head] + tail + end + + def tail + super.elements.map {|elt| elt.alternative} + end + + def inline_modules + (alternatives.map {|alt| alt.inline_modules }).flatten + end + } + end + + rule sequence + head:labeled_sequence_primary tail:(space labeled_sequence_primary)+ node_class_declarations <Sequence> { + def sequence_elements + [head] + tail + end + + def tail + super.elements.map {|elt| elt.labeled_sequence_primary } + end + + def inline_modules + (sequence_elements.map {|elt| elt.inline_modules}).flatten + + [sequence_element_accessor_module] + + node_class_declarations.inline_modules + end + + def inline_module_name + node_class_declarations.inline_module_name + end + } + end + + rule alternative + sequence / primary + end + + rule primary + prefix atomic { + def compile(address, builder, parent_expression=nil) + prefix.compile(address, builder, self) + end + + def prefixed_expression + atomic + end + + def inline_modules + atomic.inline_modules + end + + def inline_module_name + nil + end + } + / + prefix space? predicate_block { + def compile(address, builder, parent_expression=nil) + prefix.compile(address, builder, self) + end + def prefixed_expression + predicate_block + end + def inline_modules + [] + end + } + / + atomic suffix node_class_declarations { + def compile(address, builder, parent_expression=nil) + suffix.compile(address, builder, self) + end + + def optional_expression + atomic + end + + def node_class_name + node_class_declarations.node_class_name + end + + def inline_modules + atomic.inline_modules + node_class_declarations.inline_modules + end + + def inline_module_name + node_class_declarations.inline_module_name + end + } + / + atomic node_class_declarations { + def compile(address, builder, parent_expression=nil) + atomic.compile(address, builder, self) + end + + def node_class_name + node_class_declarations.node_class_name + end + + def inline_modules + atomic.inline_modules + node_class_declarations.inline_modules + end + + def inline_module_name + node_class_declarations.inline_module_name + end + } + end + + rule labeled_sequence_primary + label sequence_primary { + def compile(lexical_address, builder) + sequence_primary.compile(lexical_address, builder) + end + + def inline_modules + sequence_primary.inline_modules + end + + def label_name + if label.name + label.name + elsif sequence_primary.instance_of?(Nonterminal) + sequence_primary.text_value + else + nil + end + end + } + end + + rule label + (alpha_char alphanumeric_char*) ':' { + def name + elements[0].text_value + end + } + / + '' { + def name + nil + end + } + end + + rule sequence_primary + prefix atomic { + def compile(lexical_address, builder) + prefix.compile(lexical_address, builder, self) + end + + def prefixed_expression + elements[1] + end + + def inline_modules + atomic.inline_modules + end + + def inline_module_name + nil + end + } + / + prefix space? predicate_block { + def compile(address, builder, parent_expression=nil) + prefix.compile(address, builder, self) + end + def prefixed_expression + predicate_block + end + def inline_modules + [] + end + } + / + atomic suffix { + def compile(lexical_address, builder) + suffix.compile(lexical_address, builder, self) + end + + def node_class_name + nil + end + + def inline_modules + atomic.inline_modules + end + + def inline_module_name + nil + end + } + / + atomic + end + + rule suffix + repetition_suffix / optional_suffix + end + + rule optional_suffix + '?' <Optional> + end + + rule node_class_declarations + node_class_expression trailing_inline_module { + def node_class_name + node_class_expression.node_class_name + end + + def inline_modules + trailing_inline_module.inline_modules + end + + def inline_module + trailing_inline_module.inline_module + end + + def inline_module_name + inline_module.module_name if inline_module + end + } + end + + rule repetition_suffix + '+' <OneOrMore> / '*' <ZeroOrMore> / occurrence_range + end + + rule occurrence_range + space? min:([0-9])* '..' max:([0-9])* <OccurrenceRange> + end + + rule prefix + '&' <AndPredicate> / '!' <NotPredicate> / '~' <TransientPrefix> + end + + rule atomic + terminal + / + nonterminal + / + parenthesized_expression + end + + rule parenthesized_expression + '(' space? parsing_expression space? ')' <ParenthesizedExpression> { + def inline_modules + parsing_expression.inline_modules + end + } + end + + rule nonterminal + !keyword_inside_grammar (alpha_char alphanumeric_char*) <Nonterminal> + end + + rule terminal + quoted_string / character_class / anything_symbol + end + + rule quoted_string + (single_quoted_string / double_quoted_string) { + def string + super.text_value + end + } + end + + rule double_quoted_string + '"' string:(!'"' ("\\\\" / '\"' / .))* '"' <Terminal> + end + + rule single_quoted_string + "'" string:(!"'" ("\\\\" / "\\'" / .))* "'" <Terminal> + end + + rule character_class + '[' characters:(!']' ('\\' . / bracket_expression / !'\\' .))+ ']' <CharacterClass> { + def characters + super.text_value + end + } + end + + rule bracket_expression + '[:' '^'? ( + 'alnum' / 'alpha' / 'blank' / 'cntrl' / 'digit' / 'graph' / 'lower' / + 'print' / 'punct' / 'space' / 'upper' / 'xdigit' / 'word' + ) ':]' + end + + rule anything_symbol + '.' <AnythingSymbol> + end + + rule node_class_expression + space '<' (!'>' .)+ '>' { + def node_class_name + elements[2].text_value + end + } + / + '' { + def node_class_name + nil + end + } + end + + rule trailing_inline_module + space inline_module { + def inline_modules + [inline_module] + end + + def inline_module_name + inline_module.module_name + end + } + / + '' { + def inline_modules + [] + end + + def inline_module + nil + end + + def inline_module_name + nil + end + } + end + + rule predicate_block + '' inline_module <PredicateBlock> + end + + rule inline_module + '{' (inline_module / ![{}] .)* '}' <InlineModule> + end + + rule keyword_inside_grammar + ('rule' / 'end') !non_space_char + end + + rule non_space_char + !space . + end + + rule alpha_char + [A-Za-z_] + end + + rule alphanumeric_char + alpha_char / [0-9] + end + + rule space + (white / comment_to_eol)+ + end + + rule comment_to_eol + '#' (!"\n" .)* + end + + rule white + [ \t\n\r] + end + end + end +end diff --git a/tests/examplefiles/mg_sample.pro b/tests/examplefiles/mg_sample.pro new file mode 100644 index 00000000..814d510d --- /dev/null +++ b/tests/examplefiles/mg_sample.pro @@ -0,0 +1,73 @@ +; docformat = 'rst' + +; Example IDL (Interactive Data Language) source code. + +;+ +; Get `nIndices` random indices for an array of size `nValues` (without +; repeating an index). +; +; :Examples: +; Try:: +; +; IDL> r = randomu(seed, 10) +; IDL> print, r, format='(4F)' +; 0.6297589 0.7815896 0.2508559 0.7546844 +; 0.1353382 0.1245834 0.8733745 0.0753110 +; 0.8054136 0.9513228 +; IDL> ind = mg_sample(10, 3, seed=seed) +; IDL> print, ind +; 2 4 7 +; IDL> print, r[ind] +; 0.250856 0.135338 0.0753110 +; +; :Returns: +; lonarr(`nIndices`) +; +; :Params: +; nValues : in, required, type=long +; size of array to choose indices from +; nIndices : in, required, type=long +; number of indices needed +; +; :Keywords: +; seed : in, out, optional, type=integer or lonarr(36) +; seed to use for random number generation, leave undefined to use a +; seed generated from the system clock; new seed will be output +;- +function mg_sample, nValues, nIndices, seed=seed + compile_opt strictarr + + ; get random nIndices by finding the indices of the smallest nIndices in a + ; array of random values + values = randomu(seed, nValues) + + ; our random values are uniformly distributed, so ideally the nIndices + ; smallest values are in the first bin of the below histogram + nBins = nValues / nIndices + h = histogram(values, nbins=nBins, reverse_indices=ri) + + ; the candidates for being in the first nIndices will live in bins 0..bin + nCandidates = 0L + for bin = 0L, nBins - 1L do begin + nCandidates += h[bin] + if (nCandidates ge nIndices) then break + endfor + + ; get the candidates and sort them + candidates = ri[ri[0] : ri[bin + 1L] - 1L] + sortedCandidates = sort(values[candidates]) + + ; return the first nIndices of them + return, (candidates[sortedCandidates])[0:nIndices-1L] +end + + +; main-level example program + +r = randomu(seed, 10) +print, r +ind = mg_sample(10, 3, seed=seed) +print, ind +print, r[ind] + +end
\ No newline at end of file diff --git a/tests/examplefiles/minehunt.qml b/tests/examplefiles/minehunt.qml new file mode 100644 index 00000000..548e7e89 --- /dev/null +++ b/tests/examplefiles/minehunt.qml @@ -0,0 +1,112 @@ + /**************************************************************************** + ** + ** Copyright (C) 2011 Nokia Corporation and/or its subsidiary(-ies). + ** All rights reserved. + ** Contact: Nokia Corporation (qt-info@nokia.com) + ** + ** This file is part of the QtDeclarative module of the Qt Toolkit. + ** + ** $QT_BEGIN_LICENSE:LGPL$ + ** GNU Lesser General Public License Usage + ** This file may be used under the terms of the GNU Lesser General Public + ** License version 2.1 as published by the Free Software Foundation and + ** appearing in the file LICENSE.LGPL included in the packaging of this + ** file. Please review the following information to ensure the GNU Lesser + ** General Public License version 2.1 requirements will be met: + ** http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html. + ** + ** In addition, as a special exception, Nokia gives you certain additional + ** rights. These rights are described in the Nokia Qt LGPL Exception + ** version 1.1, included in the file LGPL_EXCEPTION.txt in this package. + ** + ** GNU General Public License Usage + ** Alternatively, this file may be used under the terms of the GNU General + ** Public License version 3.0 as published by the Free Software Foundation + ** and appearing in the file LICENSE.GPL included in the packaging of this + ** file. Please review the following information to ensure the GNU General + ** Public License version 3.0 requirements will be met: + ** http://www.gnu.org/copyleft/gpl.html. + ** + ** Other Usage + ** Alternatively, this file may be used in accordance with the terms and + ** conditions contained in a signed written agreement between you and Nokia. + ** + ** + ** + ** + ** + ** $QT_END_LICENSE$ + ** + ****************************************************************************/ + + import QtQuick 1.0 + import "MinehuntCore" 1.0 + + Item { + id: field + property int clickx: 0 + property int clicky: 0 + + width: 450; height: 450 + + Image { source: "MinehuntCore/pics/background.png"; anchors.fill: parent; fillMode: Image.Tile } + + Grid { + anchors.horizontalCenter: parent.horizontalCenter + columns: 9; spacing: 1 + + Repeater { + id: repeater + model: tiles + delegate: Tile {} + } + } + + Row { + id: gamedata + x: 20; spacing: 20 + anchors.bottom: field.bottom; anchors.bottomMargin: 15 + + Image { + source: "MinehuntCore/pics/quit.png" + scale: quitMouse.pressed ? 0.8 : 1.0 + smooth: quitMouse.pressed + y: 10 + MouseArea { + id: quitMouse + anchors.fill: parent + anchors.margins: -20 + onClicked: Qt.quit() + } + } + Column { + spacing: 2 + Image { source: "MinehuntCore/pics/bomb-color.png" } + Text { anchors.horizontalCenter: parent.horizontalCenter; color: "white"; text: numMines } + } + + Column { + spacing: 2 + Image { source: "MinehuntCore/pics/flag-color.png" } + Text { anchors.horizontalCenter: parent.horizontalCenter; color: "white"; text: numFlags } + } + } + + Image { + anchors.bottom: field.bottom; anchors.bottomMargin: 15 + anchors.right: field.right; anchors.rightMargin: 20 + source: isPlaying ? 'MinehuntCore/pics/face-smile.png' : + hasWon ? 'MinehuntCore/pics/face-smile-big.png': 'MinehuntCore/pics/face-sad.png' + + MouseArea { anchors.fill: parent; onPressed: reset() } + } + Text { + anchors.centerIn: parent; width: parent.width - 20 + horizontalAlignment: Text.AlignHCenter + wrapMode: Text.WordWrap + text: "Minehunt demo has to be compiled to run.\n\nPlease see README." + color: "white"; font.bold: true; font.pixelSize: 14 + visible: tiles == undefined + } + + } diff --git a/tests/examplefiles/nanomsg.intr b/tests/examplefiles/nanomsg.intr new file mode 100644 index 00000000..d21f62cc --- /dev/null +++ b/tests/examplefiles/nanomsg.intr @@ -0,0 +1,95 @@ +module: nanomsg +synopsis: generated bindings for the nanomsg library +author: Bruce Mitchener, Jr. +copyright: See LICENSE file in this distribution. + +define simple-C-mapped-subtype <C-buffer-offset> (<C-char*>) + export-map <machine-word>, export-function: identity; +end; + +define interface + #include { + "sp/sp.h", + "sp/fanin.h", + "sp/inproc.h", + "sp/pair.h", + "sp/reqrep.h", + "sp/survey.h", + "sp/fanout.h", + "sp/ipc.h", + "sp/pubsub.h", + "sp/tcp.h" + }, + + exclude: { + "SP_HAUSNUMERO", + "SP_PAIR_ID", + "SP_PUBSUB_ID", + "SP_REQREP_ID", + "SP_FANIN_ID", + "SP_FANOUT_ID", + "SP_SURVEY_ID" + }, + + equate: {"char *" => <c-string>}, + + rename: { + "sp_recv" => %sp-recv, + "sp_send" => %sp-send, + "sp_setsockopt" => %sp-setsockopt + }; + + function "sp_version", + output-argument: 1, + output-argument: 2, + output-argument: 3; + + function "sp_send", + map-argument: { 2 => <C-buffer-offset> }; + + function "sp_recv", + map-argument: { 2 => <C-buffer-offset> }; + +end interface; + +// Function for adding the base address of the repeated slots of a <buffer> +// to an offset and returning the result as a <machine-word>. This is +// necessary for passing <buffer> contents across the FFI. + +define function buffer-offset + (the-buffer :: <buffer>, data-offset :: <integer>) + => (result-offset :: <machine-word>) + u%+(data-offset, + primitive-wrap-machine-word + (primitive-repeated-slot-as-raw + (the-buffer, primitive-repeated-slot-offset(the-buffer)))) +end function; + +define inline function sp-send (socket :: <integer>, data :: <buffer>, flags :: <integer>) => (res :: <integer>) + %sp-send(socket, buffer-offset(data, 0), data.size, flags) +end; + +define inline function sp-recv (socket :: <integer>, data :: <buffer>, flags :: <integer>) => (res :: <integer>) + %sp-recv(socket, buffer-offset(data, 0), data.size, flags); +end; + +define inline method sp-setsockopt (socket :: <integer>, level :: <integer>, option :: <integer>, value :: <integer>) + with-stack-structure (int :: <C-int*>) + pointer-value(int) := value; + let setsockopt-result = + %sp-setsockopt(socket, level, option, int, size-of(<C-int*>)); + if (setsockopt-result < 0) + // Check error! + end; + setsockopt-result + end; +end; + +define inline method sp-setsockopt (socket :: <integer>, level :: <integer>, option :: <integer>, data :: <byte-string>) + let setsockopt-result = + %sp-setsockopt(socket, level, option, as(<c-string>, data), data.size); + if (setsockopt-result < 0) + // Check error! + end; + setsockopt-result +end; diff --git a/tests/examplefiles/phpMyAdmin.spec b/tests/examplefiles/phpMyAdmin.spec new file mode 100644 index 00000000..120fbc92 --- /dev/null +++ b/tests/examplefiles/phpMyAdmin.spec @@ -0,0 +1,163 @@ +%define _myadminpath /var/www/myadmin +%define pkgrelease rc1 +%define microrelease 1 + +Name: phpMyAdmin +Version: 3.1.1 +Release: %{pkgrelease}.%{microrelease} +License: GPL +Group: Applications/Databases/Interfaces +Source0: http://prdownloads.sourceforge.net/phpmyadmin/%{name}-%{version}-%{pkgrelease}.tar.bz2 +Source1: phpMyAdmin-http.conf +URL: http://sourceforge.net/projects/phpmyadmin/ +Requires: mysql +Requires: php-mysql +Buildarch: noarch +#BuildRoot: %{_tmppath}/%{name}-root + +Summary: phpMyAdmin - web-based MySQL administration + +%description +phpMyAdmin can manage a whole MySQL-server (needs a super-user) but +also a single database. To accomplish the latter you'll need a +properly set up MySQL-user which can read/write only the desired +database. It's up to you to look up the appropiate part in the MySQL +manual. Currently phpMyAdmin can: + - create and drop databases + - create, copy, drop and alter tables + - delete, edit and add fields + - execute any SQL-statement, even batch-queries + - manage keys on fields + - load text files into tables + - create (*) and read dumps of tables + - export (*) and import data to CSV values + - administer multiple servers and single databases + - check referencial integrity + - create complex queries automatically connecting required tables + - create PDF graphics of your database layout + - communicate in more than 38 different languages + + +%prep +%setup -q -n %{name}-%{version}-%{pkgrelease} + + +%build + + +%install +[ "${RPM_BUILD_ROOT}" != "/" ] && [ -d "${RPM_BUILD_ROOT}" ] && \ + rm -rf "${RPM_BUILD_ROOT}" + +# Create directories. + +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/{css,js,lang,libraries,themes} +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/libraries/{auth,dbg,dbi,engines} +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/libraries/{export,tcpdf,import} +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/libraries/transformations +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/libraries/tcpdf/font +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/themes/{darkblue_orange,original} +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/themes/darkblue_orange/{css,img} +install -d "${RPM_BUILD_ROOT}%{_myadminpath}"/themes/original/{css,img} + +# Install files. + +install libraries/config.default.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}"/config.inc.php +install *.{php,ico} "${RPM_BUILD_ROOT}%{_myadminpath}"/ +install ChangeLog LICENSE README "${RPM_BUILD_ROOT}%{_myadminpath}"/ +install Documentation.html docs.css "${RPM_BUILD_ROOT}%{_myadminpath}"/ +install css/* "${RPM_BUILD_ROOT}%{_myadminpath}/css"/ +install js/* "${RPM_BUILD_ROOT}%{_myadminpath}/js"/ +install lang/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/lang"/ +install libraries/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/libraries"/ +install libraries/auth/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/auth"/ +install libraries/dbg/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/dbg"/ +install libraries/dbi/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/dbi"/ +install libraries/engines/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/engines"/ +install libraries/export/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/export"/ +install libraries/tcpdf/*.php "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/tcpdf"/ +install libraries/tcpdf/font/*.{php,z} \ + "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/tcpdf/font"/ +install libraries/import/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/import"/ +install libraries/transformations/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/libraries/transformations"/ +install themes/darkblue_orange/*.{php,png} \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/darkblue_orange"/ +install themes/darkblue_orange/css/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/darkblue_orange/css"/ +install themes/darkblue_orange/img/*.{png,ico} \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/darkblue_orange/img"/ +install themes/original/*.{php,png} \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/original"/ +install themes/original/css/*.php \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/original/css"/ +install themes/original/img/*.{png,ico} \ + "${RPM_BUILD_ROOT}%{_myadminpath}/themes/original/img"/ + +# Create documentation directories. + +DOCROOT="${RPM_BUILD_ROOT}%{_docdir}/%{name}-%{version}" +install -d "${DOCROOT}" +install -d "${DOCROOT}"/{lang,scripts,transformations} + +# Install documentation files. + +install RELEASE-DATE-* "${DOCROOT}"/ +install CREDITS ChangeLog INSTALL LICENSE "${DOCROOT}"/ +install README TODO "${DOCROOT}"/ +install Documentation.* docs.css "${DOCROOT}"/ +install translators.html "${DOCROOT}"/ +install lang/*.sh "${DOCROOT}"/lang/ +install scripts/* "${DOCROOT}"/scripts/ +install libraries/tcpdf/README "${DOCROOT}"/README.tcpdf +install libraries/import/README "${DOCROOT}"/README.import +install libraries/transformations/README "${DOCROOT}"/transformations/ +install libraries/transformations/TEMPLATE* "${DOCROOT}"/transformations/ +install libraries/transformations/*.sh "${DOCROOT}"/transformations/ + +# Install configuration file for Apache. + +install -d "${RPM_BUILD_ROOT}%{_sysconfdir}/httpd/conf.d" +install "%{SOURCE1}" \ + "${RPM_BUILD_ROOT}%{_sysconfdir}/httpd/conf.d/phpMyAdmin.conf" + +# Generate non-configuration file list. + +(cd "${RPM_BUILD_ROOT}"; ls -d ."%{_myadminpath}"/*) | + sed -e '/\/config\.inc\.php$/d' -e 's/^.//' > files.list + + + +%clean +[ "${RPM_BUILD_ROOT}" != "/" ] && [ -d "${RPM_BUILD_ROOT}" ] && \ + rm -rf "${RPM_BUILD_ROOT}" + + +%files -f files.list +%defattr(644, root, root, 755) +%doc %{_docdir}/%{name}-%{version} +%dir %{_myadminpath} +%attr(640,root,apache) %config(noreplace) %verify(not size mtime md5) %{_myadminpath}/config.inc.php +%config(noreplace) %verify(not size mtime md5) %{_sysconfdir}/httpd/conf.d/* + + +%changelog +* Thu Feb 23 2006 Patrick Monnerat <pm@datasphere.ch> +- Version 2.8.0-rc1.1. + +* Thu Dec 22 2005 Patrick Monnerat <patrick.monnerat@econophone.ch> +- Path "nullpw" to allow trying connection with null password after failure. +- Version 2.7.0-pl1.1. + +* Mon Aug 22 2005 Patrick Monnerat <patrick.monnerat@econophone.ch> +- Version 2.6.3-pl1. + +* Wed Jul 21 2004 Patrick Monnerat <patrick.monnerat@econophone.ch> +- Version 2.5.7-pl1. + +* Fri Nov 22 2002 Patrick Monnerat <patrick.monnerat@econophone.ch> +- Version 2.3.0-rc1. diff --git a/tests/examplefiles/pytb_test3.pytb b/tests/examplefiles/pytb_test3.pytb new file mode 100644 index 00000000..6947c1ef --- /dev/null +++ b/tests/examplefiles/pytb_test3.pytb @@ -0,0 +1,4 @@ +>>> 3/"3" +Traceback (most recent call last): + File "<stdin>", line 1, in <module> +TypeError: unsupported operand type(s) for /: 'int' and 'str' diff --git a/tests/examplefiles/robotframework.txt b/tests/examplefiles/robotframework.txt new file mode 100644 index 00000000..63ba63e6 --- /dev/null +++ b/tests/examplefiles/robotframework.txt @@ -0,0 +1,39 @@ +*** Settings *** +Documentation Simple example demonstrating syntax highlighting. +Library ExampleLibrary +Test Setup Keyword argument argument with ${VARIABLE} + +*** Variables *** +${VARIABLE} Variable value +@{LIST} List variable here + +*** Test Cases *** +Keyword-driven example + Initialize System + Do Something + Result Should Be 42 + [Teardown] Cleanup System + +Data-driven example + [Template] Keyword + argument1 argument2 + argument ${VARIABLE} + @{LIST} + +Gherkin + Given system is initialized + When something is done + Then result should be "42" + +| Pipes | +| | [Documentation] | Also pipe separated format is supported. | +| | Log | As this example demonstrates. | + +*** Keywords *** +Result Should Be + [Arguments] ${expected} + ${actual} = Get Value + Should be Equal ${actual} ${expected} + +Then result should be "${expected}" + Result Should Be ${expected} diff --git a/tests/examplefiles/rust_example.rs b/tests/examplefiles/rust_example.rs index af791fbc..1c0a70c3 100644 --- a/tests/examplefiles/rust_example.rs +++ b/tests/examplefiles/rust_example.rs @@ -1,743 +1,233 @@ - -#[doc = "json serialization"]; - -import result::{result, ok, err}; -import io; -import io::{reader_util, writer_util}; -import map; -import map::hashmap; - -export json; -export error; -export to_writer; -export to_str; -export from_reader; -export from_str; -export eq; - -export num; -export string; -export boolean; -export list; -export dict; -export null; - -#[doc = "Represents a json value"] -enum json { - num(float), - string(str), - boolean(bool), - list([json]), - dict(map::hashmap<str,json>), - null, +// Copyright 2012 The Rust Project Developers. See the COPYRIGHT +// file at the top-level directory of this distribution and at +// http://rust-lang.org/COPYRIGHT. +// +// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or +// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license +// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your +// option. This file may not be copied, modified, or distributed +// except according to those terms. + +// based on: +// http://shootout.alioth.debian.org/u32/benchmark.php?test=nbody&lang=java + +extern mod std; + +use core::os; + +// Using sqrt from the standard library is way slower than using libc +// directly even though std just calls libc, I guess it must be +// because the the indirection through another dynamic linker +// stub. Kind of shocking. Might be able to make it faster still with +// an llvm intrinsic. +#[nolink] +extern mod libc { + #[legacy_exports]; + fn sqrt(n: float) -> float; } -type error = { - line: uint, - col: uint, - msg: str, -}; - -#[doc = "Serializes a json value into a io::writer"] -fn to_writer(wr: io::writer, j: json) { - alt j { - num(n) { wr.write_str(float::to_str(n, 6u)); } - string(s) { - wr.write_char('"'); - let mut escaped = ""; - str::chars_iter(s) { |c| - alt c { - '"' { escaped += "\\\""; } - '\\' { escaped += "\\\\"; } - '\x08' { escaped += "\\b"; } - '\x0c' { escaped += "\\f"; } - '\n' { escaped += "\\n"; } - '\r' { escaped += "\\r"; } - '\t' { escaped += "\\t"; } - _ { escaped += str::from_char(c); } - } - }; - wr.write_str(escaped); - wr.write_char('"'); - } - boolean(b) { - wr.write_str(if b { "true" } else { "false" }); - } - list(v) { - wr.write_char('['); - let mut first = true; - vec::iter(v) { |item| - if !first { - wr.write_str(", "); - } - first = false; - to_writer(wr, item); - }; - wr.write_char(']'); - } - dict(d) { - if d.size() == 0u { - wr.write_str("{}"); - ret; - } - - wr.write_str("{ "); - let mut first = true; - d.items { |key, value| - if !first { - wr.write_str(", "); - } - first = false; - to_writer(wr, string(key)); - wr.write_str(": "); - to_writer(wr, value); - }; - wr.write_str(" }"); - } - null { - wr.write_str("null"); - } - } +fn main() { + let args = os::args(); + let args = if os::getenv(~"RUST_BENCH").is_some() { + ~[~"", ~"4000000"] + } else if args.len() <= 1u { + ~[~"", ~"100000"] + } else { + args + }; + let n = int::from_str(args[1]).get(); + let mut bodies: ~[Body::props] = NBodySystem::make(); + io::println(fmt!("%f", NBodySystem::energy(bodies))); + let mut i = 0; + while i < n { + NBodySystem::advance(bodies, 0.01); + i += 1; + } + io::println(fmt!("%f", NBodySystem::energy(bodies))); } -#[doc = "Serializes a json value into a string"] -fn to_str(j: json) -> str { - io::with_str_writer { |wr| to_writer(wr, j) } -} +mod NBodySystem { + use Body; -type parser = { - rdr: io::reader, - mut ch: char, - mut line: uint, - mut col: uint, -}; + pub fn make() -> ~[Body::props] { + let mut bodies: ~[Body::props] = + ~[Body::sun(), + Body::jupiter(), + Body::saturn(), + Body::uranus(), + Body::neptune()]; -impl parser for parser { - fn eof() -> bool { self.ch == -1 as char } + let mut px = 0.0; + let mut py = 0.0; + let mut pz = 0.0; - fn bump() { - self.ch = self.rdr.read_char(); + let mut i = 0; + while i < 5 { + px += bodies[i].vx * bodies[i].mass; + py += bodies[i].vy * bodies[i].mass; + pz += bodies[i].vz * bodies[i].mass; - if self.ch == '\n' { - self.line += 1u; - self.col = 1u; - } else { - self.col += 1u; + i += 1; } - } - fn next_char() -> char { - self.bump(); - self.ch - } + // side-effecting + Body::offset_momentum(&mut bodies[0], px, py, pz); - fn error<T>(msg: str) -> result<T, error> { - err({ line: self.line, col: self.col, msg: msg }) + return bodies; } - fn parse() -> result<json, error> { - alt self.parse_value() { - ok(value) { - // Skip trailing whitespaces. - self.parse_whitespace(); - // Make sure there is no trailing characters. - if self.eof() { - ok(value) - } else { - self.error("trailing characters") + pub fn advance(bodies: &mut [Body::props], dt: float) { + let mut i = 0; + while i < 5 { + let mut j = i + 1; + while j < 5 { + advance_one(&mut bodies[i], + &mut bodies[j], dt); + j += 1; } - } - e { e } - } - } - fn parse_value() -> result<json, error> { - self.parse_whitespace(); - - if self.eof() { ret self.error("EOF while parsing value"); } - - alt self.ch { - 'n' { self.parse_ident("ull", null) } - 't' { self.parse_ident("rue", boolean(true)) } - 'f' { self.parse_ident("alse", boolean(false)) } - '0' to '9' | '-' { self.parse_number() } - '"' { - alt self.parse_str() { - ok(s) { ok(string(s)) } - err(e) { err(e) } - } - } - '[' { self.parse_list() } - '{' { self.parse_object() } - _ { self.error("invalid syntax") } + i += 1; } - } - fn parse_whitespace() { - while char::is_whitespace(self.ch) { self.bump(); } - } - - fn parse_ident(ident: str, value: json) -> result<json, error> { - if str::all(ident, { |c| c == self.next_char() }) { - self.bump(); - ok(value) - } else { - self.error("invalid syntax") + i = 0; + while i < 5 { + move_(&mut bodies[i], dt); + i += 1; } } - fn parse_number() -> result<json, error> { - let mut neg = 1f; + pub fn advance_one(bi: &mut Body::props, + bj: &mut Body::props, + dt: float) unsafe { + let dx = bi.x - bj.x; + let dy = bi.y - bj.y; + let dz = bi.z - bj.z; - if self.ch == '-' { - self.bump(); - neg = -1f; - } + let dSquared = dx * dx + dy * dy + dz * dz; - let mut res = alt self.parse_integer() { - ok(res) { res } - err(e) { ret err(e); } - }; + let distance = ::libc::sqrt(dSquared); + let mag = dt / (dSquared * distance); - if self.ch == '.' { - alt self.parse_decimal(res) { - ok(r) { res = r; } - err(e) { ret err(e); } - } - } - - if self.ch == 'e' || self.ch == 'E' { - alt self.parse_exponent(res) { - ok(r) { res = r; } - err(e) { ret err(e); } - } - } + bi.vx -= dx * bj.mass * mag; + bi.vy -= dy * bj.mass * mag; + bi.vz -= dz * bj.mass * mag; - ok(num(neg * res)) + bj.vx += dx * bi.mass * mag; + bj.vy += dy * bi.mass * mag; + bj.vz += dz * bi.mass * mag; } - fn parse_integer() -> result<float, error> { - let mut res = 0f; - - alt self.ch { - '0' { - self.bump(); - - // There can be only one leading '0'. - alt self.ch { - '0' to '9' { ret self.error("invalid number"); } - _ {} - } - } - '1' to '9' { - while !self.eof() { - alt self.ch { - '0' to '9' { - res *= 10f; - res += ((self.ch as int) - ('0' as int)) as float; - - self.bump(); - } - _ { break; } - } - } - } - _ { ret self.error("invalid number"); } - } - - ok(res) + pub fn move_(b: &mut Body::props, dt: float) { + b.x += dt * b.vx; + b.y += dt * b.vy; + b.z += dt * b.vz; } - fn parse_decimal(res: float) -> result<float, error> { - self.bump(); + pub fn energy(bodies: &[Body::props]) -> float unsafe { + let mut dx; + let mut dy; + let mut dz; + let mut distance; + let mut e = 0.0; - // Make sure a digit follows the decimal place. - alt self.ch { - '0' to '9' {} - _ { ret self.error("invalid number"); } - } - - let mut res = res; - let mut dec = 1f; - while !self.eof() { - alt self.ch { - '0' to '9' { - dec /= 10f; - res += (((self.ch as int) - ('0' as int)) as float) * dec; - - self.bump(); - } - _ { break; } - } - } - - ok(res) - } - - fn parse_exponent(res: float) -> result<float, error> { - self.bump(); + let mut i = 0; + while i < 5 { + e += + 0.5 * bodies[i].mass * + (bodies[i].vx * bodies[i].vx + bodies[i].vy * bodies[i].vy + + bodies[i].vz * bodies[i].vz); - let mut res = res; - let mut exp = 0u; - let mut neg_exp = false; + let mut j = i + 1; + while j < 5 { + dx = bodies[i].x - bodies[j].x; + dy = bodies[i].y - bodies[j].y; + dz = bodies[i].z - bodies[j].z; - alt self.ch { - '+' { self.bump(); } - '-' { self.bump(); neg_exp = true; } - _ {} - } - - // Make sure a digit follows the exponent place. - alt self.ch { - '0' to '9' {} - _ { ret self.error("invalid number"); } - } - - while !self.eof() { - alt self.ch { - '0' to '9' { - exp *= 10u; - exp += (self.ch as uint) - ('0' as uint); + distance = ::libc::sqrt(dx * dx + dy * dy + dz * dz); + e -= bodies[i].mass * bodies[j].mass / distance; - self.bump(); - } - _ { break; } + j += 1; } - } - let exp = float::pow_with_uint(10u, exp); - if neg_exp { - res /= exp; - } else { - res *= exp; + i += 1; } + return e; - ok(res) - } - - fn parse_str() -> result<str, error> { - let mut escape = false; - let mut res = ""; - - while !self.eof() { - self.bump(); - - if (escape) { - alt self.ch { - '"' { str::push_char(res, '"'); } - '\\' { str::push_char(res, '\\'); } - '/' { str::push_char(res, '/'); } - 'b' { str::push_char(res, '\x08'); } - 'f' { str::push_char(res, '\x0c'); } - 'n' { str::push_char(res, '\n'); } - 'r' { str::push_char(res, '\r'); } - 't' { str::push_char(res, '\t'); } - 'u' { - // Parse \u1234. - let mut i = 0u; - let mut n = 0u; - while i < 4u { - alt self.next_char() { - '0' to '9' { - n = n * 10u + - (self.ch as uint) - ('0' as uint); - } - _ { ret self.error("invalid \\u escape"); } - } - i += 1u; - } - - // Error out if we didn't parse 4 digits. - if i != 4u { - ret self.error("invalid \\u escape"); - } - - str::push_char(res, n as char); - } - _ { ret self.error("invalid escape"); } - } - escape = false; - } else if self.ch == '\\' { - escape = true; - } else { - if self.ch == '"' { - self.bump(); - ret ok(res); - } - str::push_char(res, self.ch); - } - } - - self.error("EOF while parsing string") - } - - fn parse_list() -> result<json, error> { - self.bump(); - self.parse_whitespace(); - - let mut values = []; - - if self.ch == ']' { - self.bump(); - ret ok(list(values)); - } - - loop { - alt self.parse_value() { - ok(v) { vec::push(values, v); } - e { ret e; } - } - - self.parse_whitespace(); - if self.eof() { - ret self.error("EOF while parsing list"); - } - - alt self.ch { - ',' { self.bump(); } - ']' { self.bump(); ret ok(list(values)); } - _ { ret self.error("expecting ',' or ']'"); } - } - }; - } - - fn parse_object() -> result<json, error> { - self.bump(); - self.parse_whitespace(); - - let values = map::str_hash(); - - if self.ch == '}' { - self.bump(); - ret ok(dict(values)); - } - - while !self.eof() { - self.parse_whitespace(); - - if self.ch != '"' { - ret self.error("key must be a string"); - } - - let key = alt self.parse_str() { - ok(key) { key } - err(e) { ret err(e); } - }; - - self.parse_whitespace(); - - if self.ch != ':' { - if self.eof() { break; } - ret self.error("expecting ':'"); - } - self.bump(); - - alt self.parse_value() { - ok(value) { values.insert(key, value); } - e { ret e; } - } - self.parse_whitespace(); - - alt self.ch { - ',' { self.bump(); } - '}' { self.bump(); ret ok(dict(values)); } - _ { - if self.eof() { break; } - ret self.error("expecting ',' or '}'"); - } - } - } - - ret self.error("EOF while parsing object"); - } -} - -#[doc = "Deserializes a json value from an io::reader"] -fn from_reader(rdr: io::reader) -> result<json, error> { - let parser = { - rdr: rdr, - mut ch: rdr.read_char(), - mut line: 1u, - mut col: 1u, - }; - - parser.parse() -} - -#[doc = "Deserializes a json value from a string"] -fn from_str(s: str) -> result<json, error> { - io::with_str_reader(s, from_reader) -} - -#[doc = "Test if two json values are equal"] -fn eq(value0: json, value1: json) -> bool { - alt (value0, value1) { - (num(f0), num(f1)) { f0 == f1 } - (string(s0), string(s1)) { s0 == s1 } - (boolean(b0), boolean(b1)) { b0 == b1 } - (list(l0), list(l1)) { vec::all2(l0, l1, eq) } - (dict(d0), dict(d1)) { - if d0.size() == d1.size() { - let mut equal = true; - d0.items { |k, v0| - alt d1.find(k) { - some(v1) { - if !eq(v0, v1) { equal = false; } } - none { equal = false; } - } - }; - equal - } else { - false - } - } - (null, null) { true } - _ { false } } } -#[cfg(test)] -mod tests { - fn mk_dict(items: [(str, json)]) -> json { - let d = map::str_hash(); - - vec::iter(items) { |item| - let (key, value) = item; - d.insert(key, value); - }; - - dict(d) - } - - #[test] - fn test_write_null() { - assert to_str(null) == "null"; - } - - #[test] - fn test_write_num() { - assert to_str(num(3f)) == "3"; - assert to_str(num(3.1f)) == "3.1"; - assert to_str(num(-1.5f)) == "-1.5"; - assert to_str(num(0.5f)) == "0.5"; - } - - #[test] - fn test_write_str() { - assert to_str(string("")) == "\"\""; - assert to_str(string("foo")) == "\"foo\""; - } - - #[test] - fn test_write_bool() { - assert to_str(boolean(true)) == "true"; - assert to_str(boolean(false)) == "false"; - } - - #[test] - fn test_write_list() { - assert to_str(list([])) == "[]"; - assert to_str(list([boolean(true)])) == "[true]"; - assert to_str(list([ - boolean(false), - null, - list([string("foo\nbar"), num(3.5f)]) - ])) == "[false, null, [\"foo\\nbar\", 3.5]]"; - } - - #[test] - fn test_write_dict() { - assert to_str(mk_dict([])) == "{}"; - assert to_str(mk_dict([("a", boolean(true))])) == "{ \"a\": true }"; - assert to_str(mk_dict([ - ("a", boolean(true)), - ("b", list([ - mk_dict([("c", string("\x0c\r"))]), - mk_dict([("d", string(""))]) - ])) - ])) == - "{ " + - "\"a\": true, " + - "\"b\": [" + - "{ \"c\": \"\\f\\r\" }, " + - "{ \"d\": \"\" }" + - "]" + - " }"; +mod Body { + use Body; + + pub const PI: float = 3.141592653589793; + pub const SOLAR_MASS: float = 39.478417604357432; + // was 4 * PI * PI originally + pub const DAYS_PER_YEAR: float = 365.24; + + pub type props = + {mut x: float, + mut y: float, + mut z: float, + mut vx: float, + mut vy: float, + mut vz: float, + mass: float}; + + pub fn jupiter() -> Body::props { + return {mut x: 4.84143144246472090e+00, + mut y: -1.16032004402742839e+00, + mut z: -1.03622044471123109e-01, + mut vx: 1.66007664274403694e-03 * DAYS_PER_YEAR, + mut vy: 7.69901118419740425e-03 * DAYS_PER_YEAR, + mut vz: -6.90460016972063023e-05 * DAYS_PER_YEAR, + mass: 9.54791938424326609e-04 * SOLAR_MASS}; + } + + pub fn saturn() -> Body::props { + return {mut x: 8.34336671824457987e+00, + mut y: 4.12479856412430479e+00, + mut z: -4.03523417114321381e-01, + mut vx: -2.76742510726862411e-03 * DAYS_PER_YEAR, + mut vy: 4.99852801234917238e-03 * DAYS_PER_YEAR, + mut vz: 2.30417297573763929e-05 * DAYS_PER_YEAR, + mass: 2.85885980666130812e-04 * SOLAR_MASS}; + } + + pub fn uranus() -> Body::props { + return {mut x: 1.28943695621391310e+01, + mut y: -1.51111514016986312e+01, + mut z: -2.23307578892655734e-01, + mut vx: 2.96460137564761618e-03 * DAYS_PER_YEAR, + mut vy: 2.37847173959480950e-03 * DAYS_PER_YEAR, + mut vz: -2.96589568540237556e-05 * DAYS_PER_YEAR, + mass: 4.36624404335156298e-05 * SOLAR_MASS}; + } + + pub fn neptune() -> Body::props { + return {mut x: 1.53796971148509165e+01, + mut y: -2.59193146099879641e+01, + mut z: 1.79258772950371181e-01, + mut vx: 2.68067772490389322e-03 * DAYS_PER_YEAR, + mut vy: 1.62824170038242295e-03 * DAYS_PER_YEAR, + mut vz: -9.51592254519715870e-05 * DAYS_PER_YEAR, + mass: 5.15138902046611451e-05 * SOLAR_MASS}; + } + + pub fn sun() -> Body::props { + return {mut x: 0.0, + mut y: 0.0, + mut z: 0.0, + mut vx: 0.0, + mut vy: 0.0, + mut vz: 0.0, + mass: SOLAR_MASS}; + } + + pub fn offset_momentum(props: &mut Body::props, + px: float, py: float, pz: float) { + props.vx = -px / SOLAR_MASS; + props.vy = -py / SOLAR_MASS; + props.vz = -pz / SOLAR_MASS; } - #[test] - fn test_trailing_characters() { - assert from_str("nulla") == - err({line: 1u, col: 5u, msg: "trailing characters"}); - assert from_str("truea") == - err({line: 1u, col: 5u, msg: "trailing characters"}); - assert from_str("falsea") == - err({line: 1u, col: 6u, msg: "trailing characters"}); - assert from_str("1a") == - err({line: 1u, col: 2u, msg: "trailing characters"}); - assert from_str("[]a") == - err({line: 1u, col: 3u, msg: "trailing characters"}); - assert from_str("{}a") == - err({line: 1u, col: 3u, msg: "trailing characters"}); - } - - #[test] - fn test_read_identifiers() { - assert from_str("n") == - err({line: 1u, col: 2u, msg: "invalid syntax"}); - assert from_str("nul") == - err({line: 1u, col: 4u, msg: "invalid syntax"}); - - assert from_str("t") == - err({line: 1u, col: 2u, msg: "invalid syntax"}); - assert from_str("truz") == - err({line: 1u, col: 4u, msg: "invalid syntax"}); - - assert from_str("f") == - err({line: 1u, col: 2u, msg: "invalid syntax"}); - assert from_str("faz") == - err({line: 1u, col: 3u, msg: "invalid syntax"}); - - assert from_str("null") == ok(null); - assert from_str("true") == ok(boolean(true)); - assert from_str("false") == ok(boolean(false)); - assert from_str(" null ") == ok(null); - assert from_str(" true ") == ok(boolean(true)); - assert from_str(" false ") == ok(boolean(false)); - } - - #[test] - fn test_read_num() { - assert from_str("+") == - err({line: 1u, col: 1u, msg: "invalid syntax"}); - assert from_str(".") == - err({line: 1u, col: 1u, msg: "invalid syntax"}); - - assert from_str("-") == - err({line: 1u, col: 2u, msg: "invalid number"}); - assert from_str("00") == - err({line: 1u, col: 2u, msg: "invalid number"}); - assert from_str("1.") == - err({line: 1u, col: 3u, msg: "invalid number"}); - assert from_str("1e") == - err({line: 1u, col: 3u, msg: "invalid number"}); - assert from_str("1e+") == - err({line: 1u, col: 4u, msg: "invalid number"}); - - assert from_str("3") == ok(num(3f)); - assert from_str("3.1") == ok(num(3.1f)); - assert from_str("-1.2") == ok(num(-1.2f)); - assert from_str("0.4") == ok(num(0.4f)); - assert from_str("0.4e5") == ok(num(0.4e5f)); - assert from_str("0.4e+15") == ok(num(0.4e15f)); - assert from_str("0.4e-01") == ok(num(0.4e-01f)); - assert from_str(" 3 ") == ok(num(3f)); - } - - #[test] - fn test_read_str() { - assert from_str("\"") == - err({line: 1u, col: 2u, msg: "EOF while parsing string"}); - assert from_str("\"lol") == - err({line: 1u, col: 5u, msg: "EOF while parsing string"}); - - assert from_str("\"\"") == ok(string("")); - assert from_str("\"foo\"") == ok(string("foo")); - assert from_str("\"\\\"\"") == ok(string("\"")); - assert from_str("\"\\b\"") == ok(string("\x08")); - assert from_str("\"\\n\"") == ok(string("\n")); - assert from_str("\"\\r\"") == ok(string("\r")); - assert from_str("\"\\t\"") == ok(string("\t")); - assert from_str(" \"foo\" ") == ok(string("foo")); - } - - #[test] - fn test_read_list() { - assert from_str("[") == - err({line: 1u, col: 2u, msg: "EOF while parsing value"}); - assert from_str("[1") == - err({line: 1u, col: 3u, msg: "EOF while parsing list"}); - assert from_str("[1,") == - err({line: 1u, col: 4u, msg: "EOF while parsing value"}); - assert from_str("[1,]") == - err({line: 1u, col: 4u, msg: "invalid syntax"}); - assert from_str("[6 7]") == - err({line: 1u, col: 4u, msg: "expecting ',' or ']'"}); - - assert from_str("[]") == ok(list([])); - assert from_str("[ ]") == ok(list([])); - assert from_str("[true]") == ok(list([boolean(true)])); - assert from_str("[ false ]") == ok(list([boolean(false)])); - assert from_str("[null]") == ok(list([null])); - assert from_str("[3, 1]") == ok(list([num(3f), num(1f)])); - assert from_str("\n[3, 2]\n") == ok(list([num(3f), num(2f)])); - assert from_str("[2, [4, 1]]") == - ok(list([num(2f), list([num(4f), num(1f)])])); - } - - #[test] - fn test_read_dict() { - assert from_str("{") == - err({line: 1u, col: 2u, msg: "EOF while parsing object"}); - assert from_str("{ ") == - err({line: 1u, col: 3u, msg: "EOF while parsing object"}); - assert from_str("{1") == - err({line: 1u, col: 2u, msg: "key must be a string"}); - assert from_str("{ \"a\"") == - err({line: 1u, col: 6u, msg: "EOF while parsing object"}); - assert from_str("{\"a\"") == - err({line: 1u, col: 5u, msg: "EOF while parsing object"}); - assert from_str("{\"a\" ") == - err({line: 1u, col: 6u, msg: "EOF while parsing object"}); - - assert from_str("{\"a\" 1") == - err({line: 1u, col: 6u, msg: "expecting ':'"}); - assert from_str("{\"a\":") == - err({line: 1u, col: 6u, msg: "EOF while parsing value"}); - assert from_str("{\"a\":1") == - err({line: 1u, col: 7u, msg: "EOF while parsing object"}); - assert from_str("{\"a\":1 1") == - err({line: 1u, col: 8u, msg: "expecting ',' or '}'"}); - assert from_str("{\"a\":1,") == - err({line: 1u, col: 8u, msg: "EOF while parsing object"}); - - assert eq(result::get(from_str("{}")), mk_dict([])); - assert eq(result::get(from_str("{\"a\": 3}")), - mk_dict([("a", num(3.0f))])); - - assert eq(result::get(from_str("{ \"a\": null, \"b\" : true }")), - mk_dict([("a", null), ("b", boolean(true))])); - assert eq(result::get(from_str("\n{ \"a\": null, \"b\" : true }\n")), - mk_dict([("a", null), ("b", boolean(true))])); - assert eq(result::get(from_str("{\"a\" : 1.0 ,\"b\": [ true ]}")), - mk_dict([ - ("a", num(1.0)), - ("b", list([boolean(true)])) - ])); - assert eq(result::get(from_str( - "{" + - "\"a\": 1.0, " + - "\"b\": [" + - "true," + - "\"foo\\nbar\", " + - "{ \"c\": {\"d\": null} } " + - "]" + - "}")), - mk_dict([ - ("a", num(1.0f)), - ("b", list([ - boolean(true), - string("foo\nbar"), - mk_dict([ - ("c", mk_dict([("d", null)])) - ]) - ])) - ])); - } - - #[test] - fn test_multiline_errors() { - assert from_str("{\n \"foo\":\n \"bar\"") == - err({line: 3u, col: 8u, msg: "EOF while parsing object"}); - } } diff --git a/tests/examplefiles/session.dylan-console b/tests/examplefiles/session.dylan-console new file mode 100644 index 00000000..6f289c8e --- /dev/null +++ b/tests/examplefiles/session.dylan-console @@ -0,0 +1,9 @@ +? 7 * 52; +=> 364 +? define variable *your-variable* = $foo; +? begin + let yours = "apple"; + let mine = yours; + mine == yours; + end; +=> #t diff --git a/tests/examplefiles/test.opa b/tests/examplefiles/test.opa new file mode 100644 index 00000000..ec287ac5 --- /dev/null +++ b/tests/examplefiles/test.opa @@ -0,0 +1,10 @@ +function sample_page() { + <header> + <h3>HTML in Opa</h3> + </header> + <article> + <div class=container> + <p>Learning by examples.</p> + </div> + </article> +} diff --git a/tests/examplefiles/test2.pypylog b/tests/examplefiles/test2.pypylog new file mode 100644 index 00000000..543e21dd --- /dev/null +++ b/tests/examplefiles/test2.pypylog @@ -0,0 +1,120 @@ +[2f1dd6c3b8b7] {jit-log-opt-loop +# Loop 0 (<Function object at 0xb720e550> ds1dr4 dsdr3 ds1dr4) : loop with 115 ops +[p0, p1] ++33: label(p0, p1, descr=TargetToken(-1223434224)) +debug_merge_point(0, 0, '<Function object at 0xb710b120> ds1dr4 dsdr3 ds1dr4') ++33: guard_nonnull_class(p1, 138371488, descr=<Guard2>) [p1, p0] ++54: p3 = getfield_gc_pure(p1, descr=<FieldP pyhaskell.interpreter.haskell.Substitution.inst_rhs 8>) ++57: guard_value(p3, ConstPtr(ptr4), descr=<Guard3>) [p1, p0, p3] ++69: p5 = getfield_gc_pure(p1, descr=<FieldP pyhaskell.interpreter.haskell.Substitution.inst_subst 12>) ++72: p7 = getarrayitem_gc(p5, 0, descr=<ArrayP 4>) ++75: guard_class(p7, 138371552, descr=<Guard4>) [p0, p5, p7] ++88: p9 = getfield_gc(p7, descr=<FieldP pyhaskell.interpreter.haskell.Thunk.inst_application 8>) ++91: guard_nonnull_class(p9, 138373024, descr=<Guard5>) [p0, p5, p7, p9] ++109: p12 = getarrayitem_gc(p5, 1, descr=<ArrayP 4>) ++112: guard_class(p12, 138371552, descr=<Guard6>) [p0, p5, p12, p7] ++125: p14 = getfield_gc(p12, descr=<FieldP pyhaskell.interpreter.haskell.Thunk.inst_application 8>) ++128: guard_nonnull_class(p14, 138373024, descr=<Guard7>) [p0, p5, p12, p14, p7] +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') ++146: p16 = getfield_gc_pure(p9, descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) ++149: guard_value(p16, ConstPtr(ptr17), descr=<Guard8>) [p16, p9, p0, p12, p7] ++161: p18 = getfield_gc_pure(p9, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg0 12>) ++164: guard_class(p18, 138371648, descr=<Guard9>) [p18, p9, p0, p12, p7] ++177: p20 = getfield_gc_pure(p9, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg1 16>) ++180: guard_class(p20, 138371648, descr=<Guard10>) [p20, p9, p18, p0, p12, p7] ++193: p22 = getfield_gc_pure(p9, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg2 20>) ++196: guard_class(p22, 138371936, descr=<Guard11>) [p22, p9, p20, p18, p0, p12, p7] +debug_merge_point(0, 0, 'None') ++209: p24 = getfield_gc_pure(p22, descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) ++215: guard_value(p24, ConstPtr(ptr25), descr=<Guard12>) [p24, p22, p9, None, None, p0, p12, p7] ++227: p27 = getfield_gc_pure(p22, descr=<FieldP pyhaskell.interpreter.haskell.Application1.inst_arg0 12>) ++230: guard_class(p27, 138371648, descr=<Guard13>) [p22, p27, p9, None, None, p0, p12, p7] +debug_merge_point(0, 0, '_') +debug_merge_point(0, 0, 'None') ++243: p30 = getfield_gc(ConstPtr(ptr29), descr=<FieldP pyhaskell.interpreter.module.CoreMod.inst_qvars 24>) ++249: i34 = call(ConstClass(ll_dict_lookup_trampoline__v64___simple_call__function_ll), p30, ConstPtr(ptr32), 360200661, descr=<Calli 4 rri EF=4>) ++281: guard_no_exception(, descr=<Guard14>) [p27, p20, p18, i34, p30, None, None, None, p0, p12, p7] ++294: i36 = int_and(i34, -2147483648) ++302: i37 = int_is_true(i36) +guard_false(i37, descr=<Guard15>) [p27, p20, p18, i34, p30, None, None, None, p0, p12, p7] ++311: p38 = getfield_gc(p30, descr=<FieldP dicttable.entries 12>) ++314: p39 = getinteriorfield_gc(p38, i34, descr=<InteriorFieldDescr <FieldP dictentry.value 4>>) ++318: i40 = instance_ptr_eq(p18, p39) +guard_true(i40, descr=<Guard16>) [p27, p20, None, None, None, p0, p12, p7] +debug_merge_point(0, 0, 'None') ++327: i41 = getfield_gc_pure(p20, descr=<FieldS pyhaskell.interpreter.primtype.Int.inst_value 8>) ++330: i42 = getfield_gc_pure(p27, descr=<FieldS pyhaskell.interpreter.primtype.Int.inst_value 8>) ++333: i43 = int_sub(i41, i42) +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') ++335: i45 = int_eq(0, i43) +guard_false(i45, descr=<Guard17>) [p0, i43, None, None, None, None, p12, p7] +p47 = new_with_vtable(138371648) ++393: setfield_gc(p47, i43, descr=<FieldS pyhaskell.interpreter.primtype.Int.inst_value 8>) +setfield_gc(p7, p47, descr=<FieldP pyhaskell.interpreter.haskell.Thunk.inst_application 8>) ++414: p48 = getfield_gc(p12, descr=<FieldP pyhaskell.interpreter.haskell.Thunk.inst_application 8>) ++420: guard_nonnull_class(p48, 138371648, descr=<Guard18>) [p0, p48, p12, p47, p7] +debug_merge_point(0, 0, '<PrimFunction object at 0x83f3f6c> 1 <Function object at 0xb710b3b0> 1 <Function object at 0xb710b3c0> <PrimFunction object at 0x83f3f3c> 1 dsdr3 <Function object at 0xb710b210> 1') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, '_') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, '<Function object at 0xb710b3d0> dsdr3 dsdr3') +debug_merge_point(0, 0, '<Function object at 0xb710b120> ds1dr4 dsdr3 ds1dr4') ++438: label(p0, p48, p30, p38, descr=TargetToken(-1223434176)) +debug_merge_point(0, 0, '<Function object at 0xb710b120> ds1dr4 dsdr3 ds1dr4') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, '_') +debug_merge_point(0, 0, 'None') ++438: i50 = call(ConstClass(ll_dict_lookup_trampoline__v64___simple_call__function_ll), p30, ConstPtr(ptr32), 360200661, descr=<Calli 4 rri EF=4>) ++464: guard_no_exception(, descr=<Guard19>) [p48, i50, p30, p0] ++477: i51 = int_and(i50, -2147483648) ++485: i52 = int_is_true(i51) +guard_false(i52, descr=<Guard20>) [p48, i50, p30, p0] ++494: p53 = getinteriorfield_gc(p38, i50, descr=<InteriorFieldDescr <FieldP dictentry.value 4>>) ++501: i55 = instance_ptr_eq(ConstPtr(ptr54), p53) +guard_true(i55, descr=<Guard21>) [p48, p0] +debug_merge_point(0, 0, 'None') ++513: i56 = getfield_gc_pure(p48, descr=<FieldS pyhaskell.interpreter.primtype.Int.inst_value 8>) ++516: i58 = int_sub(i56, 1) +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') ++519: i59 = int_eq(0, i58) +guard_false(i59, descr=<Guard22>) [i58, p48, p0] +debug_merge_point(0, 0, '<PrimFunction object at 0x83f3f6c> 1 <Function object at 0xb710b3b0> 1 <Function object at 0xb710b3c0> <PrimFunction object at 0x83f3f3c> 1 dsdr3 <Function object at 0xb710b210> 1') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, '_') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, 'None') +debug_merge_point(0, 0, '<Function object at 0xb710b3d0> dsdr3 dsdr3') +debug_merge_point(0, 0, '<Function object at 0xb710b120> ds1dr4 dsdr3 ds1dr4') +p61 = new_with_vtable(138371700) +p63 = new_with_vtable(138373024) +p65 = new_with_vtable(138371936) ++606: setfield_gc(p63, ConstPtr(ptr66), descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) +p68 = new_with_vtable(138373024) ++632: setfield_gc(p65, ConstPtr(ptr69), descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) +p71 = new_with_vtable(138371936) ++658: setfield_gc(p68, ConstPtr(ptr17), descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) ++665: setfield_gc(p71, ConstPtr(ptr72), descr=<FieldP pyhaskell.interpreter.haskell.Application1.inst_arg0 12>) ++672: setfield_gc(p68, p71, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg2 20>) ++675: setfield_gc(p68, p48, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg1 16>) ++678: setfield_gc(p68, ConstPtr(ptr54), descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg0 12>) +p73 = new_with_vtable(138371648) ++701: setfield_gc(p61, p0, descr=<FieldP pyhaskell.interpreter.haskell.StackElement.inst_next 8>) ++716: setfield_gc(p61, 2, descr=<FieldS pyhaskell.interpreter.haskell.CopyStackElement.inst_index 16>) ++723: setfield_gc(p71, ConstPtr(ptr25), descr=<FieldP pyhaskell.interpreter.haskell.Application.inst_function 8>) ++730: setfield_gc(p65, p68, descr=<FieldP pyhaskell.interpreter.haskell.Application1.inst_arg0 12>) ++733: setfield_gc(p63, p65, descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg2 20>) ++736: setfield_gc(p63, ConstPtr(ptr75), descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg1 16>) ++743: setfield_gc(p63, ConstPtr(ptr54), descr=<FieldP pyhaskell.interpreter.haskell.Application3.inst_arg0 12>) ++750: setfield_gc(p61, p63, descr=<FieldP pyhaskell.interpreter.haskell.CopyStackElement.inst_application 12>) ++753: setfield_gc(p73, i58, descr=<FieldS pyhaskell.interpreter.primtype.Int.inst_value 8>) ++762: jump(p61, p73, p30, p38, descr=TargetToken(-1223434176)) ++775: --end of the loop-- +[2f1dd6da3b99] jit-log-opt-loop} diff --git a/tests/examplefiles/unix-io.lid b/tests/examplefiles/unix-io.lid new file mode 100644 index 00000000..617fcaa4 --- /dev/null +++ b/tests/examplefiles/unix-io.lid @@ -0,0 +1,37 @@ +Library: io +Synopsis: A portable IO library +Author: Gail Zacharias +Files: library + streams/defs + streams/stream + streams/sequence-stream + streams/native-buffer + streams/buffer + streams/typed-stream + streams/external-stream + streams/buffered-stream + streams/convenience + streams/wrapper-stream + streams/cleanup-streams + streams/native-speed + streams/async-writes + streams/file-stream + streams/multi-buffered-streams + pprint + print + print-double-integer-kludge + format + buffered-format + format-condition + unix-file-accessor + unix-standard-io + unix-interface + format-out +C-Source-Files: unix-portability.c +Major-Version: 2 +Minor-Version: 1 +Target-Type: dll +Copyright: Original Code is Copyright (c) 1995-2004 Functional Objects, Inc. + All rights reserved. +License: See License.txt in this distribution for details. +Warranty: Distributed WITHOUT WARRANTY OF ANY KIND diff --git a/tests/old_run.py b/tests/old_run.py index 7929d743..4f7cef16 100644 --- a/tests/old_run.py +++ b/tests/old_run.py @@ -8,7 +8,7 @@ python run.py [testfile ...] - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/run.py b/tests/run.py index ef92fe09..18a1d824 100644 --- a/tests/run.py +++ b/tests/run.py @@ -8,7 +8,7 @@ python run.py [testfile ...] - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -25,7 +25,8 @@ if sys.version_info >= (3,): os.chdir(newroot) else: # only find tests in this directory - os.chdir(os.path.dirname(__file__)) + if os.path.dirname(__file__): + os.chdir(os.path.dirname(__file__)) try: diff --git a/tests/support/tags b/tests/support/tags new file mode 100644 index 00000000..193779f6 --- /dev/null +++ b/tests/support/tags @@ -0,0 +1,36 @@ +!_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to lines/ +!_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/ +!_TAG_PROGRAM_AUTHOR Darren Hiebert /dhiebert@users.sourceforge.net/ +!_TAG_PROGRAM_NAME Exuberant Ctags // +!_TAG_PROGRAM_URL http://ctags.sourceforge.net /official site/ +!_TAG_PROGRAM_VERSION 5.8 // +HtmlFormatter test_html_formatter.py 19;" i +HtmlFormatterTest test_html_formatter.py 34;" c +NullFormatter test_html_formatter.py 19;" i +PythonLexer test_html_formatter.py 18;" i +StringIO test_html_formatter.py 13;" i +dirname test_html_formatter.py 16;" i +escape_html test_html_formatter.py 20;" i +fp test_html_formatter.py 27;" v +inspect test_html_formatter.py 15;" i +isfile test_html_formatter.py 16;" i +join test_html_formatter.py 16;" i +os test_html_formatter.py 10;" i +re test_html_formatter.py 11;" i +subprocess test_html_formatter.py 125;" i +support test_html_formatter.py 23;" i +tempfile test_html_formatter.py 14;" i +test_all_options test_html_formatter.py 72;" m class:HtmlFormatterTest +test_correct_output test_html_formatter.py 35;" m class:HtmlFormatterTest +test_ctags test_html_formatter.py 165;" m class:HtmlFormatterTest +test_external_css test_html_formatter.py 48;" m class:HtmlFormatterTest +test_get_style_defs test_html_formatter.py 141;" m class:HtmlFormatterTest +test_lineanchors test_html_formatter.py 98;" m class:HtmlFormatterTest +test_lineanchors_with_startnum test_html_formatter.py 106;" m class:HtmlFormatterTest +test_linenos test_html_formatter.py 82;" m class:HtmlFormatterTest +test_linenos_with_startnum test_html_formatter.py 90;" m class:HtmlFormatterTest +test_unicode_options test_html_formatter.py 155;" m class:HtmlFormatterTest +test_valid_output test_html_formatter.py 114;" m class:HtmlFormatterTest +tokensource test_html_formatter.py 29;" v +uni_open test_html_formatter.py 21;" i +unittest test_html_formatter.py 12;" i diff --git a/tests/test_basic_api.py b/tests/test_basic_api.py index bc4eb771..00dc26f0 100644 --- a/tests/test_basic_api.py +++ b/tests/test_basic_api.py @@ -3,7 +3,7 @@ Pygments basic API tests ~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -93,7 +93,8 @@ def test_lexer_options(): 'PythonConsoleLexer', 'RConsoleLexer', 'RubyConsoleLexer', 'SqliteConsoleLexer', 'MatlabSessionLexer', 'ErlangShellLexer', 'BashSessionLexer', 'LiterateHaskellLexer', 'PostgresConsoleLexer', - 'ElixirConsoleLexer', 'JuliaConsoleLexer'): + 'ElixirConsoleLexer', 'JuliaConsoleLexer', 'RobotFrameworkLexer', + 'DylanConsoleLexer', 'ShellSessionLexer'): inst = cls(ensurenl=False) ensure(inst.get_tokens('a\nb'), 'a\nb') inst = cls(ensurenl=False, stripall=True) diff --git a/tests/test_clexer.py b/tests/test_clexer.py index 08fb42cf..8b37bf57 100644 --- a/tests/test_clexer.py +++ b/tests/test_clexer.py @@ -3,7 +3,7 @@ Basic CLexer Test ~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_cmdline.py b/tests/test_cmdline.py index 56036183..5ad815c0 100644 --- a/tests/test_cmdline.py +++ b/tests/test_cmdline.py @@ -3,7 +3,7 @@ Command line test ~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_examplefiles.py b/tests/test_examplefiles.py index 41acf4ef..d785cf3b 100644 --- a/tests/test_examplefiles.py +++ b/tests/test_examplefiles.py @@ -3,7 +3,7 @@ Pygments tests with example files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -58,6 +58,8 @@ def check_lexer(lx, absfn, outfn): text = text.strip(b('\n')) + b('\n') try: text = text.decode('utf-8') + if text.startswith(u'\ufeff'): + text = text[len(u'\ufeff'):] except UnicodeError: text = text.decode('latin1') ntext = [] diff --git a/tests/test_html_formatter.py b/tests/test_html_formatter.py index 284a6c75..f7e7a542 100644 --- a/tests/test_html_formatter.py +++ b/tests/test_html_formatter.py @@ -3,7 +3,7 @@ Pygments HTML formatter tests ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -160,3 +160,19 @@ class HtmlFormatterTest(unittest.TestCase): tfile = os.fdopen(handle, 'w+b') fmt.format(tokensource, tfile) tfile.close() + + def test_ctags(self): + try: + import ctags + except ImportError: + # we can't check without the ctags module, but at least check the exception + self.assertRaises(RuntimeError, HtmlFormatter, tagsfile='support/tags') + else: + # this tagfile says that test_ctags() is on line 165, even if it isn't + # anymore in the actual source + fmt = HtmlFormatter(tagsfile='support/tags', lineanchors='L', + tagurlformat='%(fname)s%(fext)s') + outfile = StringIO.StringIO() + fmt.format(tokensource, outfile) + self.assertTrue('<a href="test_html_formatter.py#L-165">test_ctags</a>' + in outfile.getvalue()) diff --git a/tests/test_latex_formatter.py b/tests/test_latex_formatter.py index 8412ec41..06a74c3d 100644 --- a/tests/test_latex_formatter.py +++ b/tests/test_latex_formatter.py @@ -3,7 +3,7 @@ Pygments LaTeX formatter tests ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_perllexer.py b/tests/test_perllexer.py index b9c3cb74..315b20e3 100644 --- a/tests/test_perllexer.py +++ b/tests/test_perllexer.py @@ -3,7 +3,7 @@ Pygments regex lexer tests ~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_regexlexer.py b/tests/test_regexlexer.py index b1392d3a..28d9689b 100644 --- a/tests/test_regexlexer.py +++ b/tests/test_regexlexer.py @@ -3,7 +3,7 @@ Pygments regex lexer tests ~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_token.py b/tests/test_token.py index a9d1edeb..6a5b00b7 100644 --- a/tests/test_token.py +++ b/tests/test_token.py @@ -3,7 +3,7 @@ Test suite for the token module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_using_api.py b/tests/test_using_api.py index 83d3f18e..bb89d1e2 100644 --- a/tests/test_using_api.py +++ b/tests/test_using_api.py @@ -3,7 +3,7 @@ Pygments tests for using() ~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ diff --git a/tests/test_util.py b/tests/test_util.py index be1662f9..dbbc66ce 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -3,7 +3,7 @@ Test suite for the util module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :copyright: Copyright 2006-2012 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -98,8 +98,10 @@ class UtilTest(unittest.TestCase): self.assertTrue(util.shebang_matches('#!C:\\Python2.4\\Python.exe', r'python(2\.\d)?')) - self.assertFalse(util.shebang_matches('#!/usr/bin/python-ruby', r'python(2\.\d)?')) - self.assertFalse(util.shebang_matches('#!/usr/bin/python/ruby', r'python(2\.\d)?')) + self.assertFalse(util.shebang_matches('#!/usr/bin/python-ruby', + r'python(2\.\d)?')) + self.assertFalse(util.shebang_matches('#!/usr/bin/python/ruby', + r'python(2\.\d)?')) self.assertFalse(util.shebang_matches('#!', r'python')) def test_doctype_matches(self): @@ -131,4 +133,3 @@ class UtilTest(unittest.TestCase): m = r.match(first_non_bmp * 2) self.assertTrue(m) self.assertEquals(m.end(), len(first_non_bmp) * 2) - |