diff options
| author | Georg Brandl <georg@python.org> | 2013-05-19 10:13:05 +0200 |
|---|---|---|
| committer | Georg Brandl <georg@python.org> | 2013-05-19 10:13:05 +0200 |
| commit | 938dae70613eebfc98823b0c4369e4deb13f104e (patch) | |
| tree | 35e257b162cf34e1798b13f968b0de30902a0180 | |
| parent | e98108876f81e9aead17a82a0898d45e395f717d (diff) | |
| parent | 93947af62a2629af18bb569bc11fe18be29fb671 (diff) | |
| download | pygments-938dae70613eebfc98823b0c4369e4deb13f104e.tar.gz | |
merge with bitbucket.org/christian/pygments-main, pull request #171
42 files changed, 7225 insertions, 2264 deletions
@@ -6,8 +6,9 @@ Major developers are Tim Hatch <tim@timhatch.com> and Armin Ronacher Other contributors, listed alphabetically, are: * Sam Aaron -- Ioke lexer -* Kumar Appaiah -- Debian control lexer * Ali Afshar -- image formatter +* Thomas Aglassinger -- Rexx lexer +* Kumar Appaiah -- Debian control lexer * Andreas Amann -- AppleScript lexer * Timothy Armstrong -- Dart lexer fixes * Jeffrey Arnold -- R/S, Rd, BUGS, Jags, and Stan lexers @@ -29,20 +30,25 @@ Other contributors, listed alphabetically, are: * Christian Jann -- ShellSession lexer * Christopher Creutzig -- MuPAD lexer * Pete Curry -- bugfixes -* Owen Durni -- haXe lexer +* Bryan Davis -- EBNF lexer +* Owen Durni -- Haxe lexer * Nick Efford -- Python 3 lexer * Sven Efftinge -- Xtend lexer * Artem Egorkine -- terminal256 formatter * James H. Fisher -- PostScript lexer +* William S. Fulton -- SWIG lexer * Carlos Galdino -- Elixir and Elixir Console lexers * Michael Galloy -- IDL lexer * Naveen Garg -- Autohotkey lexer * Laurent Gautier -- R/S lexer * Alex Gaynor -- PyPy log lexer +* Richard Gerkin -- Igor Pro lexer * Alain Gilbert -- TypeScript lexer +* Alex Gilding -- BlitzBasic lexer * Bertrand Goetzmann -- Groovy lexer * Krzysiek Goj -- Scala lexer * Matt Good -- Genshi, Cheetah lexers +* Michał Górny -- vim modeline support * Patrick Gotthardt -- PHP namespaces support * Olivier Guibe -- Asymptote lexer * Jordi Gutiérrez Hermoso -- Octave lexer @@ -53,6 +59,7 @@ Other contributors, listed alphabetically, are: * Greg Hendershott -- Racket lexer * David Hess, Fish Software, Inc. -- Objective-J lexer * Varun Hiremath -- Debian control lexer +* Rob Hoelz -- Perl 6 lexer * Doug Hogan -- Mscgen lexer * Ben Hollis -- Mason lexer * Dustin Howett -- Logos lexer @@ -64,6 +71,7 @@ Other contributors, listed alphabetically, are: * Igor Kalnitsky -- vhdl lexer * Pekka Klärck -- Robot Framework lexer * Eric Knibbe -- Lasso lexer +* Stepan Koltsov -- Clay lexer * Adam Koprowski -- Opa lexer * Benjamin Kowarsch -- Modula-2 lexer * Alexander Kriegisch -- Kconfig and AspectJ lexers @@ -97,6 +105,7 @@ Other contributors, listed alphabetically, are: * Mike Nolta -- Julia lexer * Jonas Obrist -- BBCode lexer * David Oliva -- Rebol lexer +* Pat Pannuto -- nesC lexer * Jon Parise -- Protocol buffers lexer * Ronny Pfannschmidt -- BBCode lexer * Benjamin Peterson -- Test suite refactoring @@ -6,6 +6,49 @@ Issue numbers refer to the tracker at pull request numbers to the requests at <http://bitbucket.org/birkenfeld/pygments-main/pull-requests/merged>. +Version 1.7 +----------- +(under development) + +- Lexers added: + + * Clay (PR#184) + * Perl 6 (PR#181) + * Swig (PR#168) + * nesC (PR#166) + * BlitzBasic (PR#197) + * EBNF (PR#193) + * Igor Pro (PR#172) + * Rexx (PR#199) + +- Pygments will now recognize "vim" modelines when guessing the lexer for + a file based on content (PR#118). + +- Python 3 lexer: add new exceptions from PEP 3151. + +- Opa lexer: add new keywords (PR#170). + +- Julia lexer: add keywords and underscore-separated number + literals (PR#176). + +- Lasso lexer: fix method highlighting, update builtins. Fix + guessing so that plain XML isn't always taken as Lasso (PR#163). + +- Objective C/C++ lexers: allow "@" prefixing any expression (#871). + +- Ruby lexer: fix lexing of Name::Space tokens (#860). + +- Stan lexer: update for version 1.3.0 of the language (PR#162). + +- JavaScript lexer: add the "yield" keyword (PR#196). + +- HTTP lexer: support for PATCH method (PR#190). + +- Koka lexer: update to newest language spec (PR#201). + +- Haxe lexer: rewrite and support for Haxe 3 (PR#174). + + Version 1.6 ----------- (released Feb 3, 2013) @@ -259,7 +302,7 @@ Version 1.3 * Ada * Coldfusion * Modula-2 - * haXe + * Haxe * R console * Objective-J * Haml and Sass @@ -318,7 +361,7 @@ Version 1.2 * CMake * Ooc * Coldfusion - * haXe + * Haxe * R console - Added options for rendering LaTeX in source code comments in the diff --git a/docs/src/lexerdevelopment.txt b/docs/src/lexerdevelopment.txt index 6ffc4b72..730a08b2 100644 --- a/docs/src/lexerdevelopment.txt +++ b/docs/src/lexerdevelopment.txt @@ -83,6 +83,58 @@ If no rule matches at the current position, the current char is emitted as an 1. +Adding and testing a new lexer +============================== + +To make pygments aware of your new lexer, you have to perform the following +steps: + +First, change to the current directory containing the pygments source code: + +.. sourcecode:: console + + $ cd .../pygments-main + +Next, make sure the lexer is known from outside of the module. All modules in +the ``pygments.lexers`` specify ``__all__``. For example, ``other.py`` sets: + +.. sourcecode:: python + + __all__ = ['BrainfuckLexer', 'BefungeLexer', ...] + +Simply add the name of your lexer class to this list. + +Finally the lexer can be made publically known by rebuilding the lexer +mapping: + +.. sourcecode:: console + + $ make mapfiles + +To test the new lexer, store an example file with the proper extension in +``tests/examplefiles``. For example, to test your ``DiffLexer``, add a +``tests/examplefiles/example.diff`` containing a sample diff output. + +Now you can use pygmentize to render your example to HTML: + +.. sourcecode:: console + + $ ./pygmentize -O full -f html -o /tmp/example.html tests/examplefiles/example.diff + +Note that this explicitely calls the ``pygmentize`` in the current directory +by preceding it with ``./``. This ensures your modifications are used. +Otherwise a possibly already installed, unmodified version without your new +lexer would have been called from the system search path (``$PATH``). + +To view the result, open ``/tmp/example.html`` in your browser. + +Once the example renders as expected, you should run the complete test suite: + +.. sourcecode:: console + + $ make test + + Regex Flags =========== diff --git a/external/lasso-builtins-generator-9.lasso b/external/lasso-builtins-generator-9.lasso index bea8b2ab..6a619106 100755 --- a/external/lasso-builtins-generator-9.lasso +++ b/external/lasso-builtins-generator-9.lasso @@ -5,13 +5,19 @@ This is the shell script that was used to extract Lasso 9's built-in keywords and generate most of the _lassobuiltins.py file. When run, it creates a file - named "lassobuiltins-9.py" containing the types, traits, and methods of the - currently-installed version of Lasso 9. + named "lassobuiltins-9.py" containing the types, traits, methods, and members + of the currently-installed version of Lasso 9. - A partial list of keywords in Lasso 8 can be generated with this code: + A list of tags in Lasso 8 can be generated with this code: <?LassoScript - local('l8tags' = list); + local('l8tags' = list, + 'l8libs' = array('Cache','ChartFX','Client','Database','File','HTTP', + 'iCal','Lasso','Link','List','PDF','Response','Stock','String', + 'Thread','Valid','WAP','XML')); + iterate(#l8libs, local('library')); + local('result' = namespace_load(#library)); + /iterate; iterate(tags_list, local('i')); #l8tags->insert(string_removeleading(#i, -pattern='_global_')); /iterate; @@ -30,9 +36,12 @@ local(f) = file("lassobuiltins-9.py") #f->writeString('# -*- coding: utf-8 -*- """ pygments.lexers._lassobuiltins - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Built-in Lasso types, traits, methods, and members. - Built-in Lasso types, traits, and methods. + :copyright: Copyright 2006-'+date->year+' by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. """ ') @@ -42,16 +51,16 @@ lcapi_loadModules // Load all of the libraries from builtins and lassoserver // This forces all possible available types and methods to be registered local(srcs = - tie( - dir(sys_masterHomePath + 'LassoLibraries/builtins/')->eachFilePath, - dir(sys_masterHomePath + 'LassoLibraries/lassoserver/')->eachFilePath - ) + tie( + dir(sys_masterHomePath + 'LassoLibraries/builtins/')->eachFilePath, + dir(sys_masterHomePath + 'LassoLibraries/lassoserver/')->eachFilePath + ) ) with topLevelDir in #srcs -where !#topLevelDir->lastComponent->beginsWith('.') +where not #topLevelDir->lastComponent->beginsWith('.') do protect => { - handle_error => { + handle_error => { stdoutnl('Unable to load: ' + #topLevelDir + ' ' + error_msg) } library_thread_loader->loadLibrary(#topLevelDir) @@ -61,60 +70,74 @@ do protect => { local( typesList = list(), traitsList = list(), - methodsList = list() + unboundMethodsList = list(), + memberMethodsList = list() ) -// unbound methods -with method in sys_listUnboundMethods -where !#method->methodName->asString->endsWith('=') -where #method->methodName->asString->isalpha(1) -where #methodsList !>> #method->methodName->asString -do #methodsList->insert(#method->methodName->asString) +// types +with type in sys_listTypes +where #typesList !>> #type +do { + #typesList->insert(#type) + with method in #type->getType->listMethods + let name = #method->methodName + where not #name->asString->endsWith('=') // skip setter methods + where #name->asString->isAlpha(1) // skip unpublished methods + where #memberMethodsList !>> #name + do #memberMethodsList->insert(#name) +} // traits with trait in sys_listTraits -where !#trait->asString->beginsWith('$') -where #traitsList !>> #trait->asString +where not #trait->asString->beginsWith('$') // skip combined traits +where #traitsList !>> #trait do { - #traitsList->insert(#trait->asString) - with tmethod in tie(#trait->getType->provides, #trait->getType->requires) - where !#tmethod->methodName->asString->endsWith('=') - where #tmethod->methodName->asString->isalpha(1) - where #methodsList !>> #tmethod->methodName->asString - do #methodsList->insert(#tmethod->methodName->asString) + #traitsList->insert(#trait) + with method in tie(#trait->getType->provides, #trait->getType->requires) + let name = #method->methodName + where not #name->asString->endsWith('=') // skip setter methods + where #name->asString->isAlpha(1) // skip unpublished methods + where #memberMethodsList !>> #name + do #memberMethodsList->insert(#name) } -// types -with type in sys_listTypes -where #typesList !>> #type->asString -do { - #typesList->insert(#type->asString) - with tmethod in #type->getType->listMethods - where !#tmethod->methodName->asString->endsWith('=') - where #tmethod->methodName->asString->isalpha(1) - where #methodsList !>> #tmethod->methodName->asString - do #methodsList->insert(#tmethod->methodName->asString) -} +// unbound methods +with method in sys_listUnboundMethods +let name = #method->methodName +where not #name->asString->endsWith('=') // skip setter methods +where #name->asString->isAlpha(1) // skip unpublished methods +where #typesList !>> #name +where #traitsList !>> #name +where #unboundMethodsList !>> #name +do #unboundMethodsList->insert(#name) #f->writeString("BUILTINS = { 'Types': [ ") with t in #typesList -do #f->writeString(" '"+string_lowercase(#t)+"',\n") +do !#t->asString->endsWith('$') ? #f->writeString(" '"+string_lowercase(#t->asString)+"',\n") #f->writeString(" ], 'Traits': [ ") with t in #traitsList -do #f->writeString(" '"+string_lowercase(#t)+"',\n") +do #f->writeString(" '"+string_lowercase(#t->asString)+"',\n") #f->writeString(" ], - 'Methods': [ + 'Unbound Methods': [ ") -with t in #methodsList -do #f->writeString(" '"+string_lowercase(#t)+"',\n") +with t in #unboundMethodsList +do #f->writeString(" '"+string_lowercase(#t->asString)+"',\n") -#f->writeString(" ], +#f->writeString(" ] +} +MEMBERS = { + 'Member Methods': [ +") +with t in #memberMethodsList +do #f->writeString(" '"+string_lowercase(#t->asString)+"',\n") + +#f->writeString(" ] } ") diff --git a/pygmentize b/pygmentize index e2379199..8b3b2067 100755..100644 --- a/pygmentize +++ b/pygmentize @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python2 import sys, pygments.cmdline try: diff --git a/pygments/lexers/__init__.py b/pygments/lexers/__init__.py index 9af6ce68..dbfe4351 100644 --- a/pygments/lexers/__init__.py +++ b/pygments/lexers/__init__.py @@ -15,6 +15,7 @@ import fnmatch from os.path import basename from pygments.lexers._mapping import LEXERS +from pygments.modeline import get_filetype_from_buffer from pygments.plugin import find_plugin_lexers from pygments.util import ClassNotFound, bytes @@ -197,6 +198,16 @@ def guess_lexer(_text, **options): """ Guess a lexer by strong distinctions in the text (eg, shebang). """ + + # try to get a vim modeline first + ft = get_filetype_from_buffer(_text) + + if ft is not None: + try: + return get_lexer_by_name(ft, **options) + except ClassNotFound: + pass + best_lexer = [0.0, None] for lexer in _iter_lexerclasses(): rv = lexer.analyse_text(_text) diff --git a/pygments/lexers/_lassobuiltins.py b/pygments/lexers/_lassobuiltins.py index 08b65f37..f3e5147e 100644 --- a/pygments/lexers/_lassobuiltins.py +++ b/pygments/lexers/_lassobuiltins.py @@ -3,7 +3,7 @@ pygments.lexers._lassobuiltins ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Built-in Lasso types, traits, and methods. + Built-in Lasso types, traits, methods, and members. :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -62,7 +62,6 @@ BUILTINS = { 'regexp', 'zip_impl', 'zip_file_impl', - 'library_thread_loader_thread$', 'library_thread_loader', 'generateforeachunkeyed', 'generateforeachkeyed', @@ -98,6 +97,7 @@ BUILTINS = { 'map_node', 'map', 'file', + 'date', 'dir', 'magick_image', 'ldap', @@ -113,7 +113,6 @@ BUILTINS = { 'sqlite_table', 'sqlite_column', 'curl', - 'date', 'debugging_stack', 'dbgp_server', 'dbgp_packet', @@ -179,9 +178,7 @@ BUILTINS = { 'fcgi_record', 'web_request_impl', 'fcgi_request', - 'include_cache_thread$', 'include_cache', - 'atbegin_thread$', 'atbegin', 'fastcgi_each_fcgi_param', 'fastcgi_server', @@ -242,7 +239,6 @@ BUILTINS = { 'lassoapp_livesrc_fileresource', 'lassoapp_livesrc_appsource', 'lassoapp_long_expiring_bytes', - 'lassoapp_zip_file_server_thread$', 'lassoapp_zip_file_server', 'lassoapp_zipsrc_fileresource', 'lassoapp_zipsrc_appsource', @@ -258,7 +254,6 @@ BUILTINS = { 'sqlite_session_driver_impl', 'mysql_session_driver_impl', 'odbc_session_driver_impl', - 'session_delete_expired_thread_thread$', 'session_delete_expired_thread', 'email_smtp', 'client_address', @@ -348,15 +343,8 @@ BUILTINS = { 'web_node_content_css_specialized', 'web_node_content_js_specialized' ], - 'Methods': [ + 'Unbound Methods': [ 'fail_now', - 'staticarray', - 'integer', - 'decimal', - 'string', - 'bytes', - 'keyword', - 'signature', 'register', 'register_thread', 'escape_tag', @@ -372,8 +360,6 @@ BUILTINS = { 'failure_clear', 'var_keys', 'var_values', - 'null', - 'trait', 'staticarray_join', 'suspend', 'main_thread_only', @@ -381,7 +367,6 @@ BUILTINS = { 'capture_nearestloopcount', 'capture_nearestloopcontinue', 'capture_nearestloopabort', - 'pair', 'io_file_o_rdonly', 'io_file_o_wronly', 'io_file_o_rdwr', @@ -549,11 +534,9 @@ BUILTINS = { 'io_file_f_ulock', 'io_file_f_tlock', 'io_file_f_test', - 'dirdesc', 'io_file_stdin', 'io_file_stdout', 'io_file_stderr', - 'filedesc', 'uchar_alphabetic', 'uchar_ascii_hex_digit', 'uchar_bidi_control', @@ -696,7 +679,6 @@ BUILTINS = { 'u_nt_decimal', 'u_nt_digit', 'u_nt_numeric', - 'locale', 'locale_english', 'locale_french', 'locale_german', @@ -724,7 +706,6 @@ BUILTINS = { 'locale_isolanguages', 'locale_availablelocales', 'ucal_listtimezones', - 'ucal', 'ucal_era', 'ucal_year', 'ucal_month', @@ -750,7 +731,6 @@ BUILTINS = { 'ucal_lenient', 'ucal_firstdayofweek', 'ucal_daysinfirstweek', - 'xml_domimplementation', 'sys_sigalrm', 'sys_sighup', 'sys_sigkill', @@ -869,7 +849,6 @@ BUILTINS = { 'sys_is_full_path', 'lcapi_loadmodule', 'lcapi_listdatasources', - 'dsinfo', 'encrypt_blowfish', 'decrypt_blowfish', 'cipher_digest', @@ -887,11 +866,7 @@ BUILTINS = { 'cipher_encrypt_private', 'cipher_encrypt_public', 'cipher_generate_key', - 'xmlstream', - 'sourcefile', - 'tag', 'tag_exists', - 'mime_reader', 'curl_easy_init', 'curl_easy_duphandle', 'curl_easy_cleanup', @@ -1132,9 +1107,6 @@ BUILTINS = { 'curle_ssl_engine_initfailed', 'curle_login_denied', 'curlmsg_done', - 'regexp', - 'array', - 'boolean', 'zip_open', 'zip_name_locate', 'zip_fopen', @@ -1172,7 +1144,6 @@ BUILTINS = { 'evdns_resolve_ipv6', 'evdns_resolve_reverse', 'evdns_resolve_reverse_ipv6', - 'library_thread_loader', 'stdout', 'stdoutnl', 'fail', @@ -1205,85 +1176,18 @@ BUILTINS = { 'error_code_noerror', 'abort', 'protect', - 'trait_asstring', - 'any', - 'trait_generator', - 'trait_decompose_assignment', - 'trait_foreach', - 'trait_generatorcentric', 'generateforeach', - 'generateforeachunkeyed', - 'generateforeachkeyed', - 'trait_foreachtextelement', - 'trait_finite', - 'trait_finiteforeach', - 'trait_keyed', - 'trait_keyedfinite', - 'trait_keyedforeach', - 'trait_frontended', - 'trait_backended', - 'trait_doubleended', - 'trait_positionallykeyed', - 'trait_expandable', - 'trait_frontexpandable', - 'trait_backexpandable', - 'trait_contractible', - 'trait_frontcontractible', - 'trait_backcontractible', - 'trait_fullymutable', - 'trait_keyedmutable', - 'trait_endedfullymutable', - 'trait_setoperations', - 'trait_searchable', - 'trait_positionallysearchable', - 'trait_pathcomponents', - 'trait_readbytes', - 'trait_writebytes', - 'trait_setencoding', - 'trait_readstring', - 'trait_writestring', - 'trait_hashable', - 'eacher', - 'trait_each_sub', - 'trait_stack', - 'trait_list', - 'trait_array', - 'trait_map', - 'trait_close', - 'trait_file', - 'trait_scalar', 'method_name', - 'trait_queriablelambda', - 'trait_queriable', - 'queriable_asstring', - 'queriable_where', 'queriable_do', 'queriable_sum', 'queriable_average', 'queriable_min', 'queriable_max', - 'queriable_select', - 'queriable_selectmany', - 'queriable_groupby', - 'queriable_join', - 'queriable_groupjoin', - 'queriable_orderby', - 'queriable_orderbydescending', - 'queriable_thenby', - 'queriable_thenbydescending', - 'queriable_skip', - 'queriable_take', - 'queriable_grouping', 'queriable_internal_combinebindings', 'queriable_defaultcompare', 'queriable_reversecompare', 'queriable_qsort', - 'generateseries', 'timer', - 'tie', - 'pairup', - 'delve', - 'repeat', 'thread_var_push', 'thread_var_pop', 'thread_var_get', @@ -1301,24 +1205,10 @@ BUILTINS = { 'loop', 'sys_while', 'sys_iterate', - 'pair_compare', - 'serialization_object_identity_compare', - 'serialization_element', - 'trait_serializable', - 'serialization_writer_standin', - 'serialization_writer_ref', - 'serialization_writer', - 'serialization_reader', 'string_validcharset', 'eol', 'encoding_utf8', 'encoding_iso88591', - 'trait_treenode', - 'tree_nullnode', - 'tree_node', - 'tree_base', - 'map_node', - 'map', 'integer_random', 'integer_bitor', 'millis', @@ -1395,7 +1285,6 @@ BUILTINS = { 'file_modechar', 'file_forceroot', 'file_tempfile', - 'file', 'file_stdin', 'file_stdout', 'file_stderr', @@ -1446,14 +1335,10 @@ BUILTINS = { 'sys_usercapimodulepath', 'sys_appspath', 'sys_userstartuppath', - 'dir', - 'magick_image', - 'ldap', 'ldap_scope_base', 'ldap_scope_onelevel', 'ldap_scope_subtree', 'mysqlds', - 'os_process', 'odbc', 'sqliteconnector', 'sqlite_createdb', @@ -1472,7 +1357,6 @@ BUILTINS = { 'database_initialize', 'database_util_cleanpath', 'database_adddefaultsqlitehost', - 'database_registry', 'sqlite_ok', 'sqlite_error', 'sqlite_internal', @@ -1507,18 +1391,11 @@ BUILTINS = { 'sqlite_blob', 'sqlite_null', 'sqlite_text', - 'sqlite3', - 'sqlite_db', - 'sqlite_results', - 'sqlite_currentrow', - 'sqlite_table', - 'sqlite_column', 'bom_utf16be', 'bom_utf16le', 'bom_utf32be', 'bom_utf32le', 'bom_utf8', - 'curl', 'include_url', 'ftp_getdata', 'ftp_getfile', @@ -1526,7 +1403,6 @@ BUILTINS = { 'ftp_putdata', 'ftp_putfile', 'ftp_deletefile', - 'date', 'debugging_step_in', 'debugging_get_stack', 'debugging_get_context', @@ -1544,11 +1420,7 @@ BUILTINS = { 'debugging_context_locals', 'debugging_context_vars', 'debugging_context_self', - 'debugging_stack', 'dbgp_stop_stack_name', - 'dbgp_server', - 'dbgp_packet', - 'duration', 'encrypt_md5', 'inline_columninfo_pos', 'inline_resultrows_pos', @@ -1561,7 +1433,6 @@ BUILTINS = { 'inline_namedget', 'inline_namedput', 'inline', - 'inline_type', 'resultset_count', 'resultset', 'resultsets', @@ -1601,32 +1472,14 @@ BUILTINS = { 'rows_array', 'records_array', 'records_map', - 'trait_json_serialize', 'json_serialize', 'json_consume_string', 'json_consume_token', 'json_consume_array', 'json_consume_object', 'json_deserialize', - 'json_literal', - 'json_object', 'json_rpccall', - 'list_node', - 'list', - 'jchar', - 'jchararray', - 'jbyte', - 'jbytearray', - 'jfloat', - 'jint', - 'jshort', 'ljapi_initialize', - 'formattingbase', - 'currency', - 'scientific', - 'percent', - 'dateandtime', - 'timeonly', 'locale_format_style_full', 'locale_format_style_long', 'locale_format_style_medium', @@ -1644,12 +1497,6 @@ BUILTINS = { 'net_waitread', 'net_waittimeout', 'net_waitwrite', - 'trait_net', - 'net_tcp', - 'net_tcpssl', - 'net_named_pipe', - 'net_udppacket', - 'net_udp', 'admin_initialize', 'admin_getpref', 'admin_setpref', @@ -1658,29 +1505,9 @@ BUILTINS = { 'admin_lassoservicepath', 'pdf_package', 'pdf_rectangle', - 'pdf_typebase', - 'pdf_doc', - 'pdf_color', - 'pdf_barcode', - 'pdf_font', - 'pdf_image', - 'pdf_list', - 'pdf_read', - 'pdf_table', - 'pdf_text', - 'pdf_hyphenator', - 'pdf_chunk', - 'pdf_phrase', - 'pdf_paragraph', 'pdf_serve', - 'queue', 'random_seed', - 'set', - 'sys_process', - 'worker_pool', 'xml', - 'trait_xml_elementcompat', - 'trait_xml_nodecompat', 'xml_transform', 'zip_create', 'zip_excl', @@ -1740,11 +1567,6 @@ BUILTINS = { 'zip_em_rc2', 'zip_em_rc4', 'zip_em_unknown', - 'zip_file', - 'zip', - 'cache_server_element', - 'cache_server', - 'dns_response', 'dns_lookup', 'dns_default', 'string_charfromname', @@ -1775,20 +1597,12 @@ BUILTINS = { 'string_uppercase', 'string_lowercase', 'document', - 'component_render_state', - 'component', - 'component_container', - 'document_base', - 'document_body', - 'document_header', - 'text_document', - 'data_document', 'email_attachment_mime_type', 'email_translatebreakstocrlf', 'email_findemails', 'email_fix_address', 'email_fix_address_list', - 'email_compose', + 'encode_qheader', 'email_send', 'email_queue', 'email_immediate', @@ -1797,9 +1611,6 @@ BUILTINS = { 'email_token', 'email_merge', 'email_batch', - 'encode_qheader', - 'email_pop', - 'email_parse', 'email_safeemail', 'email_extract', 'email_pop_priv_substring', @@ -1809,9 +1620,7 @@ BUILTINS = { 'email_digestresponse', 'encrypt_hmac', 'encrypt_crammd5', - 'email_queue_impl_base', 'email_fs_error_clean', - 'email_stage_impl_base', 'email_initialize', 'email_mxlookup', 'lasso_errorreporting', @@ -1840,26 +1649,17 @@ BUILTINS = { 'fcgi_max_reqs', 'fcgi_mpxs_conns', 'fcgi_read_timeout_seconds', - 'fcgi_record', 'fcgi_makeendrequestbody', 'fcgi_bodychunksize', 'fcgi_makestdoutbody', 'fcgi_readparam', - 'web_connection', - 'web_request_impl', 'web_request', - 'fcgi_request', 'include_cache_compare', - 'include_cache', - 'atbegin', 'fastcgi_initialize', 'fastcgi_handlecon', 'fastcgi_handlereq', - 'fastcgi_each_fcgi_param', 'fastcgi_createfcgirequest', - 'fastcgi_server', 'web_handlefcgirequest', - 'filemaker_datasource', 'filemakerds_initialize', 'filemakerds', 'value_listitem', @@ -1867,55 +1667,6 @@ BUILTINS = { 'selected', 'checked', 'value_list', - 'http_document', - 'http_document_header', - 'http_header_field', - 'html_document_head', - 'html_document_body', - 'raw_document_body', - 'bytes_document_body', - 'html_element_coreattrs', - 'html_element_i18nattrs', - 'html_element_eventsattrs', - 'html_attributed', - 'html_attr', - 'html_atomic_element', - 'html_container_element', - 'http_error', - 'html_script', - 'html_text', - 'html_raw', - 'html_binary', - 'html_json', - 'html_cdata', - 'html_eol', - 'html_div', - 'html_span', - 'html_br', - 'html_hr', - 'html_h1', - 'html_h2', - 'html_h3', - 'html_h4', - 'html_h5', - 'html_h6', - 'html_meta', - 'html_link', - 'html_object', - 'html_style', - 'html_base', - 'html_table', - 'html_tr', - 'html_td', - 'html_th', - 'html_img', - 'html_form', - 'html_fieldset', - 'html_legend', - 'html_input', - 'html_label', - 'html_option', - 'html_select', 'http_char_space', 'http_char_htab', 'http_char_cr', @@ -1923,39 +1674,21 @@ BUILTINS = { 'http_char_question', 'http_char_colon', 'http_read_timeout_secs', - 'http_server_web_connection', - 'http_server', - 'http_server_connection_handler', - 'image', + 'http_default_files', + 'http_server_apps_path', 'jdbc_initialize', 'lassoapp_settingsdb', - 'lassoapp_resource', 'lassoapp_format_mod_date', 'lassoapp_include_current', 'lassoapp_include', 'lassoapp_find_missing_file', - 'lassoapp_source', - 'lassoapp_capabilities', 'lassoapp_get_capabilities_name', 'lassoapp_exists', 'lassoapp_path_to_method_name', 'lassoapp_invoke_resource', - 'lassoapp_installer', 'lassoapp_initialize_db', 'lassoapp_initialize', - 'lassoapp_content_rep_halt', 'lassoapp_issourcefileextension', - 'lassoapp_dirsrc_fileresource', - 'lassoapp_dirsrc_appsource', - 'lassoapp_livesrc_fileresource', - 'lassoapp_livesrc_appsource', - 'lassoapp_long_expiring_bytes', - 'lassoapp_zip_file_server', - 'lassoapp_zipsrc_fileresource', - 'lassoapp_zipsrc_appsource', - 'lassoapp_compiledsrc_fileresource', - 'lassoapp_compiledsrc_appsource', - 'lassoapp_manualsrc_appsource', 'lassoapp_current_include', 'lassoapp_current_app', 'lassoapp_do_with_include', @@ -1983,6 +1716,7 @@ BUILTINS = { 'lassoapp_mime_type_svg', 'lassoapp_mime_type_ttf', 'lassoapp_mime_type_woff', + 'lassoapp_mime_type_swf', 'lassoapp_mime_get', 'log_level_critical', 'log_level_warning', @@ -2002,9 +1736,7 @@ BUILTINS = { 'log_deprecated', 'log_max_file_size', 'log_trim_file_size', - 'log_impl_base', 'log_initialize', - 'portal_impl', 'portal', 'security_database', 'security_table_groups', @@ -2012,8 +1744,6 @@ BUILTINS = { 'security_table_ug_map', 'security_default_realm', 'security_initialize', - 'security_registry', - 'session_driver', 'session_initialize', 'session_getdefaultdriver', 'session_setdefaultdriver', @@ -2025,23 +1755,14 @@ BUILTINS = { 'session_abort', 'session_result', 'session_deleteexpired', - 'memory_session_driver_impl_entry', - 'memory_session_driver_impl', - 'sqlite_session_driver_impl_entry', - 'sqlite_session_driver_impl', - 'mysql_session_driver_impl', 'odbc_session_driver_mssql', - 'odbc_session_driver_impl', 'session_decorate', - 'session_delete_expired_thread', - 'email_smtp', 'auth_admin', 'auth_check', 'auth_custom', 'auth_group', 'auth_prompt', 'auth_user', - 'client_address', 'client_addr', 'client_authorization', 'client_browser', @@ -2056,7 +1777,6 @@ BUILTINS = { 'client_getparam', 'client_headers', 'client_integertoip', - 'client_ip', 'client_iptointeger', 'client_password', 'client_postargs', @@ -2109,246 +1829,1237 @@ BUILTINS = { 'content_replaceheader', 'content_body', 'html_comment', - 'web_node_content_json_specialized', - 'web_node', - 'web_node_container', - 'web_node_content_representation', - 'web_node_content', - 'web_node_content_document', - 'web_node_postable', - 'web_node_base', 'web_node_forpath', 'web_nodes_requesthandler', 'web_nodes_normalizeextension', 'web_nodes_processcontentnode', - 'web_node_root', 'web_nodes_initialize', - 'web_node_content_representation_xhr_container', 'web_node_content_representation_xhr', - 'web_node_content_html_specialized', - 'web_node_content_representation_html_specialized', 'web_node_content_representation_html', - 'web_node_content_css_specialized', - 'web_node_content_representation_css_specialized', 'web_node_content_representation_css', - 'web_node_content_js_specialized', - 'web_node_content_representation_js_specialized', 'web_node_content_representation_js', - 'web_node_echo', 'web_response_nodesentry', - 'web_error_atend', - 'web_response_impl', 'web_response', 'web_router_database', - 'web_router_initialize', - 'web_router', - 'asstring', - 'isnota', - 'isallof', - 'isanyof', + 'web_router_initialize' + ], + 'Lasso 8 Tags': [ + '__char', + '__sync_timestamp__', + '_admin_addgroup', + '_admin_adduser', + '_admin_defaultconnector', + '_admin_defaultconnectornames', + '_admin_defaultdatabase', + '_admin_defaultfield', + '_admin_defaultgroup', + '_admin_defaulthost', + '_admin_defaulttable', + '_admin_defaultuser', + '_admin_deleteconnector', + '_admin_deletedatabase', + '_admin_deletefield', + '_admin_deletegroup', + '_admin_deletehost', + '_admin_deletetable', + '_admin_deleteuser', + '_admin_duplicategroup', + '_admin_internaldatabase', + '_admin_listconnectors', + '_admin_listdatabases', + '_admin_listfields', + '_admin_listgroups', + '_admin_listhosts', + '_admin_listtables', + '_admin_listusers', + '_admin_refreshconnector', + '_admin_refreshsecurity', + '_admin_servicepath', + '_admin_updateconnector', + '_admin_updatedatabase', + '_admin_updatefield', + '_admin_updategroup', + '_admin_updatehost', + '_admin_updatetable', + '_admin_updateuser', + '_chartfx_activation_string', + '_chartfx_getchallengestring', + '_chop_args', + '_chop_mimes', + '_client_addr_old', + '_client_address_old', + '_client_ip_old', + '_database_names', + '_datasource_reload', + '_date_current', + '_date_format', + '_date_msec', + '_date_parse', + '_execution_timelimit', + '_file_chmod', + '_initialize', + '_jdbc_acceptsurl', + '_jdbc_debug', + '_jdbc_deletehost', + '_jdbc_driverclasses', + '_jdbc_driverinfo', + '_jdbc_metainfo', + '_jdbc_propertyinfo', + '_jdbc_setdriver', + '_lasso_param', + '_log_helper', + '_proc_noparam', + '_proc_withparam', + '_recursion_limit', + '_request_param', + '_security_binaryexpiration', + '_security_flushcaches', + '_security_isserialized', + '_security_serialexpiration', + '_srand', + '_strict_literals', + '_substring', + '_xmlrpc_exconverter', + '_xmlrpc_inconverter', + '_xmlrpc_xmlinconverter', + 'abort', + 'action_addinfo', + 'action_addrecord', + 'action_param', + 'action_params', + 'action_setfoundcount', + 'action_setrecordid', + 'action_settotalcount', + 'action_statement', + 'admin_allowedfileroots', + 'admin_changeuser', + 'admin_createuser', + 'admin_currentgroups', + 'admin_currentuserid', + 'admin_currentusername', + 'admin_getpref', + 'admin_groupassignuser', + 'admin_grouplistusers', + 'admin_groupremoveuser', + 'admin_lassoservicepath', + 'admin_listgroups', + 'admin_refreshlicensing', + 'admin_refreshsecurity', + 'admin_reloaddatasource', + 'admin_removepref', + 'admin_setpref', + 'admin_userexists', + 'admin_userlistgroups', + 'all', + 'and', + 'array', + 'array_iterator', + 'auth', + 'auth_admin', + 'auth_auth', + 'auth_custom', + 'auth_group', + 'auth_prompt', + 'auth_user', + 'base64', + 'bean', + 'bigint', + 'bom_utf16be', + 'bom_utf16le', + 'bom_utf32be', + 'bom_utf32le', + 'bom_utf8', + 'boolean', + 'bw', + 'bytes', + 'cache', + 'cache_delete', + 'cache_empty', + 'cache_exists', + 'cache_fetch', + 'cache_internal', + 'cache_maintenance', + 'cache_object', + 'cache_preferences', + 'cache_store', + 'case', + 'chartfx', + 'chartfx_records', + 'chartfx_serve', + 'checked', + 'choice_list', + 'choice_listitem', + 'choicelistitem', + 'cipher_decrypt', + 'cipher_digest', + 'cipher_encrypt', + 'cipher_hmac', + 'cipher_keylength', + 'cipher_list', + 'click_text', + 'client_addr', + 'client_address', + 'client_authorization', + 'client_browser', + 'client_contentlength', + 'client_contenttype', + 'client_cookielist', + 'client_cookies', + 'client_encoding', + 'client_formmethod', + 'client_getargs', + 'client_getparams', + 'client_headers', + 'client_ip', + 'client_ipfrominteger', + 'client_iptointeger', + 'client_password', + 'client_postargs', + 'client_postparams', + 'client_type', + 'client_url', + 'client_username', + 'cn', + 'column', + 'column_name', + 'column_names', + 'compare_beginswith', + 'compare_contains', + 'compare_endswith', + 'compare_equalto', + 'compare_greaterthan', + 'compare_greaterthanorequals', + 'compare_greaterthanorequls', + 'compare_lessthan', + 'compare_lessthanorequals', + 'compare_notbeginswith', + 'compare_notcontains', + 'compare_notendswith', + 'compare_notequalto', + 'compare_notregexp', + 'compare_regexp', + 'compare_strictequalto', + 'compare_strictnotequalto', + 'compiler_removecacheddoc', + 'compiler_setdefaultparserflags', + 'compress', + 'content_body', + 'content_encoding', + 'content_header', + 'content_type', + 'cookie', + 'cookie_set', + 'curl_ftp_getfile', + 'curl_ftp_getlisting', + 'curl_ftp_putfile', + 'curl_include_url', + 'currency', + 'database_changecolumn', + 'database_changefield', + 'database_createcolumn', + 'database_createfield', + 'database_createtable', + 'database_fmcontainer', + 'database_hostinfo', + 'database_inline', + 'database_name', + 'database_nameitem', + 'database_names', + 'database_realname', + 'database_removecolumn', + 'database_removefield', + 'database_removetable', + 'database_repeating', + 'database_repeating_valueitem', + 'database_repeatingvalueitem', + 'database_schemanameitem', + 'database_schemanames', + 'database_tablecolumn', + 'database_tablenameitem', + 'database_tablenames', + 'datasource_name', + 'datasource_register', + 'date', + 'date__date_current', + 'date__date_format', + 'date__date_msec', + 'date__date_parse', + 'date_add', + 'date_date', + 'date_difference', + 'date_duration', + 'date_format', + 'date_getcurrentdate', + 'date_getday', + 'date_getdayofweek', + 'date_gethour', + 'date_getlocaltimezone', + 'date_getminute', + 'date_getmonth', + 'date_getsecond', + 'date_gettime', + 'date_getyear', + 'date_gmttolocal', + 'date_localtogmt', + 'date_maximum', + 'date_minimum', + 'date_msec', + 'date_setformat', + 'date_subtract', + 'db_layoutnameitem', + 'db_layoutnames', + 'db_nameitem', + 'db_names', + 'db_tablenameitem', + 'db_tablenames', + 'dbi_column_names', + 'dbi_field_names', + 'decimal', + 'decimal_setglobaldefaultprecision', + 'decode_base64', + 'decode_bheader', + 'decode_hex', + 'decode_html', + 'decode_json', + 'decode_qheader', + 'decode_quotedprintable', + 'decode_quotedprintablebytes', + 'decode_url', + 'decode_xml', + 'decompress', + 'decrypt_blowfish', + 'decrypt_blowfish2', + 'default', + 'define_atbegin', + 'define_atend', + 'define_constant', + 'define_prototype', + 'define_tag', + 'define_tagp', + 'define_type', + 'define_typep', + 'deserialize', + 'directory_directorynameitem', + 'directory_lister', + 'directory_nameitem', + 'directorynameitem', + 'dns_default', + 'dns_lookup', + 'dns_response', + 'duration', + 'else', + 'email_batch', + 'email_compose', + 'email_digestchallenge', + 'email_digestresponse', + 'email_extract', + 'email_findemails', + 'email_immediate', + 'email_merge', + 'email_mxerror', + 'email_mxlookup', + 'email_parse', + 'email_pop', + 'email_queue', + 'email_result', + 'email_safeemail', + 'email_send', + 'email_smtp', + 'email_status', + 'email_token', + 'email_translatebreakstocrlf', + 'encode_base64', + 'encode_bheader', + 'encode_break', + 'encode_breaks', + 'encode_crc32', + 'encode_hex', + 'encode_html', + 'encode_htmltoxml', + 'encode_json', + 'encode_qheader', + 'encode_quotedprintable', + 'encode_quotedprintablebytes', + 'encode_set', + 'encode_smart', + 'encode_sql', + 'encode_sql92', + 'encode_stricturl', + 'encode_url', + 'encode_xml', + 'encrypt_blowfish', + 'encrypt_blowfish2', + 'encrypt_crammd5', + 'encrypt_hmac', + 'encrypt_md5', + 'eq', + 'error_adderror', + 'error_code', + 'error_code_aborted', + 'error_code_assert', + 'error_code_bof', + 'error_code_connectioninvalid', + 'error_code_couldnotclosefile', + 'error_code_couldnotcreateoropenfile', + 'error_code_couldnotdeletefile', + 'error_code_couldnotdisposememory', + 'error_code_couldnotlockmemory', + 'error_code_couldnotreadfromfile', + 'error_code_couldnotunlockmemory', + 'error_code_couldnotwritetofile', + 'error_code_criterianotmet', + 'error_code_datasourceerror', + 'error_code_directoryfull', + 'error_code_diskfull', + 'error_code_dividebyzero', + 'error_code_eof', + 'error_code_failure', + 'error_code_fieldrestriction', + 'error_code_file', + 'error_code_filealreadyexists', + 'error_code_filecorrupt', + 'error_code_fileinvalid', + 'error_code_fileinvalidaccessmode', + 'error_code_fileisclosed', + 'error_code_fileisopen', + 'error_code_filelocked', + 'error_code_filenotfound', + 'error_code_fileunlocked', + 'error_code_httpfilenotfound', + 'error_code_illegalinstruction', + 'error_code_illegaluseoffrozeninstance', + 'error_code_invaliddatabase', + 'error_code_invalidfilename', + 'error_code_invalidmemoryobject', + 'error_code_invalidparameter', + 'error_code_invalidpassword', + 'error_code_invalidpathname', + 'error_code_invalidusername', + 'error_code_ioerror', + 'error_code_loopaborted', + 'error_code_memory', + 'error_code_network', + 'error_code_nilpointer', + 'error_code_noerr', + 'error_code_nopermission', + 'error_code_outofmemory', + 'error_code_outofstackspace', + 'error_code_overflow', + 'error_code_postconditionfailed', + 'error_code_preconditionfailed', + 'error_code_resnotfound', + 'error_code_resource', + 'error_code_streamreaderror', + 'error_code_streamwriteerror', + 'error_code_syntaxerror', + 'error_code_tagnotfound', + 'error_code_unknownerror', + 'error_code_varnotfound', + 'error_code_volumedoesnotexist', + 'error_code_webactionnotsupported', + 'error_code_webadderror', + 'error_code_webdeleteerror', + 'error_code_webmodulenotfound', + 'error_code_webnosuchobject', + 'error_code_webrepeatingrelatedfield', + 'error_code_webrequiredfieldmissing', + 'error_code_webtimeout', + 'error_code_webupdateerror', + 'error_columnrestriction', + 'error_currenterror', + 'error_databaseconnectionunavailable', + 'error_databasetimeout', + 'error_deleteerror', + 'error_fieldrestriction', + 'error_filenotfound', + 'error_invaliddatabase', + 'error_invalidpassword', + 'error_invalidusername', + 'error_modulenotfound', + 'error_msg', + 'error_msg_aborted', + 'error_msg_assert', + 'error_msg_bof', + 'error_msg_connectioninvalid', + 'error_msg_couldnotclosefile', + 'error_msg_couldnotcreateoropenfile', + 'error_msg_couldnotdeletefile', + 'error_msg_couldnotdisposememory', + 'error_msg_couldnotlockmemory', + 'error_msg_couldnotreadfromfile', + 'error_msg_couldnotunlockmemory', + 'error_msg_couldnotwritetofile', + 'error_msg_criterianotmet', + 'error_msg_datasourceerror', + 'error_msg_directoryfull', + 'error_msg_diskfull', + 'error_msg_dividebyzero', + 'error_msg_eof', + 'error_msg_failure', + 'error_msg_fieldrestriction', + 'error_msg_file', + 'error_msg_filealreadyexists', + 'error_msg_filecorrupt', + 'error_msg_fileinvalid', + 'error_msg_fileinvalidaccessmode', + 'error_msg_fileisclosed', + 'error_msg_fileisopen', + 'error_msg_filelocked', + 'error_msg_filenotfound', + 'error_msg_fileunlocked', + 'error_msg_httpfilenotfound', + 'error_msg_illegalinstruction', + 'error_msg_illegaluseoffrozeninstance', + 'error_msg_invaliddatabase', + 'error_msg_invalidfilename', + 'error_msg_invalidmemoryobject', + 'error_msg_invalidparameter', + 'error_msg_invalidpassword', + 'error_msg_invalidpathname', + 'error_msg_invalidusername', + 'error_msg_ioerror', + 'error_msg_loopaborted', + 'error_msg_memory', + 'error_msg_network', + 'error_msg_nilpointer', + 'error_msg_noerr', + 'error_msg_nopermission', + 'error_msg_outofmemory', + 'error_msg_outofstackspace', + 'error_msg_overflow', + 'error_msg_postconditionfailed', + 'error_msg_preconditionfailed', + 'error_msg_resnotfound', + 'error_msg_resource', + 'error_msg_streamreaderror', + 'error_msg_streamwriteerror', + 'error_msg_syntaxerror', + 'error_msg_tagnotfound', + 'error_msg_unknownerror', + 'error_msg_varnotfound', + 'error_msg_volumedoesnotexist', + 'error_msg_webactionnotsupported', + 'error_msg_webadderror', + 'error_msg_webdeleteerror', + 'error_msg_webmodulenotfound', + 'error_msg_webnosuchobject', + 'error_msg_webrepeatingrelatedfield', + 'error_msg_webrequiredfieldmissing', + 'error_msg_webtimeout', + 'error_msg_webupdateerror', + 'error_noerror', + 'error_nopermission', + 'error_norecordsfound', + 'error_outofmemory', + 'error_pop', + 'error_push', + 'error_reqcolumnmissing', + 'error_reqfieldmissing', + 'error_requiredcolumnmissing', + 'error_requiredfieldmissing', + 'error_reset', + 'error_seterrorcode', + 'error_seterrormessage', + 'error_updateerror', + 'euro', + 'event_schedule', + 'ew', + 'fail', + 'fail_if', + 'false', + 'field', + 'field_name', + 'field_names', + 'file', + 'file_autoresolvefullpaths', + 'file_chmod', + 'file_control', + 'file_copy', + 'file_create', + 'file_creationdate', + 'file_currenterror', + 'file_delete', + 'file_exists', + 'file_getlinecount', + 'file_getsize', + 'file_isdirectory', + 'file_listdirectory', + 'file_moddate', + 'file_modechar', + 'file_modeline', + 'file_move', + 'file_openread', + 'file_openreadwrite', + 'file_openwrite', + 'file_openwriteappend', + 'file_openwritetruncate', + 'file_probeeol', + 'file_processuploads', + 'file_read', + 'file_readline', + 'file_rename', + 'file_serve', + 'file_setsize', + 'file_stream', + 'file_streamcopy', + 'file_uploads', + 'file_waitread', + 'file_waittimeout', + 'file_waitwrite', + 'file_write', + 'find_soap_ops', + 'form_param', + 'found_count', + 'ft', + 'ftp_getfile', + 'ftp_getlisting', + 'ftp_putfile', + 'full', + 'global', + 'global_defined', + 'global_remove', + 'global_reset', + 'globals', + 'gt', + 'gte', + 'handle', + 'handle_error', + 'header', + 'html_comment', + 'http_getfile', + 'ical_alarm', + 'ical_attribute', + 'ical_calendar', + 'ical_daylight', + 'ical_event', + 'ical_freebusy', + 'ical_item', + 'ical_journal', + 'ical_parse', + 'ical_standard', + 'ical_timezone', + 'ical_todo', + 'if', + 'if_empty', + 'if_false', + 'if_null', + 'if_true', + 'image', + 'image_url', + 'img', + 'include', + 'include_cgi', + 'include_currentpath', + 'include_once', + 'include_raw', + 'include_url', + 'inline', + 'integer', + 'iterate', + 'iterator', + 'java', + 'java_bean', + 'json_records', + 'json_rpccall', + 'keycolumn_name', + 'keycolumn_value', + 'keyfield_name', + 'keyfield_value', + 'lasso_comment', + 'lasso_currentaction', + 'lasso_datasourceis', + 'lasso_datasourceis4d', + 'lasso_datasourceisfilemaker', + 'lasso_datasourceisfilemaker7', + 'lasso_datasourceisfilemaker9', + 'lasso_datasourceisfilemakersa', + 'lasso_datasourceisjdbc', + 'lasso_datasourceislassomysql', + 'lasso_datasourceismysql', + 'lasso_datasourceisodbc', + 'lasso_datasourceisopenbase', + 'lasso_datasourceisoracle', + 'lasso_datasourceispostgresql', + 'lasso_datasourceisspotlight', + 'lasso_datasourceissqlite', + 'lasso_datasourceissqlserver', + 'lasso_datasourcemodulename', + 'lasso_datatype', + 'lasso_disableondemand', + 'lasso_errorreporting', + 'lasso_executiontimelimit', + 'lasso_parser', + 'lasso_process', + 'lasso_sessionid', + 'lasso_siteid', + 'lasso_siteisrunning', + 'lasso_sitename', + 'lasso_siterestart', + 'lasso_sitestart', + 'lasso_sitestop', + 'lasso_tagexists', + 'lasso_tagmodulename', + 'lasso_uniqueid', + 'lasso_updatecheck', + 'lasso_uptime', + 'lasso_version', + 'lassoapp_create', + 'lassoapp_dump', + 'lassoapp_flattendir', + 'lassoapp_getappdata', + 'lassoapp_link', + 'lassoapp_list', + 'lassoapp_process', + 'lassoapp_unitize', + 'layout_name', + 'ldap', + 'ldap_scope_base', + 'ldap_scope_onelevel', + 'ldap_scope_subtree', + 'ldml', + 'ldml_ldml', + 'library', + 'library_once', + 'link', + 'link_currentaction', + 'link_currentactionparams', + 'link_currentactionurl', + 'link_currentgroup', + 'link_currentgroupparams', + 'link_currentgroupurl', + 'link_currentrecord', + 'link_currentrecordparams', + 'link_currentrecordurl', + 'link_currentsearch', + 'link_currentsearchparams', + 'link_currentsearchurl', + 'link_detail', + 'link_detailparams', + 'link_detailurl', + 'link_firstgroup', + 'link_firstgroupparams', + 'link_firstgroupurl', + 'link_firstrecord', + 'link_firstrecordparams', + 'link_firstrecordurl', + 'link_lastgroup', + 'link_lastgroupparams', + 'link_lastgroupurl', + 'link_lastrecord', + 'link_lastrecordparams', + 'link_lastrecordurl', + 'link_nextgroup', + 'link_nextgroupparams', + 'link_nextgroupurl', + 'link_nextrecord', + 'link_nextrecordparams', + 'link_nextrecordurl', + 'link_params', + 'link_prevgroup', + 'link_prevgroupparams', + 'link_prevgroupurl', + 'link_prevrecord', + 'link_prevrecordparams', + 'link_prevrecordurl', + 'link_setformat', + 'link_url', + 'list', + 'list_additem', + 'list_fromlist', + 'list_fromstring', + 'list_getitem', + 'list_itemcount', + 'list_iterator', + 'list_removeitem', + 'list_replaceitem', + 'list_reverseiterator', + 'list_tostring', + 'literal', + 'ljax_end', + 'ljax_hastarget', + 'ljax_include', + 'ljax_start', + 'ljax_target', + 'local', + 'local_defined', + 'local_remove', + 'local_reset', + 'locale_format', + 'locals', + 'log', + 'log_always', + 'log_critical', + 'log_deprecated', + 'log_destination_console', + 'log_destination_database', + 'log_destination_file', + 'log_detail', + 'log_level_critical', + 'log_level_deprecated', + 'log_level_detail', + 'log_level_sql', + 'log_level_warning', + 'log_setdestination', + 'log_sql', + 'log_warning', + 'logicalop_value', + 'logicaloperator_value', + 'loop', + 'loop_abort', + 'loop_continue', + 'loop_count', + 'lt', + 'lte', + 'magick_image', + 'map', + 'map_iterator', + 'match_comparator', + 'match_notrange', + 'match_notregexp', + 'match_range', + 'match_regexp', + 'math_abs', + 'math_acos', + 'math_add', + 'math_asin', + 'math_atan', + 'math_atan2', + 'math_ceil', + 'math_converteuro', + 'math_cos', + 'math_div', + 'math_exp', + 'math_floor', + 'math_internal_rand', + 'math_internal_randmax', + 'math_internal_srand', + 'math_ln', + 'math_log', + 'math_log10', + 'math_max', + 'math_min', + 'math_mod', + 'math_mult', + 'math_pow', + 'math_random', + 'math_range', + 'math_rint', + 'math_roman', + 'math_round', + 'math_sin', + 'math_sqrt', + 'math_sub', + 'math_tan', + 'maxrecords_value', + 'memory_session_driver', + 'mime_type', + 'minimal', + 'misc__srand', + 'misc_randomnumber', + 'misc_roman', + 'misc_valid_creditcard', + 'mysql_session_driver', + 'named_param', + 'namespace_current', + 'namespace_delimiter', + 'namespace_exists', + 'namespace_file_fullpathexists', + 'namespace_global', + 'namespace_import', + 'namespace_load', + 'namespace_page', + 'namespace_unload', + 'namespace_using', + 'neq', + 'net', + 'net_connectinprogress', + 'net_connectok', + 'net_typessl', + 'net_typessltcp', + 'net_typessludp', + 'net_typetcp', + 'net_typeudp', + 'net_waitread', + 'net_waittimeout', + 'net_waitwrite', + 'no_default_output', + 'none', + 'noprocess', + 'not', + 'nrx', + 'nslookup', + 'null', + 'object', + 'once', + 'oneoff', + 'op_logicalvalue', + 'operator_logicalvalue', + 'option', + 'or', + 'os_process', + 'output', + 'output_none', + 'pair', + 'params_up', + 'pdf_barcode', + 'pdf_color', + 'pdf_doc', + 'pdf_font', + 'pdf_image', + 'pdf_list', + 'pdf_read', + 'pdf_serve', + 'pdf_table', + 'pdf_text', + 'percent', + 'portal', + 'postcondition', + 'precondition', + 'prettyprintingnsmap', + 'prettyprintingtypemap', + 'priorityqueue', + 'private', + 'proc_convert', + 'proc_convertbody', + 'proc_convertone', + 'proc_extract', + 'proc_extractone', + 'proc_find', + 'proc_first', + 'proc_foreach', + 'proc_get', + 'proc_join', + 'proc_lasso', + 'proc_last', + 'proc_map_entry', + 'proc_null', + 'proc_regexp', + 'proc_xml', + 'proc_xslt', + 'process', + 'protect', + 'queue', + 'rand', + 'randomnumber', + 'raw', + 'recid_value', + 'record_count', + 'recordcount', + 'recordid_value', + 'records', + 'records_array', + 'records_map', + 'redirect_url', + 'reference', + 'referer', + 'referer_url', + 'referrer', + 'referrer_url', + 'regexp', + 'repeating', + 'repeating_valueitem', + 'repeatingvalueitem', + 'repetition', + 'req_column', + 'req_field', + 'required_column', + 'required_field', + 'response_fileexists', + 'response_filepath', + 'response_localpath', + 'response_path', + 'response_realm', + 'resultset', + 'resultset_count', + 'return', + 'return_value', + 'reverseiterator', + 'roman', + 'row_count', + 'rows', + 'rows_array', + 'run_children', + 'rx', + 'schema_name', + 'scientific', + 'search_args', + 'search_arguments', + 'search_columnitem', + 'search_fielditem', + 'search_operatoritem', + 'search_opitem', + 'search_valueitem', + 'searchfielditem', + 'searchoperatoritem', + 'searchopitem', + 'searchvalueitem', + 'select', + 'selected', + 'self', + 'serialize', + 'series', + 'server_date', + 'server_day', + 'server_ip', + 'server_name', + 'server_port', + 'server_push', + 'server_siteisrunning', + 'server_sitestart', + 'server_sitestop', + 'server_time', + 'session_abort', + 'session_addoutputfilter', + 'session_addvar', + 'session_addvariable', + 'session_deleteexpired', + 'session_driver', + 'session_end', + 'session_id', + 'session_removevar', + 'session_removevariable', + 'session_result', + 'session_setdriver', + 'session_start', + 'set', + 'set_iterator', + 'set_reverseiterator', + 'shown_count', + 'shown_first', + 'shown_last', + 'site_atbegin', + 'site_id', + 'site_name', + 'site_restart', + 'skiprecords_value', + 'sleep', + 'soap_convertpartstopairs', + 'soap_definetag', + 'soap_info', + 'soap_lastrequest', + 'soap_lastresponse', + 'soap_stub', + 'sort_args', + 'sort_arguments', + 'sort_columnitem', + 'sort_fielditem', + 'sort_orderitem', + 'sortcolumnitem', + 'sortfielditem', + 'sortorderitem', + 'sqlite_createdb', + 'sqlite_session_driver', + 'sqlite_setsleepmillis', + 'sqlite_setsleeptries', + 'srand', + 'stack', + 'stock_quote', + 'string', + 'string_charfromname', + 'string_concatenate', + 'string_countfields', + 'string_endswith', + 'string_extract', + 'string_findposition', + 'string_findregexp', + 'string_fordigit', + 'string_getfield', + 'string_getunicodeversion', + 'string_insert', + 'string_isalpha', + 'string_isalphanumeric', + 'string_isdigit', + 'string_ishexdigit', + 'string_islower', + 'string_isnumeric', + 'string_ispunctuation', + 'string_isspace', + 'string_isupper', + 'string_length', + 'string_lowercase', + 'string_remove', + 'string_removeleading', + 'string_removetrailing', + 'string_replace', + 'string_replaceregexp', + 'string_todecimal', + 'string_tointeger', + 'string_uppercase', + 'string_validcharset', + 'table_name', + 'table_realname', + 'tag', + 'tag_name', + 'tags', + 'tags_find', + 'tags_list', + 'tcp_close', + 'tcp_open', + 'tcp_send', + 'tcp_tcp_close', + 'tcp_tcp_open', + 'tcp_tcp_send', + 'thread_abort', + 'thread_atomic', + 'thread_event', + 'thread_exists', + 'thread_getcurrentid', + 'thread_getpriority', + 'thread_info', + 'thread_list', + 'thread_lock', + 'thread_pipe', + 'thread_priority_default', + 'thread_priority_high', + 'thread_priority_low', + 'thread_rwlock', + 'thread_semaphore', + 'thread_setpriority', + 'token_value', + 'total_records', + 'treemap', + 'treemap_iterator', + 'true', + 'url_rewrite', + 'valid_creditcard', + 'valid_date', + 'valid_email', + 'valid_url', + 'value_list', + 'value_listitem', + 'valuelistitem', + 'var', + 'var_defined', + 'var_remove', + 'var_reset', + 'var_set', + 'variable', + 'variable_defined', + 'variable_set', + 'variables', + 'variant_count', + 'vars', + 'wap_isenabled', + 'wap_maxbuttons', + 'wap_maxcolumns', + 'wap_maxhorzpixels', + 'wap_maxrows', + 'wap_maxvertpixels', + 'while', + 'wsdl_extract', + 'wsdl_getbinding', + 'wsdl_getbindingforoperation', + 'wsdl_getbindingoperations', + 'wsdl_getmessagenamed', + 'wsdl_getmessageparts', + 'wsdl_getmessagetriofromporttype', + 'wsdl_getopbodystyle', + 'wsdl_getopbodyuse', + 'wsdl_getoperation', + 'wsdl_getoplocation', + 'wsdl_getopmessagetypes', + 'wsdl_getopsoapaction', + 'wsdl_getportaddress', + 'wsdl_getportsforservice', + 'wsdl_getporttype', + 'wsdl_getporttypeoperation', + 'wsdl_getservicedocumentation', + 'wsdl_getservices', + 'wsdl_gettargetnamespace', + 'wsdl_issoapoperation', + 'wsdl_listoperations', + 'wsdl_maketest', + 'xml', + 'xml_extract', + 'xml_rpc', + 'xml_rpccall', + 'xml_rw', + 'xml_serve', + 'xml_transform', + 'xml_xml', + 'xml_xmlstream', + 'xmlstream', + 'xsd_attribute', + 'xsd_blankarraybase', + 'xsd_blankbase', + 'xsd_buildtype', + 'xsd_cache', + 'xsd_checkcardinality', + 'xsd_continueall', + 'xsd_continueannotation', + 'xsd_continueany', + 'xsd_continueanyattribute', + 'xsd_continueattribute', + 'xsd_continueattributegroup', + 'xsd_continuechoice', + 'xsd_continuecomplexcontent', + 'xsd_continuecomplextype', + 'xsd_continuedocumentation', + 'xsd_continueextension', + 'xsd_continuegroup', + 'xsd_continuekey', + 'xsd_continuelist', + 'xsd_continuerestriction', + 'xsd_continuesequence', + 'xsd_continuesimplecontent', + 'xsd_continuesimpletype', + 'xsd_continueunion', + 'xsd_deserialize', + 'xsd_fullyqualifyname', + 'xsd_generate', + 'xsd_generateblankfromtype', + 'xsd_generateblanksimpletype', + 'xsd_generatetype', + 'xsd_getschematype', + 'xsd_issimpletype', + 'xsd_loadschema', + 'xsd_lookupnamespaceuri', + 'xsd_lookuptype', + 'xsd_processany', + 'xsd_processattribute', + 'xsd_processattributegroup', + 'xsd_processcomplextype', + 'xsd_processelement', + 'xsd_processgroup', + 'xsd_processimport', + 'xsd_processinclude', + 'xsd_processschema', + 'xsd_processsimpletype', + 'xsd_ref', + 'xsd_type' + ] +} +MEMBERS = { + 'Member Methods': [ + 'escape_member', 'oncompare', + 'sameas', 'isa', 'ascopy', + 'asstring', 'ascopydeep', 'type', - 'invoke', - 'atend', - 'decomposeassignment', - 'asgenerator', - 'foreach', - 'eachword', - 'eachline', - 'eachcharacter', - 'foreachwordbreak', - 'foreachlinebreak', - 'foreachcharacter', - 'isempty', - 'isnotempty', - 'ifempty', - 'ifnotempty', - 'size', - 'values', - 'asarray', - 'aslist', - 'asstaticarray', - 'join', - 'get', - 'keys', - 'askeyedgenerator', - 'eachpair', - 'eachkey', - 'foreachpair', - 'foreachkey', - 'front', - 'first', - 'back', - 'last', - 'second', - 'insert', - 'insertfront', - 'insertfirst', - 'insertback', - 'insertfrom', - 'insertlast', - 'remove', - 'removeall', - 'removefront', - 'removefirst', - 'removeback', - 'removelast', - 'difference', - 'intersection', - 'union', - 'contains', - 'find', - 'findposition', - 'componentdelimiter', - 'extensiondelimiter', - 'lastcomponent', - 'foreachpathcomponent', - 'eachcomponent', - 'striplastcomponent', - 'firstcomponent', - 'stripfirstcomponent', - 'splitextension', - 'hastrailingcomponent', - 'isfullpath', - 'findlast', - 'sub', - 'readsomebytes', - 'readbytesfully', - 'readbytes', - 'writebytes', - 'encoding', - 'readstring', - 'writestring', - 'hash', - 'foreachsub', - 'eachsub', - 'push', - 'pop', - 'top', - 'dowithclose', - 'close', - 'fd', - 'do', - 'sum', - 'average', - 'where', - 'select', - 'selectmany', - 'groupby', - 'groupjoin', - 'orderby', - 'orderbydescending', - 'thenby', - 'thenbydescending', - 'skip', - 'take', - 'serialize', - 'serializationelements', - 'acceptdeserializedelement', - 'left', - 'right', - 'up', - 'value', - 'bind', - 'listen', - 'localaddress', - 'remoteaddress', - 'shutdownrd', - 'shutdownwr', - 'shutdownrdwr', - 'setname', - 'contents', - 'tagname', - 'foreachchild', - 'eachchild', - 'foreachmatch', - 'eachmatch', - 'haschildnodes', - 'childnodes', - 'extract', - 'connection', - 'requestparams', - 'stdin', - 'mimes', - 'setstatus', - 'getstatus', - 'writeheaderline', - 'writeheaderbytes', - 'writebodybytes', - 'id', - 'class', - 'style', - 'title', - 'gethtmlattr', - 'lang', - 'onclick', - 'ondblclick', - 'onmousedown', - 'onmouseup', - 'onmouseover', - 'onmousemove', - 'onmouseout', - 'onkeypress', - 'onkeydown', - 'onkeyup', - 'sethtmlattr', - 'gethtmlattrstring', - 'hashtmlattr', - 'addcomponent', - 'attributes', - 'issourcefile', - 'resourceinvokable', - 'resourcename', - 'fullpath', - 'appname', - 'srcpath', - 'resources', - 'foo', - 'startup', - 'validatesessionstable', - 'createtable', - 'fetchdata', - 'savedata', - 'init', - 'kill', - 'expire', - 'jsonlabel', - 'jsonhtml', - 'jsonisleaf', - 'delim', - 'name', - 'path', - 'nodelist', - 'subnode', - 'subnodes', - 'representnoderesult', - 'mime', - 'extensions', - 'representnode', - 'defaultcontentrepresentation', - 'supportscontentrepresentation', - 'acceptpost', - 'htmlcontent', - 'csscontent', - 'jscontent', - 'escape_member', - 'sameas', + 'trait', 'parent', 'settrait', 'oncreate', 'listmethods', 'hasmethod', + 'invoke', 'addtrait', + 'isnota', + 'isallof', + 'isanyof', + 'size', 'gettype', 'istype', 'doccomment', 'requires', 'provides', + 'name', 'subtraits', 'description', + 'hash', 'hosttonet16', 'hosttonet32', 'nettohost16', @@ -2365,10 +3076,14 @@ BUILTINS = { 'bitnot', 'bitshiftleft', 'bitshiftright', + 'bytes', 'abs', 'div', 'dereferencepointer', 'asdecimal', + 'serializationelements', + 'acceptdeserializedelement', + 'serialize', 'deg2rad', 'asstringhex', 'asstringoct', @@ -2384,6 +3099,7 @@ BUILTINS = { 'floor', 'frexp', 'ldexp', + 'log', 'log10', 'modf', 'pow', @@ -2444,12 +3160,15 @@ BUILTINS = { 'length', 'chardigitvalue', 'private_compare', + 'remove', 'charname', 'chartype', 'decompose', 'normalize', 'digit', 'foldcase', + 'sub', + 'integer', 'private_merge', 'unescape', 'trim', @@ -2489,6 +3208,10 @@ BUILTINS = { 'getpropertyvalue', 'hasbinaryproperty', 'asbytes', + 'find', + 'findlast', + 'contains', + 'get', 'equals', 'compare', 'comparecodepointorder', @@ -2501,9 +3224,42 @@ BUILTINS = { 'beginswith', 'endswith', 'replace', + 'values', + 'foreachcharacter', + 'foreachlinebreak', + 'foreachwordbreak', 'eachwordbreak', + 'eachcharacter', + 'foreachmatch', + 'eachmatch', 'encodesql92', 'encodesql', + 'keys', + 'decomposeassignment', + 'firstcomponent', + 'ifempty', + 'eachsub', + 'stripfirstcomponent', + 'isnotempty', + 'first', + 'lastcomponent', + 'foreachpathcomponent', + 'isfullpath', + 'back', + 'second', + 'componentdelimiter', + 'isempty', + 'foreachsub', + 'front', + 'striplastcomponent', + 'eachcomponent', + 'eachline', + 'splitextension', + 'hastrailingcomponent', + 'last', + 'ifnotempty', + 'extensiondelimiter', + 'eachword', 'substring', 'setsize', 'reserve', @@ -2536,6 +3292,8 @@ BUILTINS = { 'encodemd5', 'encodehex', 'decodehex', + 'uncompress', + 'compress', 'detectcharset', 'bestcharset', 'crc', @@ -2546,6 +3304,32 @@ BUILTINS = { 'exportpointerbits', 'foreachbyte', 'eachbyte', + 'setposition', + 'position', + 'value', + 'join', + 'asstaticarray', + 'foreach', + 'findposition', + 'min', + 'groupjoin', + 'orderbydescending', + 'average', + 'take', + 'do', + 'selectmany', + 'skip', + 'select', + 'sum', + 'max', + 'asarray', + 'thenbydescending', + 'aslist', + 'orderby', + 'thenby', + 'where', + 'groupby', + 'asgenerator', 'typename', 'returntype', 'restname', @@ -2575,6 +3359,7 @@ BUILTINS = { 'hostextra', 'hostisdynamic', 'refobj', + 'connection', 'prepared', 'getset', 'addset', @@ -2586,14 +3371,35 @@ BUILTINS = { 'filename', 'expose', 'recover', + 'insert', + 'removeall', 'count', 'exchange', 'findindex', + 'foreachpair', + 'foreachkey', 'sort', + 'insertfirst', + 'difference', + 'removeback', + 'insertback', + 'removelast', + 'removefront', + 'insertfrom', + 'intersection', + 'top', + 'insertlast', + 'push', + 'union', + 'removefirst', + 'insertfront', + 'pop', + 'fd', 'family', 'isvalid', 'isssl', 'open', + 'close', 'read', 'write', 'ioctl', @@ -2637,6 +3443,7 @@ BUILTINS = { 'parse', 'add', 'roll', + 'set', 'getattr', 'setattr', 'clear', @@ -2653,10 +3460,12 @@ BUILTINS = { 'nodevalue', 'nodetype', 'parentnode', + 'childnodes', 'firstchild', 'lastchild', 'previoussibling', 'nextsibling', + 'attributes', 'ownerdocument', 'namespaceuri', 'prefix', @@ -2665,11 +3474,17 @@ BUILTINS = { 'replacechild', 'removechild', 'appendchild', + 'haschildnodes', 'clonenode', 'issupported', 'hasattributes', + 'extract', 'extractone', + 'extractfast', 'transform', + 'foreachchild', + 'eachchild', + 'extractfastone', 'data', 'substringdata', 'appenddata', @@ -2693,6 +3508,7 @@ BUILTINS = { 'createattributens', 'getelementsbytagnamens', 'getelementbyid', + 'tagname', 'getattribute', 'setattribute', 'removeattribute', @@ -2706,6 +3522,8 @@ BUILTINS = { 'setattributenodens', 'hasattribute', 'hasattributens', + 'setname', + 'contents', 'specified', 'ownerelement', 'splittext', @@ -2723,7 +3541,11 @@ BUILTINS = { 'setnameditemns', 'removenameditem', 'removenameditemns', + 'askeyedgenerator', + 'eachpair', + 'eachkey', 'next', + 'readstring', 'readattributevalue', 'attributecount', 'baseuri', @@ -2755,11 +3577,13 @@ BUILTINS = { 'bind_parameter_index', 'reset', 'column_count', + 'column_name', 'column_decltype', 'column_blob', 'column_double', 'column_int64', 'column_text', + 'column_type', 'ismultipart', 'gotfileupload', 'setmaxfilesize', @@ -2775,6 +3599,7 @@ BUILTINS = { 'setreplacepattern', 'setfindpattern', 'setignorecase', + 'output', 'appendreplacement', 'matches', 'private_replaceall', @@ -2793,6 +3618,7 @@ BUILTINS = { 'findsymbols', 'loadlibrary', 'getlibrary', + 'atend', 'f', 'r', 'form', @@ -2801,6 +3627,7 @@ BUILTINS = { 'key', 'by', 'from', + 'init', 'to', 'd', 't', @@ -2818,8 +3645,12 @@ BUILTINS = { 'asxml', 'tabstr', 'toxmlstring', + 'document', 'idmap', 'readidobjects', + 'left', + 'right', + 'up', 'red', 'root', 'getnode', @@ -2832,14 +3663,22 @@ BUILTINS = { 'private_rebalanceforinsert', 'eachnode', 'foreachnode', + 'encoding', 'resolvelinks', + 'readbytesfully', + 'dowithclose', + 'readsomebytes', + 'readbytes', + 'writestring', 'parentdir', 'aslazystring', + 'path', 'openread', 'openwrite', 'openwriteonly', 'openappend', 'opentruncate', + 'writebytes', 'exists', 'modificationtime', 'lastaccesstime', @@ -2853,9 +3692,7 @@ BUILTINS = { 'chmod', 'chown', 'isopen', - 'position', 'setmarker', - 'setposition', 'setmode', 'foreachline', 'lock', @@ -2867,6 +3704,48 @@ BUILTINS = { 'isdir', 'realpath', 'openwith', + 'asraw', + 'rawdiff', + 'getformat', + 'setformat', + 'subtract', + 'gmt', + 'dst', + 'era', + 'year', + 'month', + 'week', + 'weekofyear', + 'weekofmonth', + 'day', + 'dayofmonth', + 'dayofyear', + 'dayofweek', + 'dayofweekinmonth', + 'ampm', + 'am', + 'pm', + 'hour', + 'hourofday', + 'hourofampm', + 'minute', + 'millisecond', + 'zoneoffset', + 'dstoffset', + 'yearwoy', + 'dowlocal', + 'extendedyear', + 'julianday', + 'millisecondsinday', + 'firstdayofweek', + 'fixformat', + 'minutesbetween', + 'hoursbetween', + 'secondsbetween', + 'daysbetween', + 'businessdaysbetween', + 'pdifference', + 'getfield', 'create', 'setcwd', 'foreachentry', @@ -2900,6 +3779,7 @@ BUILTINS = { 'addcomment', 'comments', 'describe', + 'file', 'height', 'pixel', 'resolutionv', @@ -3114,6 +3994,7 @@ BUILTINS = { 'datasourcecolumnnames', 'tablecolumnnames', 'bindcount', + 'sqlite3', 'db', 'tables', 'hastable', @@ -3145,50 +4026,9 @@ BUILTINS = { 'version', 'perform', 'performonce', - 'asraw', - 'rawdiff', - 'getformat', - 'setformat', - 'subtract', - 'gmt', - 'dst', - 'era', - 'year', - 'month', - 'week', - 'weekofyear', - 'weekofmonth', - 'day', - 'dayofmonth', - 'dayofyear', - 'dayofweek', - 'dayofweekinmonth', - 'ampm', - 'am', - 'pm', - 'hour', - 'hourofday', - 'hourofampm', - 'minute', - 'millisecond', - 'zoneoffset', - 'dstoffset', - 'yearwoy', - 'dowlocal', - 'extendedyear', - 'julianday', - 'millisecondsinday', - 'firstdayofweek', - 'fixformat', - 'minutesbetween', - 'hoursbetween', - 'secondsbetween', - 'daysbetween', - 'businessdaysbetween', - 'pdifference', - 'getfield', 's', 'linediffers', + 'sourcefile', 'sourceline', 'sourcecolumn', 'continuationpacket', @@ -3227,11 +4067,14 @@ BUILTINS = { 'handlecontextget', 'handlesource', 'error', + 'setstatus', + 'getstatus', 'stoprunning', 'pollide', 'polldbg', 'runonce', 'arguments', + 'id', 'argumentvalue', 'end', 'start', @@ -3241,6 +4084,7 @@ BUILTINS = { 'actionparams', 'capi', 'doclose', + 'dsinfo', 'isnothing', 'named', 'workinginputcolumns', @@ -3258,6 +4102,13 @@ BUILTINS = { 'head', 'removenode', 'listnode', + 'bind', + 'listen', + 'remoteaddress', + 'shutdownrdwr', + 'shutdownwr', + 'shutdownrd', + 'localaddress', 'accept', 'connect', 'foreachaccept', @@ -3270,6 +4121,7 @@ BUILTINS = { 'fromname', 'fromport', 'env', + 'checked', 'getclass', 'jobjectisa', 'new', @@ -3415,6 +4267,7 @@ BUILTINS = { 'startone', 'addtask', 'waitforcompletion', + 'isidle', 'scanworkers', 'scantasks', 'z', @@ -3428,8 +4281,10 @@ BUILTINS = { 'getfile', 'meta', 'criteria', + 'map', 'valid', 'lazyvalue', + 'dns_response', 'qdcount', 'qdarray', 'answer', @@ -3446,6 +4301,7 @@ BUILTINS = { 'renderbytes', 'renderstring', 'components', + 'addcomponent', 'addcomponents', 'body', 'renderdocumentbytes', @@ -3478,6 +4334,7 @@ BUILTINS = { 'auth', 'quit', 'rset', + 'list', 'uidl', 'retr', 'dele', @@ -3504,6 +4361,7 @@ BUILTINS = { 'parse_parts', 'parse_rawhdrs', 'rawheaders', + 'content_type', 'content_transfer_encoding', 'content_disposition', 'boundary', @@ -3511,10 +4369,12 @@ BUILTINS = { 'cc', 'subject', 'bcc', + 'date', 'pause', 'continue', 'touch', 'refresh', + 'queue', 'status', 'queue_status', 'active_tick', @@ -3567,6 +4427,7 @@ BUILTINS = { 'httpacceptlanguage', 'ishttps', 'cookies', + 'cookie', 'rawheader', 'queryparam', 'postparam', @@ -3580,6 +4441,12 @@ BUILTINS = { 'isxhr', 'reqid', 'statusmsg', + 'requestparams', + 'stdin', + 'mimes', + 'writeheaderline', + 'writeheaderbytes', + 'writebodybytes', 'cap', 'n', 'proxying', @@ -3590,6 +4457,7 @@ BUILTINS = { 'handledevconnection', 'splittoprivatedev', 'getmode', + 'curl', 'novaluelists', 'makeurl', 'choosecolumntype', @@ -3598,6 +4466,7 @@ BUILTINS = { 'buildquery', 'getsortfieldspart', 'endjs', + 'title', 'addjs', 'addjstext', 'addendjs', @@ -3606,7 +4475,26 @@ BUILTINS = { 'addfavicon', 'attrs', 'dtdid', + 'lang', 'xhtml', + 'style', + 'gethtmlattr', + 'hashtmlattr', + 'onmouseover', + 'onkeydown', + 'dir', + 'onclick', + 'onkeypress', + 'onmouseout', + 'onkeyup', + 'onmousemove', + 'onmouseup', + 'ondblclick', + 'onmousedown', + 'sethtmlattr', + 'class', + 'gethtmlattrstring', + 'tag', 'code', 'msg', 'scripttype', @@ -3638,6 +4526,7 @@ BUILTINS = { 'inputtype', 'maxlength', 'for', + 'selected', 'label', 'multiple', 'buff', @@ -3653,6 +4542,7 @@ BUILTINS = { 'addoneheaderline', 'safeexport8bits', 'writeheader', + 'fail', 'connhandler', 'port', 'connectionhandler', @@ -3663,6 +4553,7 @@ BUILTINS = { 'scriptextensions', 'sendfile', 'probemimetype', + 'appname', 'inits', 'installs', 'rootmap', @@ -3677,6 +4568,12 @@ BUILTINS = { 'execinstalls', 'execinits', 'payload', + 'fullpath', + 'resourcename', + 'issourcefile', + 'resourceinvokable', + 'srcpath', + 'resources', 'eligiblepath', 'eligiblepaths', 'expiresminutes', @@ -3685,12 +4582,14 @@ BUILTINS = { 'addzip', 'getzipfilebytes', 'resourcedata', + 'zip', 'zipfile', 'zipname', 'zipfilename', 'rawinvokable', 'route', 'setdestination', + 'getprowcount', 'encodepassword', 'checkuser', 'needinitialization', @@ -3723,6 +4622,13 @@ BUILTINS = { 'nextprune', 'nextprunedelta', 'sessionsdump', + 'startup', + 'validatesessionstable', + 'createtable', + 'fetchdata', + 'savedata', + 'kill', + 'expire', 'prune', 'entry', 'host', @@ -3731,10 +4637,21 @@ BUILTINS = { 'getdefaultstorage', 'onconvert', 'send', + 'nodelist', + 'delim', + 'subnode', + 'subnodes', 'addsubnode', 'removesubnode', 'nodeforpath', + 'representnoderesult', + 'mime', + 'extensions', + 'representnode', 'jsonfornode', + 'defaultcontentrepresentation', + 'supportscontentrepresentation', + 'htmlcontent', 'appmessage', 'appstatus', 'atends', @@ -3747,10 +4664,13 @@ BUILTINS = { 'outputencoding', 'sessionsmap', 'htmlizestacktrace', + 'includes', 'respond', 'sendresponse', 'sendchunk', 'makecookieyumyum', + 'getinclude', + 'include', 'includeonce', 'includelibrary', 'includelibraryonce', @@ -3782,97 +4702,17 @@ BUILTINS = { 'shouldabort', 'gettrigger', 'trigger', - 'rule' + 'rule', + 'foo', + 'jsonlabel', + 'jsonhtml', + 'jsonisleaf', + 'acceptpost', + 'csscontent', + 'jscontent' ], - 'Lasso 8 Tags': [ - '__char', - '__sync_timestamp__', - '_admin_addgroup', - '_admin_adduser', - '_admin_defaultconnector', - '_admin_defaultconnectornames', - '_admin_defaultdatabase', - '_admin_defaultfield', - '_admin_defaultgroup', - '_admin_defaulthost', - '_admin_defaulttable', - '_admin_defaultuser', - '_admin_deleteconnector', - '_admin_deletedatabase', - '_admin_deletefield', - '_admin_deletegroup', - '_admin_deletehost', - '_admin_deletetable', - '_admin_deleteuser', - '_admin_duplicategroup', - '_admin_internaldatabase', - '_admin_listconnectors', - '_admin_listdatabases', - '_admin_listfields', - '_admin_listgroups', - '_admin_listhosts', - '_admin_listtables', - '_admin_listusers', - '_admin_refreshconnector', - '_admin_refreshsecurity', - '_admin_servicepath', - '_admin_updateconnector', - '_admin_updatedatabase', - '_admin_updatefield', - '_admin_updategroup', - '_admin_updatehost', - '_admin_updatetable', - '_admin_updateuser', - '_chartfx_activation_string', - '_chartfx_getchallengestring', - '_chop_args', - '_chop_mimes', - '_client_addr_old', - '_client_address_old', - '_client_ip_old', - '_database_names', - '_datasource_reload', - '_date_current', - '_date_format', - '_date_msec', - '_date_parse', - '_execution_timelimit', - '_file_chmod', - '_initialize', - '_jdbc_acceptsurl', - '_jdbc_debug', - '_jdbc_deletehost', - '_jdbc_driverclasses', - '_jdbc_driverinfo', - '_jdbc_metainfo', - '_jdbc_propertyinfo', - '_jdbc_setdriver', - '_lasso_param', - '_log_helper', - '_proc_noparam', - '_proc_withparam', - '_recursion_limit', - '_request_param', - '_security_binaryexpiration', - '_security_flushcaches', - '_security_isserialized', - '_security_serialexpiration', - '_srand', - '_strict_literals', - '_substring', - '_xmlrpc_exconverter', - '_xmlrpc_inconverter', - '_xmlrpc_xmlinconverter', - 'abort', + 'Lasso 8 Member Tags': [ 'accept', - 'action_addinfo', - 'action_addrecord', - 'action_param', - 'action_params', - 'action_setfoundcount', - 'action_setrecordid', - 'action_settotalcount', - 'action_statement', 'add', 'addattachment', 'addattribute', @@ -3909,36 +4749,13 @@ BUILTINS = { 'addtextarea', 'addtextfield', 'addtextpart', - 'admin_allowedfileroots', - 'admin_changeuser', - 'admin_createuser', - 'admin_currentgroups', - 'admin_currentuserid', - 'admin_currentusername', - 'admin_getpref', - 'admin_groupassignuser', - 'admin_grouplistusers', - 'admin_groupremoveuser', - 'admin_lassoservicepath', - 'admin_listgroups', - 'admin_refreshlicensing', - 'admin_refreshsecurity', - 'admin_reloaddatasource', - 'admin_removepref', - 'admin_setpref', - 'admin_userexists', - 'admin_userlistgroups', 'alarms', - 'all', - 'and', 'annotate', 'answer', 'append', 'appendreplacement', 'appendtail', 'arc', - 'array', - 'array_iterator', 'asasync', 'astype', 'atbegin', @@ -3949,23 +4766,13 @@ BUILTINS = { 'attop', 'attributecount', 'attributes', - 'auth', - 'auth_admin', - 'auth_auth', - 'auth_custom', - 'auth_group', - 'auth_prompt', - 'auth_user', 'authenticate', 'authorize', 'backward', - 'base64', 'baseuri', 'bcc', - 'bean', 'beanproperties', 'beginswith', - 'bigint', 'bind', 'bitand', 'bitclear', @@ -3980,217 +4787,46 @@ BUILTINS = { 'bitxor', 'blur', 'body', - 'bom_utf16be', - 'bom_utf16le', - 'bom_utf32be', - 'bom_utf32le', - 'bom_utf8', - 'boolean', 'boundary', - 'bw', 'bytes', - 'cache', - 'cache_delete', - 'cache_empty', - 'cache_exists', - 'cache_fetch', - 'cache_internal', - 'cache_maintenance', - 'cache_object', - 'cache_preferences', - 'cache_store', 'call', 'cancel', 'capabilities', - 'case', 'cc', 'chardigitvalue', 'charname', 'charset', - 'chartfx', - 'chartfx_records', - 'chartfx_serve', 'chartype', - 'checked', 'children', - 'choice_list', - 'choice_listitem', - 'choicelistitem', - 'cipher_decrypt', - 'cipher_digest', - 'cipher_encrypt', - 'cipher_hmac', - 'cipher_keylength', - 'cipher_list', 'circle', - 'click_text', - 'client_addr', - 'client_address', - 'client_authorization', - 'client_browser', - 'client_contentlength', - 'client_contenttype', - 'client_cookielist', - 'client_cookies', - 'client_encoding', - 'client_formmethod', - 'client_getargs', - 'client_getparams', - 'client_headers', - 'client_ip', - 'client_ipfrominteger', - 'client_iptointeger', - 'client_password', - 'client_postargs', - 'client_postparams', - 'client_type', - 'client_url', - 'client_username', 'close', 'closepath', 'closewrite', - 'cn', 'code', 'colorspace', - 'column', - 'column_name', - 'column_names', 'command', 'comments', 'compare', - 'compare_beginswith', - 'compare_contains', - 'compare_endswith', - 'compare_equalto', - 'compare_greaterthan', - 'compare_greaterthanorequals', - 'compare_greaterthanorequls', - 'compare_lessthan', - 'compare_lessthanorequals', - 'compare_notbeginswith', - 'compare_notcontains', - 'compare_notendswith', - 'compare_notequalto', - 'compare_notregexp', - 'compare_regexp', - 'compare_strictequalto', - 'compare_strictnotequalto', 'comparecodepointorder', 'compile', - 'compiler_removecacheddoc', - 'compiler_setdefaultparserflags', 'composite', - 'compress', 'connect', 'contains', - 'content_body', 'content_disposition', - 'content_encoding', - 'content_header', 'content_transfer_encoding', 'content_type', 'contents', 'contrast', 'convert', - 'cookie', - 'cookie_set', 'crop', - 'curl_ftp_getfile', - 'curl_ftp_getlisting', - 'curl_ftp_putfile', - 'curl_include_url', - 'currency', 'curveto', 'data', - 'database_changecolumn', - 'database_changefield', - 'database_createcolumn', - 'database_createfield', - 'database_createtable', - 'database_fmcontainer', - 'database_hostinfo', - 'database_inline', - 'database_name', - 'database_nameitem', - 'database_names', - 'database_realname', - 'database_removecolumn', - 'database_removefield', - 'database_removetable', - 'database_repeating', - 'database_repeating_valueitem', - 'database_repeatingvalueitem', - 'database_schemanameitem', - 'database_schemanames', - 'database_tablecolumn', - 'database_tablenameitem', - 'database_tablenames', - 'datasource_name', - 'datasource_register', 'date', - 'date__date_current', - 'date__date_format', - 'date__date_msec', - 'date__date_parse', - 'date_add', - 'date_date', - 'date_difference', - 'date_duration', - 'date_format', - 'date_getcurrentdate', - 'date_getday', - 'date_getdayofweek', - 'date_gethour', - 'date_getlocaltimezone', - 'date_getminute', - 'date_getmonth', - 'date_getsecond', - 'date_gettime', - 'date_getyear', - 'date_gmttolocal', - 'date_localtogmt', - 'date_maximum', - 'date_minimum', - 'date_msec', - 'date_setformat', - 'date_subtract', 'day', 'daylights', 'dayofweek', 'dayofyear', - 'db_layoutnameitem', - 'db_layoutnames', - 'db_nameitem', - 'db_names', - 'db_tablenameitem', - 'db_tablenames', - 'dbi_column_names', - 'dbi_field_names', - 'decimal', - 'decimal_setglobaldefaultprecision', - 'decode_base64', - 'decode_bheader', - 'decode_hex', - 'decode_html', - 'decode_json', - 'decode_qheader', - 'decode_quotedprintable', - 'decode_quotedprintablebytes', - 'decode_url', - 'decode_xml', - 'decompress', 'decrement', - 'decrypt_blowfish', - 'decrypt_blowfish2', - 'default', - 'define_atbegin', - 'define_atend', - 'define_constant', - 'define_prototype', - 'define_tag', - 'define_tagp', - 'define_type', - 'define_typep', 'delete', 'depth', 'describe', @@ -4200,242 +4836,17 @@ BUILTINS = { 'detachreference', 'difference', 'digit', - 'directory_directorynameitem', - 'directory_lister', - 'directory_nameitem', - 'directorynameitem', - 'dns_default', - 'dns_lookup', - 'dns_response', 'document', 'down', 'drawtext', 'dst', 'dump', - 'duration', - 'else', - 'email_batch', - 'email_compose', - 'email_digestchallenge', - 'email_digestresponse', - 'email_extract', - 'email_findemails', - 'email_immediate', - 'email_merge', - 'email_mxerror', - 'email_mxlookup', - 'email_parse', - 'email_pop', - 'email_queue', - 'email_result', - 'email_safeemail', - 'email_send', - 'email_smtp', - 'email_status', - 'email_token', - 'email_translatebreakstocrlf', - 'encode_base64', - 'encode_bheader', - 'encode_break', - 'encode_breaks', - 'encode_crc32', - 'encode_hex', - 'encode_html', - 'encode_htmltoxml', - 'encode_json', - 'encode_qheader', - 'encode_quotedprintable', - 'encode_quotedprintablebytes', - 'encode_set', - 'encode_smart', - 'encode_sql', - 'encode_sql92', - 'encode_stricturl', - 'encode_url', - 'encode_xml', - 'encrypt_blowfish', - 'encrypt_blowfish2', - 'encrypt_crammd5', - 'encrypt_hmac', - 'encrypt_md5', 'endswith', 'enhance', - 'eq', 'equals', - 'error_adderror', - 'error_code', - 'error_code_aborted', - 'error_code_assert', - 'error_code_bof', - 'error_code_connectioninvalid', - 'error_code_couldnotclosefile', - 'error_code_couldnotcreateoropenfile', - 'error_code_couldnotdeletefile', - 'error_code_couldnotdisposememory', - 'error_code_couldnotlockmemory', - 'error_code_couldnotreadfromfile', - 'error_code_couldnotunlockmemory', - 'error_code_couldnotwritetofile', - 'error_code_criterianotmet', - 'error_code_datasourceerror', - 'error_code_directoryfull', - 'error_code_diskfull', - 'error_code_dividebyzero', - 'error_code_eof', - 'error_code_failure', - 'error_code_fieldrestriction', - 'error_code_file', - 'error_code_filealreadyexists', - 'error_code_filecorrupt', - 'error_code_fileinvalid', - 'error_code_fileinvalidaccessmode', - 'error_code_fileisclosed', - 'error_code_fileisopen', - 'error_code_filelocked', - 'error_code_filenotfound', - 'error_code_fileunlocked', - 'error_code_httpfilenotfound', - 'error_code_illegalinstruction', - 'error_code_illegaluseoffrozeninstance', - 'error_code_invaliddatabase', - 'error_code_invalidfilename', - 'error_code_invalidmemoryobject', - 'error_code_invalidparameter', - 'error_code_invalidpassword', - 'error_code_invalidpathname', - 'error_code_invalidusername', - 'error_code_ioerror', - 'error_code_loopaborted', - 'error_code_memory', - 'error_code_network', - 'error_code_nilpointer', - 'error_code_noerr', - 'error_code_nopermission', - 'error_code_outofmemory', - 'error_code_outofstackspace', - 'error_code_overflow', - 'error_code_postconditionfailed', - 'error_code_preconditionfailed', - 'error_code_resnotfound', - 'error_code_resource', - 'error_code_streamreaderror', - 'error_code_streamwriteerror', - 'error_code_syntaxerror', - 'error_code_tagnotfound', - 'error_code_unknownerror', - 'error_code_varnotfound', - 'error_code_volumedoesnotexist', - 'error_code_webactionnotsupported', - 'error_code_webadderror', - 'error_code_webdeleteerror', - 'error_code_webmodulenotfound', - 'error_code_webnosuchobject', - 'error_code_webrepeatingrelatedfield', - 'error_code_webrequiredfieldmissing', - 'error_code_webtimeout', - 'error_code_webupdateerror', - 'error_columnrestriction', - 'error_currenterror', - 'error_databaseconnectionunavailable', - 'error_databasetimeout', - 'error_deleteerror', - 'error_fieldrestriction', - 'error_filenotfound', - 'error_invaliddatabase', - 'error_invalidpassword', - 'error_invalidusername', - 'error_modulenotfound', - 'error_msg', - 'error_msg_aborted', - 'error_msg_assert', - 'error_msg_bof', - 'error_msg_connectioninvalid', - 'error_msg_couldnotclosefile', - 'error_msg_couldnotcreateoropenfile', - 'error_msg_couldnotdeletefile', - 'error_msg_couldnotdisposememory', - 'error_msg_couldnotlockmemory', - 'error_msg_couldnotreadfromfile', - 'error_msg_couldnotunlockmemory', - 'error_msg_couldnotwritetofile', - 'error_msg_criterianotmet', - 'error_msg_datasourceerror', - 'error_msg_directoryfull', - 'error_msg_diskfull', - 'error_msg_dividebyzero', - 'error_msg_eof', - 'error_msg_failure', - 'error_msg_fieldrestriction', - 'error_msg_file', - 'error_msg_filealreadyexists', - 'error_msg_filecorrupt', - 'error_msg_fileinvalid', - 'error_msg_fileinvalidaccessmode', - 'error_msg_fileisclosed', - 'error_msg_fileisopen', - 'error_msg_filelocked', - 'error_msg_filenotfound', - 'error_msg_fileunlocked', - 'error_msg_httpfilenotfound', - 'error_msg_illegalinstruction', - 'error_msg_illegaluseoffrozeninstance', - 'error_msg_invaliddatabase', - 'error_msg_invalidfilename', - 'error_msg_invalidmemoryobject', - 'error_msg_invalidparameter', - 'error_msg_invalidpassword', - 'error_msg_invalidpathname', - 'error_msg_invalidusername', - 'error_msg_ioerror', - 'error_msg_loopaborted', - 'error_msg_memory', - 'error_msg_network', - 'error_msg_nilpointer', - 'error_msg_noerr', - 'error_msg_nopermission', - 'error_msg_outofmemory', - 'error_msg_outofstackspace', - 'error_msg_overflow', - 'error_msg_postconditionfailed', - 'error_msg_preconditionfailed', - 'error_msg_resnotfound', - 'error_msg_resource', - 'error_msg_streamreaderror', - 'error_msg_streamwriteerror', - 'error_msg_syntaxerror', - 'error_msg_tagnotfound', - 'error_msg_unknownerror', - 'error_msg_varnotfound', - 'error_msg_volumedoesnotexist', - 'error_msg_webactionnotsupported', - 'error_msg_webadderror', - 'error_msg_webdeleteerror', - 'error_msg_webmodulenotfound', - 'error_msg_webnosuchobject', - 'error_msg_webrepeatingrelatedfield', - 'error_msg_webrequiredfieldmissing', - 'error_msg_webtimeout', - 'error_msg_webupdateerror', - 'error_noerror', - 'error_nopermission', - 'error_norecordsfound', - 'error_outofmemory', - 'error_pop', - 'error_push', - 'error_reqcolumnmissing', - 'error_reqfieldmissing', - 'error_requiredcolumnmissing', - 'error_requiredfieldmissing', - 'error_reset', - 'error_seterrorcode', - 'error_seterrormessage', - 'error_updateerror', 'errors', - 'euro', 'eval', - 'event_schedule', 'events', - 'ew', 'execute', 'export16bits', 'export32bits', @@ -4445,54 +4856,11 @@ BUILTINS = { 'exportstring', 'extract', 'extractone', - 'fail', - 'fail_if', - 'false', - 'field', - 'field_name', - 'field_names', 'fieldnames', 'fieldtype', 'fieldvalue', 'file', - 'file_autoresolvefullpaths', - 'file_chmod', - 'file_control', - 'file_copy', - 'file_create', - 'file_creationdate', - 'file_currenterror', - 'file_delete', - 'file_exists', - 'file_getlinecount', - 'file_getsize', - 'file_isdirectory', - 'file_listdirectory', - 'file_moddate', - 'file_modechar', - 'file_modeline', - 'file_move', - 'file_openread', - 'file_openreadwrite', - 'file_openwrite', - 'file_openwriteappend', - 'file_openwritetruncate', - 'file_probeeol', - 'file_processuploads', - 'file_read', - 'file_readline', - 'file_rename', - 'file_serve', - 'file_setsize', - 'file_stream', - 'file_streamcopy', - 'file_uploads', - 'file_waitread', - 'file_waittimeout', - 'file_waitwrite', - 'file_write', 'find', - 'find_soap_ops', 'findindex', 'findnamespace', 'findnamespacebyhref', @@ -4505,19 +4873,12 @@ BUILTINS = { 'flush', 'foldcase', 'foreach', - 'form_param', 'format', 'forward', - 'found_count', 'freebusies', 'freezetype', 'freezevalue', 'from', - 'ft', - 'ftp_getfile', - 'ftp_getlisting', - 'ftp_putfile', - 'full', 'fulltype', 'generatechecksum', 'get', @@ -4556,17 +4917,8 @@ BUILTINS = { 'gettextalignment', 'gettextsize', 'gettype', - 'global', - 'global_defined', - 'global_remove', - 'global_reset', - 'globals', 'gmt', 'groupcount', - 'gt', - 'gte', - 'handle', - 'handle_error', 'hasattribute', 'haschildren', 'hasvalue', @@ -4577,44 +4929,15 @@ BUILTINS = { 'hosttonet16', 'hosttonet32', 'hour', - 'html_comment', - 'http_getfile', - 'ical_alarm', - 'ical_attribute', - 'ical_calendar', - 'ical_daylight', - 'ical_event', - 'ical_freebusy', - 'ical_item', - 'ical_journal', - 'ical_parse', - 'ical_standard', - 'ical_timezone', - 'ical_todo', 'id', - 'if', - 'if_empty', - 'if_false', - 'if_null', - 'if_true', 'ignorecase', - 'image', - 'image_url', - 'img', 'import16bits', 'import32bits', 'import64bits', 'import8bits', 'importfdf', 'importstring', - 'include', - 'include_cgi', - 'include_currentpath', - 'include_once', - 'include_raw', - 'include_url', 'increment', - 'inline', 'input', 'insert', 'insertatcurrent', @@ -4644,234 +4967,32 @@ BUILTINS = { 'isuuppercase', 'isuwhitespace', 'iswhitespace', - 'iterate', 'iterator', - 'java', - 'java_bean', 'javascript', 'join', 'journals', - 'json_records', - 'json_rpccall', 'key', - 'keycolumn_name', - 'keycolumn_value', - 'keyfield_name', - 'keyfield_value', 'keys', - 'lasso_comment', - 'lasso_currentaction', - 'lasso_datasourceis', - 'lasso_datasourceis4d', - 'lasso_datasourceisfilemaker', - 'lasso_datasourceisfilemaker7', - 'lasso_datasourceisfilemaker9', - 'lasso_datasourceisfilemakersa', - 'lasso_datasourceisjdbc', - 'lasso_datasourceislassomysql', - 'lasso_datasourceismysql', - 'lasso_datasourceisodbc', - 'lasso_datasourceisopenbase', - 'lasso_datasourceisoracle', - 'lasso_datasourceispostgresql', - 'lasso_datasourceisspotlight', - 'lasso_datasourceissqlite', - 'lasso_datasourceissqlserver', - 'lasso_datasourcemodulename', - 'lasso_datatype', - 'lasso_disableondemand', - 'lasso_errorreporting', - 'lasso_executiontimelimit', - 'lasso_parser', - 'lasso_process', - 'lasso_sessionid', - 'lasso_siteid', - 'lasso_siteisrunning', - 'lasso_sitename', - 'lasso_siterestart', - 'lasso_sitestart', - 'lasso_sitestop', - 'lasso_tagexists', - 'lasso_tagmodulename', - 'lasso_uniqueid', - 'lasso_updatecheck', - 'lasso_uptime', - 'lasso_version', - 'lassoapp_create', - 'lassoapp_dump', - 'lassoapp_flattendir', - 'lassoapp_getappdata', - 'lassoapp_link', - 'lassoapp_list', - 'lassoapp_process', - 'lassoapp_unitize', 'last', 'lastchild', 'lasterror', - 'layout_name', - 'ldap', - 'ldap_scope_base', - 'ldap_scope_onelevel', - 'ldap_scope_subtree', - 'ldml', - 'ldml_ldml', 'left', 'length', - 'library', - 'library_once', 'line', - 'link', - 'link_currentaction', - 'link_currentactionparams', - 'link_currentactionurl', - 'link_currentgroup', - 'link_currentgroupparams', - 'link_currentgroupurl', - 'link_currentrecord', - 'link_currentrecordparams', - 'link_currentrecordurl', - 'link_currentsearch', - 'link_currentsearchparams', - 'link_currentsearchurl', - 'link_detail', - 'link_detailparams', - 'link_detailurl', - 'link_firstgroup', - 'link_firstgroupparams', - 'link_firstgroupurl', - 'link_firstrecord', - 'link_firstrecordparams', - 'link_firstrecordurl', - 'link_lastgroup', - 'link_lastgroupparams', - 'link_lastgroupurl', - 'link_lastrecord', - 'link_lastrecordparams', - 'link_lastrecordurl', - 'link_nextgroup', - 'link_nextgroupparams', - 'link_nextgroupurl', - 'link_nextrecord', - 'link_nextrecordparams', - 'link_nextrecordurl', - 'link_params', - 'link_prevgroup', - 'link_prevgroupparams', - 'link_prevgroupurl', - 'link_prevrecord', - 'link_prevrecordparams', - 'link_prevrecordurl', - 'link_setformat', - 'link_url', - 'list', - 'list_additem', - 'list_fromlist', - 'list_fromstring', - 'list_getitem', - 'list_itemcount', - 'list_iterator', - 'list_removeitem', - 'list_replaceitem', - 'list_reverseiterator', - 'list_tostring', 'listen', - 'literal', - 'ljax_end', - 'ljax_hastarget', - 'ljax_include', - 'ljax_start', - 'ljax_target', - 'local', - 'local_defined', - 'local_remove', - 'local_reset', 'localaddress', - 'locale_format', 'localname', - 'locals', 'lock', - 'log', - 'log_always', - 'log_critical', - 'log_deprecated', - 'log_destination_console', - 'log_destination_database', - 'log_destination_file', - 'log_detail', - 'log_level_critical', - 'log_level_deprecated', - 'log_level_detail', - 'log_level_sql', - 'log_level_warning', - 'log_setdestination', - 'log_sql', - 'log_warning', - 'logicalop_value', - 'logicaloperator_value', 'lookupnamespace', - 'loop', - 'loop_abort', - 'loop_continue', - 'loop_count', 'lowercase', - 'lt', - 'lte', - 'magick_image', - 'map', - 'map_iterator', 'marker', - 'match_comparator', - 'match_notrange', - 'match_notregexp', - 'match_range', - 'match_regexp', 'matches', 'matchesstart', 'matchposition', 'matchstring', - 'math_abs', - 'math_acos', - 'math_add', - 'math_asin', - 'math_atan', - 'math_atan2', - 'math_ceil', - 'math_converteuro', - 'math_cos', - 'math_div', - 'math_exp', - 'math_floor', - 'math_internal_rand', - 'math_internal_randmax', - 'math_internal_srand', - 'math_ln', - 'math_log', - 'math_log10', - 'math_max', - 'math_min', - 'math_mod', - 'math_mult', - 'math_pow', - 'math_random', - 'math_range', - 'math_rint', - 'math_roman', - 'math_round', - 'math_sin', - 'math_sqrt', - 'math_sub', - 'math_tan', - 'maxrecords_value', - 'memory_session_driver', 'merge', 'millisecond', - 'mime_type', - 'minimal', 'minute', - 'misc__srand', - 'misc_randomnumber', - 'misc_roman', - 'misc_valid_creditcard', 'mode', 'modulate', 'month', @@ -4880,113 +5001,30 @@ BUILTINS = { 'movetoelement', 'movetofirstattribute', 'movetonextattribute', - 'mysql_session_driver', 'name', - 'named_param', - 'namespace_current', - 'namespace_delimiter', - 'namespace_exists', - 'namespace_file_fullpathexists', - 'namespace_global', - 'namespace_import', - 'namespace_load', - 'namespace_page', - 'namespace_unload', - 'namespace_using', 'namespaces', 'namespaceuri', - 'neq', - 'net', - 'net_connectinprogress', - 'net_connectok', - 'net_typessl', - 'net_typessltcp', - 'net_typessludp', - 'net_typetcp', - 'net_typeudp', - 'net_waitread', - 'net_waittimeout', - 'net_waitwrite', 'nettohost16', 'nettohost32', 'newchild', 'next', 'nextsibling', - 'no_default_output', 'nodetype', - 'none', - 'noprocess', - 'not', - 'nrx', - 'nslookup', - 'null', - 'object', - 'once', - 'oneoff', - 'op_logicalvalue', 'open', - 'operator_logicalvalue', - 'option', - 'or', - 'os_process', 'output', - 'output_none', 'padleading', 'padtrailing', 'pagecount', 'pagesize', - 'pair', 'paraminfo', 'params', - 'params_up', 'parent', 'path', - 'pdf_barcode', - 'pdf_color', - 'pdf_doc', - 'pdf_font', - 'pdf_image', - 'pdf_list', - 'pdf_read', - 'pdf_serve', - 'pdf_table', - 'pdf_text', - 'percent', 'pixel', - 'portal', 'position', - 'postcondition', - 'precondition', 'prefix', - 'prettyprintingnsmap', - 'prettyprintingtypemap', 'previoussibling', - 'priorityqueue', - 'private', - 'proc_convert', - 'proc_convertbody', - 'proc_convertone', - 'proc_extract', - 'proc_extractone', - 'proc_find', - 'proc_first', - 'proc_foreach', - 'proc_get', - 'proc_join', - 'proc_lasso', - 'proc_last', - 'proc_map_entry', - 'proc_null', - 'proc_regexp', - 'proc_xml', - 'proc_xslt', - 'process', 'properties', - 'protect', - 'queue', - 'rand', - 'randomnumber', - 'raw', 'rawheaders', 'read', 'readattributevalue', @@ -4996,24 +5034,10 @@ BUILTINS = { 'readlock', 'readstring', 'readunlock', - 'recid_value', 'recipients', - 'record_count', - 'recordcount', - 'recordid_value', - 'records', - 'records_array', - 'records_map', 'rect', - 'redirect_url', 'refcount', - 'reference', - 'referer', - 'referer_url', 'referrals', - 'referrer', - 'referrer_url', - 'regexp', 'remoteaddress', 'remove', 'removeall', @@ -5026,96 +5050,31 @@ BUILTINS = { 'removenamespace', 'removetrailing', 'render', - 'repeating', - 'repeating_valueitem', - 'repeatingvalueitem', - 'repetition', 'replace', 'replaceall', 'replacefirst', 'replacepattern', 'replacewith', - 'req_column', - 'req_field', - 'required_column', - 'required_field', 'reserve', 'reset', 'resolutionh', 'resolutionv', 'response', - 'response_fileexists', - 'response_filepath', - 'response_localpath', - 'response_path', - 'response_realm', 'results', - 'resultset', - 'resultset_count', 'retrieve', - 'return', - 'return_value', 'returntype', 'reverse', 'reverseiterator', 'right', - 'roman', 'rotate', - 'row_count', - 'rows', - 'rows_array', 'run', - 'run_children', - 'rx', 'save', 'scale', - 'schema_name', - 'scientific', 'search', - 'search_args', - 'search_arguments', - 'search_columnitem', - 'search_fielditem', - 'search_operatoritem', - 'search_opitem', - 'search_valueitem', - 'searchfielditem', - 'searchoperatoritem', - 'searchopitem', - 'searchvalueitem', 'second', - 'select', - 'selected', - 'self', 'send', 'serialize', - 'series', - 'server_date', - 'server_day', - 'server_ip', - 'server_name', - 'server_port', - 'server_push', - 'server_siteisrunning', - 'server_sitestart', - 'server_sitestop', - 'server_time', - 'session_abort', - 'session_addoutputfilter', - 'session_addvar', - 'session_addvariable', - 'session_deleteexpired', - 'session_driver', - 'session_end', - 'session_id', - 'session_removevar', - 'session_removevariable', - 'session_result', - 'session_setdriver', - 'session_start', 'set', - 'set_iterator', - 'set_reverseiterator', 'setalignment', 'setbarheight', 'setbarmultiplier', @@ -5163,126 +5122,30 @@ BUILTINS = { 'showchecksum', 'showcode39startstop', 'showeanguardbars', - 'shown_count', - 'shown_first', - 'shown_last', 'signal', 'signalall', - 'site_atbegin', - 'site_id', - 'site_name', - 'site_restart', 'size', - 'skiprecords_value', - 'sleep', 'smooth', - 'soap_convertpartstopairs', - 'soap_definetag', - 'soap_info', - 'soap_lastrequest', - 'soap_lastresponse', - 'soap_stub', 'sort', - 'sort_args', - 'sort_arguments', - 'sort_columnitem', - 'sort_fielditem', - 'sort_orderitem', - 'sortcolumnitem', - 'sortfielditem', - 'sortorderitem', 'sortwith', 'split', - 'sqlite_createdb', - 'sqlite_session_driver', - 'sqlite_setsleepmillis', - 'sqlite_setsleeptries', - 'srand', - 'stack', 'standards', 'steal', - 'stock_quote', - 'string', - 'string_charfromname', - 'string_concatenate', - 'string_countfields', - 'string_endswith', - 'string_extract', - 'string_findposition', - 'string_findregexp', - 'string_fordigit', - 'string_getfield', - 'string_getunicodeversion', - 'string_insert', - 'string_isalpha', - 'string_isalphanumeric', - 'string_isdigit', - 'string_ishexdigit', - 'string_islower', - 'string_isnumeric', - 'string_ispunctuation', - 'string_isspace', - 'string_isupper', - 'string_length', - 'string_lowercase', - 'string_remove', - 'string_removeleading', - 'string_removetrailing', - 'string_replace', - 'string_replaceregexp', - 'string_todecimal', - 'string_tointeger', - 'string_uppercase', - 'string_validcharset', 'subject', 'substring', 'subtract', 'swapbytes', - 'table_name', - 'table_realname', - 'tag', - 'tag_name', - 'tags', - 'tags_find', - 'tags_list', - 'tcp_close', - 'tcp_open', - 'tcp_send', - 'tcp_tcp_close', - 'tcp_tcp_open', - 'tcp_tcp_send', 'textwidth', - 'thread_abort', - 'thread_atomic', - 'thread_event', - 'thread_exists', - 'thread_getcurrentid', - 'thread_getpriority', - 'thread_info', - 'thread_list', - 'thread_lock', - 'thread_pipe', - 'thread_priority_default', - 'thread_priority_high', - 'thread_priority_low', - 'thread_rwlock', - 'thread_semaphore', - 'thread_setpriority', 'time', 'timezones', 'titlecase', 'to', 'todos', - 'token_value', 'tolower', - 'total_records', 'totitle', 'toupper', 'transform', - 'treemap', - 'treemap_iterator', 'trim', - 'true', 'type', 'unescape', 'union', @@ -5291,126 +5154,19 @@ BUILTINS = { 'unserialize', 'up', 'uppercase', - 'url_rewrite', - 'valid_creditcard', - 'valid_date', - 'valid_email', - 'valid_url', 'value', - 'value_list', - 'value_listitem', - 'valuelistitem', 'values', 'valuetype', - 'var', - 'var_defined', - 'var_remove', - 'var_reset', - 'var_set', - 'variable', - 'variable_defined', - 'variable_set', - 'variables', - 'variant_count', - 'vars', 'wait', - 'wap_isenabled', - 'wap_maxbuttons', - 'wap_maxcolumns', - 'wap_maxhorzpixels', - 'wap_maxrows', - 'wap_maxvertpixels', 'waskeyword', 'week', - 'while', 'width', 'write', 'writelock', 'writeto', 'writeunlock', - 'wsdl_extract', - 'wsdl_getbinding', - 'wsdl_getbindingforoperation', - 'wsdl_getbindingoperations', - 'wsdl_getmessagenamed', - 'wsdl_getmessageparts', - 'wsdl_getmessagetriofromporttype', - 'wsdl_getopbodystyle', - 'wsdl_getopbodyuse', - 'wsdl_getoperation', - 'wsdl_getoplocation', - 'wsdl_getopmessagetypes', - 'wsdl_getopsoapaction', - 'wsdl_getportaddress', - 'wsdl_getportsforservice', - 'wsdl_getporttype', - 'wsdl_getporttypeoperation', - 'wsdl_getservicedocumentation', - 'wsdl_getservices', - 'wsdl_gettargetnamespace', - 'wsdl_issoapoperation', - 'wsdl_listoperations', - 'wsdl_maketest', - 'xml', - 'xml_extract', - 'xml_rpc', - 'xml_rpccall', - 'xml_rw', - 'xml_serve', - 'xml_transform', - 'xml_xml', - 'xml_xmlstream', 'xmllang', 'xmlschematype', - 'xmlstream', - 'xsd_attribute', - 'xsd_blankarraybase', - 'xsd_blankbase', - 'xsd_buildtype', - 'xsd_cache', - 'xsd_checkcardinality', - 'xsd_continueall', - 'xsd_continueannotation', - 'xsd_continueany', - 'xsd_continueanyattribute', - 'xsd_continueattribute', - 'xsd_continueattributegroup', - 'xsd_continuechoice', - 'xsd_continuecomplexcontent', - 'xsd_continuecomplextype', - 'xsd_continuedocumentation', - 'xsd_continueextension', - 'xsd_continuegroup', - 'xsd_continuekey', - 'xsd_continuelist', - 'xsd_continuerestriction', - 'xsd_continuesequence', - 'xsd_continuesimplecontent', - 'xsd_continuesimpletype', - 'xsd_continueunion', - 'xsd_deserialize', - 'xsd_fullyqualifyname', - 'xsd_generate', - 'xsd_generateblankfromtype', - 'xsd_generateblanksimpletype', - 'xsd_generatetype', - 'xsd_getschematype', - 'xsd_issimpletype', - 'xsd_loadschema', - 'xsd_lookupnamespaceuri', - 'xsd_lookuptype', - 'xsd_processany', - 'xsd_processattribute', - 'xsd_processattributegroup', - 'xsd_processcomplextype', - 'xsd_processelement', - 'xsd_processgroup', - 'xsd_processimport', - 'xsd_processinclude', - 'xsd_processschema', - 'xsd_processsimpletype', - 'xsd_ref', - 'xsd_type', 'year' ] } diff --git a/pygments/lexers/_mapping.py b/pygments/lexers/_mapping.py index 53e09176..a8c54d39 100644 --- a/pygments/lexers/_mapping.py +++ b/pygments/lexers/_mapping.py @@ -32,14 +32,15 @@ LEXERS = { 'AspectJLexer': ('pygments.lexers.jvm', 'AspectJ', ('aspectj',), ('*.aj',), ('text/x-aspectj',)), 'AsymptoteLexer': ('pygments.lexers.other', 'Asymptote', ('asy', 'asymptote'), ('*.asy',), ('text/x-asymptote',)), 'AutoItLexer': ('pygments.lexers.other', 'AutoIt', ('autoit', 'Autoit'), ('*.au3',), ('text/x-autoit',)), - 'AutohotkeyLexer': ('pygments.lexers.other', 'autohotkey', ('ahk',), ('*.ahk', '*.ahkl'), ('text/x-autohotkey',)), + 'AutohotkeyLexer': ('pygments.lexers.other', 'autohotkey', ('ahk', 'autohotkey'), ('*.ahk', '*.ahkl'), ('text/x-autohotkey',)), 'AwkLexer': ('pygments.lexers.other', 'Awk', ('awk', 'gawk', 'mawk', 'nawk'), ('*.awk',), ('application/x-awk',)), 'BBCodeLexer': ('pygments.lexers.text', 'BBCode', ('bbcode',), (), ('text/x-bbcode',)), 'BaseMakefileLexer': ('pygments.lexers.text', 'Base Makefile', ('basemake',), (), ()), 'BashLexer': ('pygments.lexers.shell', 'Bash', ('bash', 'sh', 'ksh'), ('*.sh', '*.ksh', '*.bash', '*.ebuild', '*.eclass', '.bashrc', 'bashrc', '.bash_*', 'bash_*'), ('application/x-sh', 'application/x-shellscript')), 'BashSessionLexer': ('pygments.lexers.shell', 'Bash Session', ('console',), ('*.sh-session',), ('application/x-shell-session',)), - 'BatchLexer': ('pygments.lexers.shell', 'Batchfile', ('bat',), ('*.bat', '*.cmd'), ('application/x-dos-batch',)), + 'BatchLexer': ('pygments.lexers.shell', 'Batchfile', ('bat', 'dosbatch', 'winbatch'), ('*.bat', '*.cmd'), ('application/x-dos-batch',)), 'BefungeLexer': ('pygments.lexers.other', 'Befunge', ('befunge',), ('*.befunge',), ('application/x-befunge',)), + 'BlitzBasicLexer': ('pygments.lexers.compiled', 'BlitzBasic', ('blitzbasic', 'b3d', 'bplus'), ('*.bb', '*.decls'), ('text/x-bb',)), 'BlitzMaxLexer': ('pygments.lexers.compiled', 'BlitzMax', ('blitzmax', 'bmax'), ('*.bmx',), ('text/x-bmx',)), 'BooLexer': ('pygments.lexers.dotnet', 'Boo', ('boo',), ('*.boo',), ('text/x-boo',)), 'BrainfuckLexer': ('pygments.lexers.other', 'Brainfuck', ('brainfuck', 'bf'), ('*.bf', '*.b'), ('application/x-brainfuck',)), @@ -54,17 +55,18 @@ LEXERS = { 'CbmBasicV2Lexer': ('pygments.lexers.other', 'CBM BASIC V2', ('cbmbas',), ('*.bas',), ()), 'CeylonLexer': ('pygments.lexers.jvm', 'Ceylon', ('ceylon',), ('*.ceylon',), ('text/x-ceylon',)), 'Cfengine3Lexer': ('pygments.lexers.other', 'CFEngine3', ('cfengine3', 'cf3'), ('*.cf',), ()), - 'CheetahHtmlLexer': ('pygments.lexers.templates', 'HTML+Cheetah', ('html+cheetah', 'html+spitfire'), (), ('text/html+cheetah', 'text/html+spitfire')), + 'CheetahHtmlLexer': ('pygments.lexers.templates', 'HTML+Cheetah', ('html+cheetah', 'html+spitfire', 'htmlcheetah'), (), ('text/html+cheetah', 'text/html+spitfire')), 'CheetahJavascriptLexer': ('pygments.lexers.templates', 'JavaScript+Cheetah', ('js+cheetah', 'javascript+cheetah', 'js+spitfire', 'javascript+spitfire'), (), ('application/x-javascript+cheetah', 'text/x-javascript+cheetah', 'text/javascript+cheetah', 'application/x-javascript+spitfire', 'text/x-javascript+spitfire', 'text/javascript+spitfire')), 'CheetahLexer': ('pygments.lexers.templates', 'Cheetah', ('cheetah', 'spitfire'), ('*.tmpl', '*.spt'), ('application/x-cheetah', 'application/x-spitfire')), 'CheetahXmlLexer': ('pygments.lexers.templates', 'XML+Cheetah', ('xml+cheetah', 'xml+spitfire'), (), ('application/xml+cheetah', 'application/xml+spitfire')), + 'ClayLexer': ('pygments.lexers.compiled', 'Clay', ('clay',), ('*.clay',), ('text/x-clay',)), 'ClojureLexer': ('pygments.lexers.jvm', 'Clojure', ('clojure', 'clj'), ('*.clj',), ('text/x-clojure', 'application/x-clojure')), 'CobolFreeformatLexer': ('pygments.lexers.compiled', 'COBOLFree', ('cobolfree',), ('*.cbl', '*.CBL'), ()), 'CobolLexer': ('pygments.lexers.compiled', 'COBOL', ('cobol',), ('*.cob', '*.COB', '*.cpy', '*.CPY'), ('text/x-cobol',)), - 'CoffeeScriptLexer': ('pygments.lexers.web', 'CoffeeScript', ('coffee-script', 'coffeescript'), ('*.coffee',), ('text/coffeescript',)), + 'CoffeeScriptLexer': ('pygments.lexers.web', 'CoffeeScript', ('coffee-script', 'coffeescript', 'coffee'), ('*.coffee',), ('text/coffeescript',)), 'ColdfusionHtmlLexer': ('pygments.lexers.templates', 'Coldfusion HTML', ('cfm',), ('*.cfm', '*.cfml', '*.cfc'), ('application/x-coldfusion',)), 'ColdfusionLexer': ('pygments.lexers.templates', 'cfstatement', ('cfs',), (), ()), - 'CommonLispLexer': ('pygments.lexers.functional', 'Common Lisp', ('common-lisp', 'cl'), ('*.cl', '*.lisp', '*.el'), ('text/x-common-lisp',)), + 'CommonLispLexer': ('pygments.lexers.functional', 'Common Lisp', ('common-lisp', 'cl', 'lisp'), ('*.cl', '*.lisp', '*.el'), ('text/x-common-lisp',)), 'CoqLexer': ('pygments.lexers.functional', 'Coq', ('coq',), ('*.v',), ('text/x-coq',)), 'CppLexer': ('pygments.lexers.compiled', 'C++', ('cpp', 'c++'), ('*.cpp', '*.hpp', '*.c++', '*.h++', '*.cc', '*.hh', '*.cxx', '*.hxx', '*.C', '*.H', '*.cp', '*.CPP'), ('text/x-c++hdr', 'text/x-c++src')), 'CppObjdumpLexer': ('pygments.lexers.asm', 'cpp-objdump', ('cpp-objdump', 'c++-objdumb', 'cxx-objdump'), ('*.cpp-objdump', '*.c++-objdump', '*.cxx-objdump'), ('text/x-cpp-objdump',)), @@ -76,12 +78,12 @@ LEXERS = { 'CssPhpLexer': ('pygments.lexers.templates', 'CSS+PHP', ('css+php',), (), ('text/css+php',)), 'CssSmartyLexer': ('pygments.lexers.templates', 'CSS+Smarty', ('css+smarty',), (), ('text/css+smarty',)), 'CudaLexer': ('pygments.lexers.compiled', 'CUDA', ('cuda', 'cu'), ('*.cu', '*.cuh'), ('text/x-cuda',)), - 'CythonLexer': ('pygments.lexers.compiled', 'Cython', ('cython', 'pyx'), ('*.pyx', '*.pxd', '*.pxi'), ('text/x-cython', 'application/x-cython')), + 'CythonLexer': ('pygments.lexers.compiled', 'Cython', ('cython', 'pyx', 'pyrex'), ('*.pyx', '*.pxd', '*.pxi'), ('text/x-cython', 'application/x-cython')), 'DLexer': ('pygments.lexers.compiled', 'D', ('d',), ('*.d', '*.di'), ('text/x-dsrc',)), 'DObjdumpLexer': ('pygments.lexers.asm', 'd-objdump', ('d-objdump',), ('*.d-objdump',), ('text/x-d-objdump',)), 'DarcsPatchLexer': ('pygments.lexers.text', 'Darcs Patch', ('dpatch',), ('*.dpatch', '*.darcspatch'), ()), 'DartLexer': ('pygments.lexers.web', 'Dart', ('dart',), ('*.dart',), ('text/x-dart',)), - 'DebianControlLexer': ('pygments.lexers.text', 'Debian Control file', ('control',), ('control',), ()), + 'DebianControlLexer': ('pygments.lexers.text', 'Debian Control file', ('control', 'debcontrol'), ('control',), ()), 'DelphiLexer': ('pygments.lexers.compiled', 'Delphi', ('delphi', 'pas', 'pascal', 'objectpascal'), ('*.pas',), ('text/x-pascal',)), 'DgLexer': ('pygments.lexers.agile', 'dg', ('dg',), ('*.dg',), ('text/x-dg',)), 'DiffLexer': ('pygments.lexers.text', 'Diff', ('diff', 'udiff'), ('*.diff', '*.patch'), ('text/x-diff', 'text/x-patch')), @@ -93,6 +95,7 @@ LEXERS = { 'DylanLidLexer': ('pygments.lexers.compiled', 'DylanLID', ('dylan-lid', 'lid'), ('*.lid', '*.hdp'), ('text/x-dylan-lid',)), 'ECLLexer': ('pygments.lexers.other', 'ECL', ('ecl',), ('*.ecl',), ('application/x-ecl',)), 'ECLexer': ('pygments.lexers.compiled', 'eC', ('ec',), ('*.ec', '*.eh'), ('text/x-echdr', 'text/x-ecsrc')), + 'EbnfLexer': ('pygments.lexers.text', 'EBNF', ('ebnf',), ('*.ebnf',), ('text/x-ebnf',)), 'ElixirConsoleLexer': ('pygments.lexers.functional', 'Elixir iex session', ('iex',), (), ('text/x-elixir-shellsession',)), 'ElixirLexer': ('pygments.lexers.functional', 'Elixir', ('elixir', 'ex', 'exs'), ('*.ex', '*.exs'), ('text/x-elixir',)), 'ErbLexer': ('pygments.lexers.templates', 'ERB', ('erb',), (), ('application/x-ruby-templating',)), @@ -109,7 +112,7 @@ LEXERS = { 'FortranLexer': ('pygments.lexers.compiled', 'Fortran', ('fortran',), ('*.f', '*.f90', '*.F', '*.F90'), ('text/x-fortran',)), 'FoxProLexer': ('pygments.lexers.foxpro', 'FoxPro', ('Clipper', 'XBase'), ('*.PRG', '*.prg'), ()), 'GLShaderLexer': ('pygments.lexers.compiled', 'GLSL', ('glsl',), ('*.vert', '*.frag', '*.geo'), ('text/x-glslsrc',)), - 'GasLexer': ('pygments.lexers.asm', 'GAS', ('gas',), ('*.s', '*.S'), ('text/x-gas',)), + 'GasLexer': ('pygments.lexers.asm', 'GAS', ('gas', 'asm'), ('*.s', '*.S'), ('text/x-gas',)), 'GenshiLexer': ('pygments.lexers.templates', 'Genshi', ('genshi', 'kid', 'xml+genshi', 'xml+kid'), ('*.kid',), ('application/x-genshi', 'application/x-kid')), 'GenshiTextLexer': ('pygments.lexers.templates', 'Genshi Text', ('genshitext',), (), ('application/x-genshi-text', 'text/x-genshi')), 'GettextLexer': ('pygments.lexers.text', 'Gettext Catalog', ('pot', 'po'), ('*.pot', '*.po'), ('application/x-gettext', 'text/x-gettext', 'text/gettext')), @@ -123,8 +126,8 @@ LEXERS = { 'GroovyLexer': ('pygments.lexers.jvm', 'Groovy', ('groovy',), ('*.groovy',), ('text/x-groovy',)), 'HamlLexer': ('pygments.lexers.web', 'Haml', ('haml', 'HAML'), ('*.haml',), ('text/x-haml',)), 'HaskellLexer': ('pygments.lexers.functional', 'Haskell', ('haskell', 'hs'), ('*.hs',), ('text/x-haskell',)), - 'HaxeLexer': ('pygments.lexers.web', 'haXe', ('hx', 'haXe'), ('*.hx',), ('text/haxe',)), - 'HtmlDjangoLexer': ('pygments.lexers.templates', 'HTML+Django/Jinja', ('html+django', 'html+jinja'), (), ('text/html+django', 'text/html+jinja')), + 'HaxeLexer': ('pygments.lexers.web', 'Haxe', ('hx', 'Haxe', 'haxe', 'haXe', 'hxsl'), ('*.hx', '*.hxsl'), ('text/haxe', 'text/x-haxe', 'text/x-hx')), + 'HtmlDjangoLexer': ('pygments.lexers.templates', 'HTML+Django/Jinja', ('html+django', 'html+jinja', 'htmldjango'), (), ('text/html+django', 'text/html+jinja')), 'HtmlGenshiLexer': ('pygments.lexers.templates', 'HTML+Genshi', ('html+genshi', 'html+kid'), (), ('text/html+genshi',)), 'HtmlLexer': ('pygments.lexers.web', 'HTML', ('html',), ('*.html', '*.htm', '*.xhtml', '*.xslt'), ('text/html', 'application/xhtml+xml')), 'HtmlPhpLexer': ('pygments.lexers.templates', 'HTML+PHP', ('html+php',), ('*.phtml',), ('application/x-php', 'application/x-httpd-php', 'application/x-httpd-php3', 'application/x-httpd-php4', 'application/x-httpd-php5')), @@ -133,7 +136,8 @@ LEXERS = { 'HxmlLexer': ('pygments.lexers.text', 'Hxml', ('haxeml', 'hxml'), ('*.hxml',), ()), 'HybrisLexer': ('pygments.lexers.other', 'Hybris', ('hybris', 'hy'), ('*.hy', '*.hyb'), ('text/x-hybris', 'application/x-hybris')), 'IDLLexer': ('pygments.lexers.math', 'IDL', ('idl',), ('*.pro',), ('text/idl',)), - 'IniLexer': ('pygments.lexers.text', 'INI', ('ini', 'cfg'), ('*.ini', '*.cfg'), ('text/x-ini',)), + 'IgorLexer': ('pygments.lexers.math', 'Igor', ('igor', 'igorpro'), ('*.ipf',), ('text/ipf',)), + 'IniLexer': ('pygments.lexers.text', 'INI', ('ini', 'cfg', 'dosini'), ('*.ini', '*.cfg'), ('text/x-ini',)), 'IoLexer': ('pygments.lexers.agile', 'Io', ('io',), ('*.io',), ('text/x-iosrc',)), 'IokeLexer': ('pygments.lexers.jvm', 'Ioke', ('ioke', 'ik'), ('*.ik',), ('text/x-iokesrc',)), 'IrcLogsLexer': ('pygments.lexers.text', 'IRC logs', ('irc',), ('*.weechatlog',), ('text/x-irclog',)), @@ -159,13 +163,13 @@ LEXERS = { 'LassoLexer': ('pygments.lexers.web', 'Lasso', ('lasso', 'lassoscript'), ('*.lasso', '*.lasso[89]'), ('text/x-lasso',)), 'LassoXmlLexer': ('pygments.lexers.templates', 'XML+Lasso', ('xml+lasso',), (), ('application/xml+lasso',)), 'LighttpdConfLexer': ('pygments.lexers.text', 'Lighttpd configuration file', ('lighty', 'lighttpd'), (), ('text/x-lighttpd-conf',)), - 'LiterateHaskellLexer': ('pygments.lexers.functional', 'Literate Haskell', ('lhs', 'literate-haskell'), ('*.lhs',), ('text/x-literate-haskell',)), + 'LiterateHaskellLexer': ('pygments.lexers.functional', 'Literate Haskell', ('lhs', 'literate-haskell', 'lhaskell'), ('*.lhs',), ('text/x-literate-haskell',)), 'LiveScriptLexer': ('pygments.lexers.web', 'LiveScript', ('live-script', 'livescript'), ('*.ls',), ('text/livescript',)), 'LlvmLexer': ('pygments.lexers.asm', 'LLVM', ('llvm',), ('*.ll',), ('text/x-llvm',)), 'LogosLexer': ('pygments.lexers.compiled', 'Logos', ('logos',), ('*.x', '*.xi', '*.xm', '*.xmi'), ('text/x-logos',)), 'LogtalkLexer': ('pygments.lexers.other', 'Logtalk', ('logtalk',), ('*.lgt',), ('text/x-logtalk',)), 'LuaLexer': ('pygments.lexers.agile', 'Lua', ('lua',), ('*.lua', '*.wlua'), ('text/x-lua', 'application/x-lua')), - 'MOOCodeLexer': ('pygments.lexers.other', 'MOOCode', ('moocode',), ('*.moo',), ('text/x-moocode',)), + 'MOOCodeLexer': ('pygments.lexers.other', 'MOOCode', ('moocode', 'moo'), ('*.moo',), ('text/x-moocode',)), 'MakefileLexer': ('pygments.lexers.text', 'Makefile', ('make', 'makefile', 'mf', 'bsdmake'), ('*.mak', 'Makefile', 'makefile', 'Makefile.*', 'GNUmakefile'), ('text/x-makefile',)), 'MakoCssLexer': ('pygments.lexers.templates', 'CSS+Mako', ('css+mako',), (), ('text/css+mako',)), 'MakoHtmlLexer': ('pygments.lexers.templates', 'HTML+Mako', ('html+mako',), (), ('text/html+mako',)), @@ -194,6 +198,7 @@ LEXERS = { 'NSISLexer': ('pygments.lexers.other', 'NSIS', ('nsis', 'nsi', 'nsh'), ('*.nsi', '*.nsh'), ('text/x-nsis',)), 'NasmLexer': ('pygments.lexers.asm', 'NASM', ('nasm',), ('*.asm', '*.ASM'), ('text/x-nasm',)), 'NemerleLexer': ('pygments.lexers.dotnet', 'Nemerle', ('nemerle',), ('*.n',), ('text/x-nemerle',)), + 'NesCLexer': ('pygments.lexers.compiled', 'nesC', ('nesc',), ('*.nc',), ('text/x-nescsrc',)), 'NewLispLexer': ('pygments.lexers.functional', 'NewLisp', ('newlisp',), ('*.lsp', '*.nl'), ('text/x-newlisp', 'application/x-newlisp')), 'NewspeakLexer': ('pygments.lexers.other', 'Newspeak', ('newspeak',), ('*.ns2',), ('text/x-newspeak',)), 'NginxConfLexer': ('pygments.lexers.text', 'Nginx configuration file', ('nginx',), (), ('text/x-nginx-conf',)), @@ -208,17 +213,18 @@ LEXERS = { 'OocLexer': ('pygments.lexers.compiled', 'Ooc', ('ooc',), ('*.ooc',), ('text/x-ooc',)), 'OpaLexer': ('pygments.lexers.functional', 'Opa', ('opa',), ('*.opa',), ('text/x-opa',)), 'OpenEdgeLexer': ('pygments.lexers.other', 'OpenEdge ABL', ('openedge', 'abl', 'progress'), ('*.p', '*.cls'), ('text/x-openedge', 'application/x-openedge')), + 'Perl6Lexer': ('pygments.lexers.agile', 'Perl6', ('perl6', 'pl6'), ('*.pl', '*.pm', '*.nqp', '*.p6', '*.6pl', '*.p6l', '*.pl6', '*.6pm', '*.p6m', '*.pm6'), ('text/x-perl6', 'application/x-perl6')), 'PerlLexer': ('pygments.lexers.agile', 'Perl', ('perl', 'pl'), ('*.pl', '*.pm'), ('text/x-perl', 'application/x-perl')), 'PhpLexer': ('pygments.lexers.web', 'PHP', ('php', 'php3', 'php4', 'php5'), ('*.php', '*.php[345]', '*.inc'), ('text/x-php',)), 'PlPgsqlLexer': ('pygments.lexers.sql', 'PL/pgSQL', ('plpgsql',), (), ('text/x-plpgsql',)), - 'PostScriptLexer': ('pygments.lexers.other', 'PostScript', ('postscript',), ('*.ps', '*.eps'), ('application/postscript',)), + 'PostScriptLexer': ('pygments.lexers.other', 'PostScript', ('postscript', 'postscr'), ('*.ps', '*.eps'), ('application/postscript',)), 'PostgresConsoleLexer': ('pygments.lexers.sql', 'PostgreSQL console (psql)', ('psql', 'postgresql-console', 'postgres-console'), (), ('text/x-postgresql-psql',)), 'PostgresLexer': ('pygments.lexers.sql', 'PostgreSQL SQL dialect', ('postgresql', 'postgres'), (), ('text/x-postgresql',)), 'PovrayLexer': ('pygments.lexers.other', 'POVRay', ('pov',), ('*.pov', '*.inc'), ('text/x-povray',)), - 'PowerShellLexer': ('pygments.lexers.shell', 'PowerShell', ('powershell', 'posh', 'ps1'), ('*.ps1',), ('text/x-powershell',)), + 'PowerShellLexer': ('pygments.lexers.shell', 'PowerShell', ('powershell', 'posh', 'ps1', 'psm1'), ('*.ps1', '*.psm1'), ('text/x-powershell',)), 'PrologLexer': ('pygments.lexers.compiled', 'Prolog', ('prolog',), ('*.prolog', '*.pro', '*.pl'), ('text/x-prolog',)), - 'PropertiesLexer': ('pygments.lexers.text', 'Properties', ('properties',), ('*.properties',), ('text/x-java-properties',)), - 'ProtoBufLexer': ('pygments.lexers.other', 'Protocol Buffer', ('protobuf',), ('*.proto',), ()), + 'PropertiesLexer': ('pygments.lexers.text', 'Properties', ('properties', 'jproperties'), ('*.properties',), ('text/x-java-properties',)), + 'ProtoBufLexer': ('pygments.lexers.other', 'Protocol Buffer', ('protobuf', 'proto'), ('*.proto',), ()), 'PuppetLexer': ('pygments.lexers.other', 'Puppet', ('puppet',), ('*.pp',), ()), 'PyPyLogLexer': ('pygments.lexers.text', 'PyPy Log', ('pypylog', 'pypy'), ('*.pypylog',), ('application/x-pypylog',)), 'Python3Lexer': ('pygments.lexers.agile', 'Python 3', ('python3', 'py3'), (), ('text/x-python3', 'application/x-python3')), @@ -243,6 +249,7 @@ LEXERS = { 'RebolLexer': ('pygments.lexers.other', 'REBOL', ('rebol',), ('*.r', '*.r3'), ('text/x-rebol',)), 'RedcodeLexer': ('pygments.lexers.other', 'Redcode', ('redcode',), ('*.cw',), ()), 'RegeditLexer': ('pygments.lexers.text', 'reg', ('registry',), ('*.reg',), ('text/x-windows-registry',)), + 'RexxLexer': ('pygments.lexers.other', 'Rexx', ('rexx', 'ARexx', 'arexx'), ('*.rexx', '*.rex', '*.rx', '*.arexx'), ('text/x-rexx',)), 'RhtmlLexer': ('pygments.lexers.templates', 'RHTML', ('rhtml', 'html+erb', 'html+ruby'), ('*.rhtml',), ('text/html+ruby',)), 'RobotFrameworkLexer': ('pygments.lexers.other', 'RobotFramework', ('RobotFramework', 'robotframework'), ('*.txt', '*.robot'), ('text/x-robotframework',)), 'RstLexer': ('pygments.lexers.text', 'reStructuredText', ('rst', 'rest', 'restructuredtext'), ('*.rst', '*.rest'), ('text/x-rst', 'text/prs.fallenstein.rst')), @@ -259,16 +266,17 @@ LEXERS = { 'ScssLexer': ('pygments.lexers.web', 'SCSS', ('scss',), ('*.scss',), ('text/x-scss',)), 'ShellSessionLexer': ('pygments.lexers.shell', 'Shell Session', ('shell-session',), ('*.shell-session',), ('application/x-sh-session',)), 'SmaliLexer': ('pygments.lexers.dalvik', 'Smali', ('smali',), ('*.smali',), ('text/smali',)), - 'SmalltalkLexer': ('pygments.lexers.other', 'Smalltalk', ('smalltalk', 'squeak'), ('*.st',), ('text/x-smalltalk',)), + 'SmalltalkLexer': ('pygments.lexers.other', 'Smalltalk', ('smalltalk', 'squeak', 'st'), ('*.st',), ('text/x-smalltalk',)), 'SmartyLexer': ('pygments.lexers.templates', 'Smarty', ('smarty',), ('*.tpl',), ('application/x-smarty',)), 'SnobolLexer': ('pygments.lexers.other', 'Snobol', ('snobol',), ('*.snobol',), ('text/x-snobol',)), 'SourcePawnLexer': ('pygments.lexers.other', 'SourcePawn', ('sp',), ('*.sp',), ('text/x-sourcepawn',)), - 'SourcesListLexer': ('pygments.lexers.text', 'Debian Sourcelist', ('sourceslist', 'sources.list'), ('sources.list',), ()), + 'SourcesListLexer': ('pygments.lexers.text', 'Debian Sourcelist', ('sourceslist', 'sources.list', 'debsources'), ('sources.list',), ()), 'SqlLexer': ('pygments.lexers.sql', 'SQL', ('sql',), ('*.sql',), ('text/x-sql',)), 'SqliteConsoleLexer': ('pygments.lexers.sql', 'sqlite3con', ('sqlite3',), ('*.sqlite3-console',), ('text/x-sqlite3-console',)), 'SquidConfLexer': ('pygments.lexers.text', 'SquidConf', ('squidconf', 'squid.conf', 'squid'), ('squid.conf',), ('text/x-squidconf',)), 'SspLexer': ('pygments.lexers.templates', 'Scalate Server Page', ('ssp',), ('*.ssp',), ('application/x-ssp',)), 'StanLexer': ('pygments.lexers.math', 'Stan', ('stan',), ('*.stan',), ()), + 'SwigLexer': ('pygments.lexers.compiled', 'SWIG', ('Swig', 'swig'), ('*.swg', '*.i'), ('text/swig',)), 'SystemVerilogLexer': ('pygments.lexers.hdl', 'systemverilog', ('systemverilog', 'sv'), ('*.sv', '*.svh'), ('text/x-systemverilog',)), 'TclLexer': ('pygments.lexers.agile', 'Tcl', ('tcl',), ('*.tcl',), ('text/x-tcl', 'text/x-script.tcl', 'application/x-tcl')), 'TcshLexer': ('pygments.lexers.shell', 'Tcsh', ('tcsh', 'csh'), ('*.tcsh', '*.csh'), ('application/x-csh',)), diff --git a/pygments/lexers/_robotframeworklexer.py b/pygments/lexers/_robotframeworklexer.py index 0192d289..bc64e12b 100644 --- a/pygments/lexers/_robotframeworklexer.py +++ b/pygments/lexers/_robotframeworklexer.py @@ -163,7 +163,7 @@ class RowSplitter(object): def split(self, row): splitter = (row.startswith('| ') and self._split_from_pipes or self._split_from_spaces) - for value in splitter(row.rstrip()): + for value in splitter(row): yield value yield '\n' diff --git a/pygments/lexers/_stan_builtins.py b/pygments/lexers/_stan_builtins.py index 69d8ce75..637072e4 100644 --- a/pygments/lexers/_stan_builtins.py +++ b/pygments/lexers/_stan_builtins.py @@ -1,27 +1,31 @@ # -*- coding: utf-8 -*- """ - pygments.lexers._stan_builtins - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +pygments.lexers._stan_builtins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - This file contains the names of functions for Stan used by - ``pygments.lexers.math.StanLexer. +This file contains the names of functions for Stan used by +``pygments.lexers.math.StanLexer. - :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. - :license: BSD, see LICENSE for details. +:copyright: Copyright 2013 by the Pygments team, see AUTHORS. +:license: BSD, see LICENSE for details. """ -CONSTANTS=[ 'e', - 'epsilon', - 'log10', - 'log2', - 'negative_epsilon', - 'negative_infinity', - 'not_a_number', - 'pi', - 'positive_infinity', - 'sqrt2'] +KEYWORDS = ['else', 'for', 'if', 'in', 'lower', 'lp__', 'print', 'upper', 'while'] + +TYPES = [ 'corr_matrix', + 'cov_matrix', + 'int', + 'matrix', + 'ordered', + 'positive_ordered', + 'real', + 'row_vector', + 'simplex', + 'unit_vector', + 'vector'] -FUNCTIONS=[ 'Phi', +FUNCTIONS = [ 'Phi', + 'Phi_approx', 'abs', 'acos', 'acosh', @@ -30,37 +34,66 @@ FUNCTIONS=[ 'Phi', 'atan', 'atan2', 'atanh', + 'bernoulli_cdf', 'bernoulli_log', + 'bernoulli_logit_log', + 'bernoulli_rng', + 'beta_binomial_cdf', 'beta_binomial_log', + 'beta_binomial_rng', + 'beta_cdf', 'beta_log', + 'beta_rng', 'binary_log_loss', + 'binomial_cdf', 'binomial_coefficient_log', + 'binomial_log', + 'binomial_logit_log', + 'binomial_rng', + 'block', 'categorical_log', + 'categorical_rng', + 'cauchy_cdf', 'cauchy_log', + 'cauchy_rng', 'cbrt', 'ceil', 'chi_square_log', + 'chi_square_rng', 'cholesky_decompose', 'col', 'cols', 'cos', 'cosh', + 'crossprod', + 'cumulative_sum', 'determinant', 'diag_matrix', + 'diag_post_multiply', + 'diag_pre_multiply', 'diagonal', + 'dims', 'dirichlet_log', + 'dirichlet_rng', 'dot_product', 'dot_self', 'double_exponential_log', - 'eigenvalues', + 'double_exponential_rng', + 'e', 'eigenvalues_sym', + 'eigenvectors_sym', + 'epsilon', 'erf', 'erfc', 'exp', 'exp2', + 'exp_mod_normal_cdf', + 'exp_mod_normal_log', + 'exp_mod_normal_rng', 'expm1', 'exponential_cdf', 'exponential_log', + 'exponential_rng', 'fabs', 'fdim', 'floor', @@ -69,85 +102,148 @@ FUNCTIONS=[ 'Phi', 'fmin', 'fmod', 'gamma_log', + 'gamma_rng', + 'gumbel_cdf', + 'gumbel_log', + 'gumbel_rng', 'hypergeometric_log', + 'hypergeometric_rng', 'hypot', 'if_else', 'int_step', + 'inv_chi_square_cdf', 'inv_chi_square_log', + 'inv_chi_square_rng', 'inv_cloglog', + 'inv_gamma_cdf', 'inv_gamma_log', + 'inv_gamma_rng', 'inv_logit', 'inv_wishart_log', + 'inv_wishart_rng', 'inverse', 'lbeta', 'lgamma', 'lkj_corr_cholesky_log', + 'lkj_corr_cholesky_rng', 'lkj_corr_log', + 'lkj_corr_rng', 'lkj_cov_log', 'lmgamma', 'log', 'log10', 'log1m', + 'log1m_inv_logit', 'log1p', 'log1p_exp', 'log2', + 'log_determinant', + 'log_inv_logit', 'log_sum_exp', + 'logistic_cdf', 'logistic_log', + 'logistic_rng', 'logit', 'lognormal_cdf', 'lognormal_log', + 'lognormal_rng', 'max', + 'mdivide_left_tri_low', + 'mdivide_right_tri_low', 'mean', 'min', 'multi_normal_cholesky_log', 'multi_normal_log', + 'multi_normal_prec_log', + 'multi_normal_rng', 'multi_student_t_log', + 'multi_student_t_rng', + 'multinomial_cdf', 'multinomial_log', + 'multinomial_rng', 'multiply_log', 'multiply_lower_tri_self_transpose', + 'neg_binomial_cdf', 'neg_binomial_log', + 'neg_binomial_rng', + 'negative_epsilon', + 'negative_infinity', 'normal_cdf', 'normal_log', + 'normal_rng', + 'not_a_number', 'ordered_logistic_log', + 'ordered_logistic_rng', + 'owens_t', + 'pareto_cdf', 'pareto_log', + 'pareto_rng', + 'pi', + 'poisson_cdf', 'poisson_log', + 'poisson_log_log', + 'poisson_rng', + 'positive_infinity', 'pow', 'prod', + 'rep_array', + 'rep_matrix', + 'rep_row_vector', + 'rep_vector', 'round', 'row', 'rows', + 'scaled_inv_chi_square_cdf', 'scaled_inv_chi_square_log', + 'scaled_inv_chi_square_rng', 'sd', 'sin', 'singular_values', 'sinh', + 'size', + 'skew_normal_cdf', + 'skew_normal_log', + 'skew_normal_rng', 'softmax', 'sqrt', + 'sqrt2', 'square', 'step', + 'student_t_cdf', 'student_t_log', + 'student_t_rng', 'sum', 'tan', 'tanh', + 'tcrossprod', 'tgamma', 'trace', 'trunc', 'uniform_log', + 'uniform_rng', 'variance', 'weibull_cdf', 'weibull_log', - 'wishart_log'] + 'weibull_rng', + 'wishart_log', + 'wishart_rng'] -DISTRIBUTIONS=[ 'bernoulli', +DISTRIBUTIONS = [ 'bernoulli', + 'bernoulli_logit', 'beta', 'beta_binomial', + 'binomial', + 'binomial_coefficient', + 'binomial_logit', 'categorical', 'cauchy', 'chi_square', 'dirichlet', 'double_exponential', + 'exp_mod_normal', 'exponential', 'gamma', + 'gumbel', 'hypergeometric', 'inv_chi_square', 'inv_gamma', @@ -159,16 +255,106 @@ DISTRIBUTIONS=[ 'bernoulli', 'lognormal', 'multi_normal', 'multi_normal_cholesky', + 'multi_normal_prec', 'multi_student_t', 'multinomial', + 'multiply', 'neg_binomial', 'normal', 'ordered_logistic', 'pareto', 'poisson', + 'poisson_log', 'scaled_inv_chi_square', + 'skew_normal', 'student_t', 'uniform', 'weibull', 'wishart'] +RESERVED = [ 'alignas', + 'alignof', + 'and', + 'and_eq', + 'asm', + 'auto', + 'bitand', + 'bitor', + 'bool', + 'break', + 'case', + 'catch', + 'char', + 'char16_t', + 'char32_t', + 'class', + 'compl', + 'const', + 'const_cast', + 'constexpr', + 'continue', + 'decltype', + 'default', + 'delete', + 'do', + 'double', + 'dynamic_cast', + 'enum', + 'explicit', + 'export', + 'extern', + 'false', + 'false', + 'float', + 'friend', + 'goto', + 'inline', + 'int', + 'long', + 'mutable', + 'namespace', + 'new', + 'noexcept', + 'not', + 'not_eq', + 'nullptr', + 'operator', + 'or', + 'or_eq', + 'private', + 'protected', + 'public', + 'register', + 'reinterpret_cast', + 'repeat', + 'return', + 'short', + 'signed', + 'sizeof', + 'static', + 'static_assert', + 'static_cast', + 'struct', + 'switch', + 'template', + 'then', + 'this', + 'thread_local', + 'throw', + 'true', + 'true', + 'try', + 'typedef', + 'typeid', + 'typename', + 'union', + 'unsigned', + 'until', + 'using', + 'virtual', + 'void', + 'volatile', + 'wchar_t', + 'xor', + 'xor_eq'] + diff --git a/pygments/lexers/agile.py b/pygments/lexers/agile.py index 8bcb1d46..896a3deb 100644 --- a/pygments/lexers/agile.py +++ b/pygments/lexers/agile.py @@ -12,7 +12,7 @@ import re from pygments.lexer import Lexer, RegexLexer, ExtendedRegexLexer, \ - LexerContext, include, combined, do_insertions, bygroups, using + LexerContext, include, combined, do_insertions, bygroups, using, this from pygments.token import Error, Text, Other, \ Comment, Operator, Keyword, Name, String, Number, Generic, Punctuation from pygments.util import get_bool_opt, get_list_opt, shebang_matches @@ -23,7 +23,7 @@ __all__ = ['PythonLexer', 'PythonConsoleLexer', 'PythonTracebackLexer', 'Python3Lexer', 'Python3TracebackLexer', 'RubyLexer', 'RubyConsoleLexer', 'PerlLexer', 'LuaLexer', 'MoonScriptLexer', 'CrocLexer', 'MiniDLexer', 'IoLexer', 'TclLexer', 'FactorLexer', - 'FancyLexer', 'DgLexer'] + 'FancyLexer', 'DgLexer', 'Perl6Lexer'] # b/w compatibility from pygments.lexers.functional import SchemeLexer @@ -234,7 +234,14 @@ class Python3Lexer(RegexLexer): r'TypeError|UnboundLocalError|UnicodeDecodeError|' r'UnicodeEncodeError|UnicodeError|UnicodeTranslateError|' r'UnicodeWarning|UserWarning|ValueError|VMSError|Warning|' - r'WindowsError|ZeroDivisionError)\b', Name.Exception), + r'WindowsError|ZeroDivisionError|' + # new builtin exceptions from PEP 3151 + r'BlockingIOError|ChildProcessError|ConnectionError|' + r'BrokenPipeError|ConnectionAbortedError|ConnectionRefusedError|' + r'ConnectionResetError|FileExistsError|FileNotFoundError|' + r'InterruptedError|IsADirectoryError|NotADirectoryError|' + r'PermissionError|ProcessLookupError|TimeoutError)\b', + Name.Exception), ] tokens['numbers'] = [ (r'(\d+\.\d*|\d*\.\d+)([eE][+-]?[0-9]+)?', Number.Float), @@ -421,10 +428,13 @@ class Python3TracebackLexer(RegexLexer): r'exception occurred:\n\n', Generic.Traceback), (r'^The above exception was the direct cause of the ' r'following exception:\n\n', Generic.Traceback), + (r'^(?= File "[^"]+", line \d+)', Generic.Traceback, 'intb'), ], 'intb': [ (r'^( File )("[^"]+")(, line )(\d+)(, in )(.+)(\n)', bygroups(Text, Name.Builtin, Text, Number, Text, Name, Text)), + (r'^( File )("[^"]+")(, line )(\d+)(\n)', + bygroups(Text, Name.Builtin, Text, Number, Text)), (r'^( )(.+)(\n)', bygroups(Text, using(Python3Lexer), Text)), (r'^([ \t]*)(\.\.\.)(\n)', @@ -521,7 +531,7 @@ class RubyLexer(ExtendedRegexLexer): (r":'(\\\\|\\'|[^'])*'", String.Symbol), (r"'(\\\\|\\'|[^'])*'", String.Single), (r':"', String.Symbol, 'simple-sym'), - (r'([a-zA-Z_][a-zA-Z0-9]*)(:)', + (r'([a-zA-Z_][a-zA-Z0-9]*)(:)(?!:)', bygroups(String.Symbol, Punctuation)), # Since Ruby 1.9 (r'"', String.Double, 'simple-string'), (r'(?<!\.)`', String.Backtick, 'simple-backtick'), @@ -1915,3 +1925,365 @@ class DgLexer(RegexLexer): (r"'''", String, '#pop') ], } + +class Perl6Lexer(ExtendedRegexLexer): + """ + For `Perl 6 <http://www.perl6.org>`_ source code. + + *New in Pygments 1.7.* + """ + + name = 'Perl6' + aliases = ['perl6', 'pl6'] + filenames = ['*.pl', '*.pm', '*.nqp', '*.p6', '*.6pl', '*.p6l', '*.pl6', + '*.6pm', '*.p6m', '*.pm6'] + mimetypes = ['text/x-perl6', 'application/x-perl6'] + flags = re.MULTILINE | re.DOTALL | re.UNICODE + + PERL6_IDENTIFIER_RANGE = "['a-zA-Z0-9_:-]" + + PERL6_KEYWORDS = ( + 'BEGIN', 'CATCH', 'CHECK', 'CONTROL', 'END', 'ENTER', 'FIRST', 'INIT', + 'KEEP', 'LAST', 'LEAVE', 'NEXT', 'POST', 'PRE', 'START', 'TEMP', + 'UNDO', 'as', 'assoc', 'async', 'augment', 'binary', 'break', 'but', + 'cached', 'category', 'class', 'constant', 'contend', 'continue', + 'copy', 'deep', 'default', 'defequiv', 'defer', 'die', 'do', 'else', + 'elsif', 'enum', 'equiv', 'exit', 'export', 'fail', 'fatal', 'for', + 'gather', 'given', 'goto', 'grammar', 'handles', 'has', 'if', 'inline', + 'irs', 'is', 'last', 'leave', 'let', 'lift', 'loop', 'looser', 'macro', + 'make', 'maybe', 'method', 'module', 'multi', 'my', 'next', 'of', + 'ofs', 'only', 'oo', 'ors', 'our', 'package', 'parsed', 'prec', + 'proto', 'readonly', 'redo', 'ref', 'regex', 'reparsed', 'repeat', + 'require', 'required', 'return', 'returns', 'role', 'rule', 'rw', + 'self', 'slang', 'state', 'sub', 'submethod', 'subset', 'supersede', + 'take', 'temp', 'tighter', 'token', 'trusts', 'try', 'unary', + 'unless', 'until', 'use', 'warn', 'when', 'where', 'while', 'will', + ) + + PERL6_BUILTINS = ( + 'ACCEPTS', 'HOW', 'REJECTS', 'VAR', 'WHAT', 'WHENCE', 'WHERE', 'WHICH', + 'WHO', 'abs', 'acos', 'acosec', 'acosech', 'acosh', 'acotan', 'acotanh', + 'all', 'any', 'approx', 'arity', 'asec', 'asech', 'asin', 'asinh' + 'assuming', 'atan', 'atan2', 'atanh', 'attr', 'bless', 'body', 'by' + 'bytes', 'caller', 'callsame', 'callwith', 'can', 'capitalize', 'cat', + 'ceiling', 'chars', 'chmod', 'chomp', 'chop', 'chr', 'chroot', + 'circumfix', 'cis', 'classify', 'clone', 'close', 'cmp_ok', 'codes', + 'comb', 'connect', 'contains', 'context', 'cos', 'cosec', 'cosech', + 'cosh', 'cotan', 'cotanh', 'count', 'defined', 'delete', 'diag', + 'dies_ok', 'does', 'e', 'each', 'eager', 'elems', 'end', 'eof', 'eval', + 'eval_dies_ok', 'eval_elsewhere', 'eval_lives_ok', 'evalfile', 'exists', + 'exp', 'first', 'flip', 'floor', 'flunk', 'flush', 'fmt', 'force_todo', + 'fork', 'from', 'getc', 'gethost', 'getlogin', 'getpeername', 'getpw', + 'gmtime', 'graphs', 'grep', 'hints', 'hyper', 'im', 'index', 'infix', + 'invert', 'is_approx', 'is_deeply', 'isa', 'isa_ok', 'isnt', 'iterator', + 'join', 'key', 'keys', 'kill', 'kv', 'lastcall', 'lazy', 'lc', 'lcfirst', + 'like', 'lines', 'link', 'lives_ok', 'localtime', 'log', 'log10', 'map', + 'max', 'min', 'minmax', 'name', 'new', 'nextsame', 'nextwith', 'nfc', + 'nfd', 'nfkc', 'nfkd', 'nok_error', 'nonce', 'none', 'normalize', 'not', + 'nothing', 'ok', 'once', 'one', 'open', 'opendir', 'operator', 'ord', + 'p5chomp', 'p5chop', 'pack', 'pair', 'pairs', 'pass', 'perl', 'pi', + 'pick', 'plan', 'plan_ok', 'polar', 'pop', 'pos', 'postcircumfix', + 'postfix', 'pred', 'prefix', 'print', 'printf', 'push', 'quasi', + 'quotemeta', 'rand', 're', 'read', 'readdir', 'readline', 'reduce', + 'reverse', 'rewind', 'rewinddir', 'rindex', 'roots', 'round', + 'roundrobin', 'run', 'runinstead', 'sameaccent', 'samecase', 'say', + 'sec', 'sech', 'sech', 'seek', 'shape', 'shift', 'sign', 'signature', + 'sin', 'sinh', 'skip', 'skip_rest', 'sleep', 'slurp', 'sort', 'splice', + 'split', 'sprintf', 'sqrt', 'srand', 'strand', 'subst', 'substr', 'succ', + 'sum', 'symlink', 'tan', 'tanh', 'throws_ok', 'time', 'times', 'to', + 'todo', 'trim', 'trim_end', 'trim_start', 'true', 'truncate', 'uc', + 'ucfirst', 'undef', 'undefine', 'uniq', 'unlike', 'unlink', 'unpack', + 'unpolar', 'unshift', 'unwrap', 'use_ok', 'value', 'values', 'vec', + 'version_lt', 'void', 'wait', 'want', 'wrap', 'write', 'zip', + ) + + PERL6_BUILTIN_CLASSES = ( + 'Abstraction', 'Any', 'AnyChar', 'Array', 'Associative', 'Bag', 'Bit', + 'Blob', 'Block', 'Bool', 'Buf', 'Byte', 'Callable', 'Capture', 'Char', 'Class', + 'Code', 'Codepoint', 'Comparator', 'Complex', 'Decreasing', 'Exception', + 'Failure', 'False', 'Grammar', 'Grapheme', 'Hash', 'IO', 'Increasing', + 'Int', 'Junction', 'KeyBag', 'KeyExtractor', 'KeyHash', 'KeySet', + 'KitchenSink', 'List', 'Macro', 'Mapping', 'Match', 'Matcher', 'Method', + 'Module', 'Num', 'Object', 'Ordered', 'Ordering', 'OrderingPair', + 'Package', 'Pair', 'Positional', 'Proxy', 'Range', 'Rat', 'Regex', + 'Role', 'Routine', 'Scalar', 'Seq', 'Set', 'Signature', 'Str', 'StrLen', + 'StrPos', 'Sub', 'Submethod', 'True', 'UInt', 'Undef', 'Version', 'Void', + 'Whatever', 'bit', 'bool', 'buf', 'buf1', 'buf16', 'buf2', 'buf32', + 'buf4', 'buf64', 'buf8', 'complex', 'int', 'int1', 'int16', 'int2', + 'int32', 'int4', 'int64', 'int8', 'num', 'rat', 'rat1', 'rat16', 'rat2', + 'rat32', 'rat4', 'rat64', 'rat8', 'uint', 'uint1', 'uint16', 'uint2', + 'uint32', 'uint4', 'uint64', 'uint8', 'utf16', 'utf32', 'utf8', + ) + + PERL6_OPERATORS = ( + 'X', 'Z', 'after', 'also', 'and', 'andthen', 'before', 'cmp', 'div', + 'eq', 'eqv', 'extra', 'ff', 'fff', 'ge', 'gt', 'le', 'leg', 'lt', 'm', + 'mm', 'mod', 'ne', 'or', 'orelse', 'rx', 's', 'tr', 'x', 'xor', 'xx', + '++', '--', '**', '!', '+', '-', '~', '?', '|', '||', '+^', '~^', '?^', + '^', '*', '/', '%', '%%', '+&', '+<', '+>', '~&', '~<', '~>', '?&', + 'gcd', 'lcm', '+', '-', '+|', '+^', '~|', '~^', '?|', '?^', + '~', '&', '^', 'but', 'does', '<=>', '..', '..^', '^..', '^..^', + '!=', '==', '<', '<=', '>', '>=', '~~', '===', '!eqv', + '&&', '||', '^^', '//', 'min', 'max', '??', '!!', 'ff', 'fff', 'so', + 'not', '<==', '==>', '<<==', '==>>', + ) + + # Perl 6 has a *lot* of possible bracketing characters + # this list was lifted from STD.pm6 (https://github.com/perl6/std) + PERL6_BRACKETS = { + u'\u0028' : u'\u0029', u'\u003c' : u'\u003e', u'\u005b' : u'\u005d', u'\u007b' : u'\u007d', + u'\u00ab' : u'\u00bb', u'\u0f3a' : u'\u0f3b', u'\u0f3c' : u'\u0f3d', u'\u169b' : u'\u169c', + u'\u2018' : u'\u2019', u'\u201a' : u'\u2019', u'\u201b' : u'\u2019', u'\u201c' : u'\u201d', + u'\u201e' : u'\u201d', u'\u201f' : u'\u201d', u'\u2039' : u'\u203a', u'\u2045' : u'\u2046', + u'\u207d' : u'\u207e', u'\u208d' : u'\u208e', u'\u2208' : u'\u220b', u'\u2209' : u'\u220c', + u'\u220a' : u'\u220d', u'\u2215' : u'\u29f5', u'\u223c' : u'\u223d', u'\u2243' : u'\u22cd', + u'\u2252' : u'\u2253', u'\u2254' : u'\u2255', u'\u2264' : u'\u2265', u'\u2266' : u'\u2267', + u'\u2268' : u'\u2269', u'\u226a' : u'\u226b', u'\u226e' : u'\u226f', u'\u2270' : u'\u2271', + u'\u2272' : u'\u2273', u'\u2274' : u'\u2275', u'\u2276' : u'\u2277', u'\u2278' : u'\u2279', + u'\u227a' : u'\u227b', u'\u227c' : u'\u227d', u'\u227e' : u'\u227f', u'\u2280' : u'\u2281', + u'\u2282' : u'\u2283', u'\u2284' : u'\u2285', u'\u2286' : u'\u2287', u'\u2288' : u'\u2289', + u'\u228a' : u'\u228b', u'\u228f' : u'\u2290', u'\u2291' : u'\u2292', u'\u2298' : u'\u29b8', + u'\u22a2' : u'\u22a3', u'\u22a6' : u'\u2ade', u'\u22a8' : u'\u2ae4', u'\u22a9' : u'\u2ae3', + u'\u22ab' : u'\u2ae5', u'\u22b0' : u'\u22b1', u'\u22b2' : u'\u22b3', u'\u22b4' : u'\u22b5', + u'\u22b6' : u'\u22b7', u'\u22c9' : u'\u22ca', u'\u22cb' : u'\u22cc', u'\u22d0' : u'\u22d1', + u'\u22d6' : u'\u22d7', u'\u22d8' : u'\u22d9', u'\u22da' : u'\u22db', u'\u22dc' : u'\u22dd', + u'\u22de' : u'\u22df', u'\u22e0' : u'\u22e1', u'\u22e2' : u'\u22e3', u'\u22e4' : u'\u22e5', + u'\u22e6' : u'\u22e7', u'\u22e8' : u'\u22e9', u'\u22ea' : u'\u22eb', u'\u22ec' : u'\u22ed', + u'\u22f0' : u'\u22f1', u'\u22f2' : u'\u22fa', u'\u22f3' : u'\u22fb', u'\u22f4' : u'\u22fc', + u'\u22f6' : u'\u22fd', u'\u22f7' : u'\u22fe', u'\u2308' : u'\u2309', u'\u230a' : u'\u230b', + u'\u2329' : u'\u232a', u'\u23b4' : u'\u23b5', u'\u2768' : u'\u2769', u'\u276a' : u'\u276b', + u'\u276c' : u'\u276d', u'\u276e' : u'\u276f', u'\u2770' : u'\u2771', u'\u2772' : u'\u2773', + u'\u2774' : u'\u2775', u'\u27c3' : u'\u27c4', u'\u27c5' : u'\u27c6', u'\u27d5' : u'\u27d6', + u'\u27dd' : u'\u27de', u'\u27e2' : u'\u27e3', u'\u27e4' : u'\u27e5', u'\u27e6' : u'\u27e7', + u'\u27e8' : u'\u27e9', u'\u27ea' : u'\u27eb', u'\u2983' : u'\u2984', u'\u2985' : u'\u2986', + u'\u2987' : u'\u2988', u'\u2989' : u'\u298a', u'\u298b' : u'\u298c', u'\u298d' : u'\u298e', + u'\u298f' : u'\u2990', u'\u2991' : u'\u2992', u'\u2993' : u'\u2994', u'\u2995' : u'\u2996', + u'\u2997' : u'\u2998', u'\u29c0' : u'\u29c1', u'\u29c4' : u'\u29c5', u'\u29cf' : u'\u29d0', + u'\u29d1' : u'\u29d2', u'\u29d4' : u'\u29d5', u'\u29d8' : u'\u29d9', u'\u29da' : u'\u29db', + u'\u29f8' : u'\u29f9', u'\u29fc' : u'\u29fd', u'\u2a2b' : u'\u2a2c', u'\u2a2d' : u'\u2a2e', + u'\u2a34' : u'\u2a35', u'\u2a3c' : u'\u2a3d', u'\u2a64' : u'\u2a65', u'\u2a79' : u'\u2a7a', + u'\u2a7d' : u'\u2a7e', u'\u2a7f' : u'\u2a80', u'\u2a81' : u'\u2a82', u'\u2a83' : u'\u2a84', + u'\u2a8b' : u'\u2a8c', u'\u2a91' : u'\u2a92', u'\u2a93' : u'\u2a94', u'\u2a95' : u'\u2a96', + u'\u2a97' : u'\u2a98', u'\u2a99' : u'\u2a9a', u'\u2a9b' : u'\u2a9c', u'\u2aa1' : u'\u2aa2', + u'\u2aa6' : u'\u2aa7', u'\u2aa8' : u'\u2aa9', u'\u2aaa' : u'\u2aab', u'\u2aac' : u'\u2aad', + u'\u2aaf' : u'\u2ab0', u'\u2ab3' : u'\u2ab4', u'\u2abb' : u'\u2abc', u'\u2abd' : u'\u2abe', + u'\u2abf' : u'\u2ac0', u'\u2ac1' : u'\u2ac2', u'\u2ac3' : u'\u2ac4', u'\u2ac5' : u'\u2ac6', + u'\u2acd' : u'\u2ace', u'\u2acf' : u'\u2ad0', u'\u2ad1' : u'\u2ad2', u'\u2ad3' : u'\u2ad4', + u'\u2ad5' : u'\u2ad6', u'\u2aec' : u'\u2aed', u'\u2af7' : u'\u2af8', u'\u2af9' : u'\u2afa', + u'\u2e02' : u'\u2e03', u'\u2e04' : u'\u2e05', u'\u2e09' : u'\u2e0a', u'\u2e0c' : u'\u2e0d', + u'\u2e1c' : u'\u2e1d', u'\u2e20' : u'\u2e21', u'\u3008' : u'\u3009', u'\u300a' : u'\u300b', + u'\u300c' : u'\u300d', u'\u300e' : u'\u300f', u'\u3010' : u'\u3011', u'\u3014' : u'\u3015', + u'\u3016' : u'\u3017', u'\u3018' : u'\u3019', u'\u301a' : u'\u301b', u'\u301d' : u'\u301e', + u'\ufd3e' : u'\ufd3f', u'\ufe17' : u'\ufe18', u'\ufe35' : u'\ufe36', u'\ufe37' : u'\ufe38', + u'\ufe39' : u'\ufe3a', u'\ufe3b' : u'\ufe3c', u'\ufe3d' : u'\ufe3e', u'\ufe3f' : u'\ufe40', + u'\ufe41' : u'\ufe42', u'\ufe43' : u'\ufe44', u'\ufe47' : u'\ufe48', u'\ufe59' : u'\ufe5a', + u'\ufe5b' : u'\ufe5c', u'\ufe5d' : u'\ufe5e', u'\uff08' : u'\uff09', u'\uff1c' : u'\uff1e', + u'\uff3b' : u'\uff3d', u'\uff5b' : u'\uff5d', u'\uff5f' : u'\uff60', u'\uff62' : u'\uff63', + } + + def _build_word_match(words, boundary_regex_fragment = None, prefix = '', suffix = ''): + if boundary_regex_fragment is None: + return r'\b(' + prefix + r'|'.join([ re.escape(x) for x in words]) + suffix + r')\b' + else: + return r'(?<!' + boundary_regex_fragment + ')' + prefix + '(' + \ + r'|'.join([ re.escape(x) for x in words]) + r')' + suffix + '(?!' + boundary_regex_fragment + ')' + + def brackets_callback(token_class): + def callback(lexer, match, context): + groups = match.groupdict() + opening_chars = groups['delimiter'] + n_chars = len(opening_chars) + adverbs = groups.get('adverbs') + + closer = Perl6Lexer.PERL6_BRACKETS.get(opening_chars[0]) + text = context.text + + if closer is None: # it's not a mirrored character, which means we + # just need to look for the next occurrence + + end_pos = text.find(opening_chars, match.start('delimiter') + n_chars) + else: # we need to look for the corresponding closing character, + # keep nesting in mind + closing_chars = closer * n_chars + nesting_level = 1 + + search_pos = match.start('delimiter') + + while nesting_level > 0: + next_open_pos = text.find(opening_chars, search_pos + n_chars) + next_close_pos = text.find(closing_chars, search_pos + n_chars) + + if next_close_pos == -1: + next_close_pos = len(text) + nesting_level = 0 + elif next_open_pos != -1 and next_open_pos < next_close_pos: + nesting_level += 1 + search_pos = next_open_pos + else: # next_close_pos < next_open_pos + nesting_level -= 1 + search_pos = next_close_pos + + end_pos = next_close_pos + + if adverbs is not None and re.search(r':to\b', adverbs): + heredoc_terminator = text[match.start('delimiter') + n_chars : end_pos] + end_heredoc = re.search(r'^\s*' + re.escape(heredoc_terminator) + r'\s*$', text[ match.end('delimiter') : ], re.MULTILINE) + + if end_heredoc: + end_pos = match.end('delimiter') + end_heredoc.end() + else: + end_pos = len(text) + + yield match.start(), token_class, text[match.start() : end_pos + n_chars] + context.pos = end_pos + n_chars + + return callback + + def opening_brace_callback(lexer, match, context): + stack = context.stack + + yield match.start(), Text, context.text[match.start() : match.end()] + context.pos = match.end() + + # if we encounter an opening brace and we're one level + # below a token state, it means we need to increment + # the nesting level for braces so we know later when + # we should return to the token rules. + if len(stack) > 2 and stack[-2] == 'token': + context.perl6_token_nesting_level += 1 + + def closing_brace_callback(lexer, match, context): + stack = context.stack + + yield match.start(), Text, context.text[match.start() : match.end()] + context.pos = match.end() + + # if we encounter a free closing brace and we're one level + # below a token state, it means we need to check the nesting + # level to see if we need to return to the token state. + if len(stack) > 2 and stack[-2] == 'token': + context.perl6_token_nesting_level -= 1 + if context.perl6_token_nesting_level == 0: + stack.pop() + + def embedded_perl6_callback(lexer, match, context): + context.perl6_token_nesting_level = 1 + yield match.start(), Text, context.text[match.start() : match.end()] + context.pos = match.end() + context.stack.append('root') + + # If you're modifying these rules, be careful if you need to process '{' or '}' characters. + # We have special logic for processing these characters (due to the fact that you can nest + # Perl 6 code in regex blocks), so if you need to process one of them, make sure you also + # process the corresponding one! + tokens = { + 'common' : [ + (r'#[`|=](?P<delimiter>(?P<first_char>[' + ''.join(PERL6_BRACKETS.keys()) + r'])(?P=first_char)*)', brackets_callback(Comment.Multiline)), + (r'#[^\n]*$', Comment.Singleline), + (r'^(\s*)=begin\s+(\w+)\b.*?^\1=end\s+\2', Comment.Multiline), + (r'^(\s*)=for.*?\n\s*?\n', Comment.Multiline), + (r'^=.*?\n\s*?\n', Comment.Multiline), + (r'(regex|token|rule)(\s*' + PERL6_IDENTIFIER_RANGE + '+:sym)', bygroups(Keyword, Name), 'token-sym-brackets'), + (r'(regex|token|rule)(?!' + PERL6_IDENTIFIER_RANGE + ')(\s*' + PERL6_IDENTIFIER_RANGE + '+)?', bygroups(Keyword, Name), 'pre-token'), + # deal with a special case in the Perl 6 grammar (role q { ... }) + (r'(role)(\s+)(q)(\s*)', bygroups(Keyword, Text, Name, Text)), + (_build_word_match(PERL6_KEYWORDS, PERL6_IDENTIFIER_RANGE), Keyword), + (_build_word_match(PERL6_BUILTIN_CLASSES, PERL6_IDENTIFIER_RANGE, suffix = '(?::[UD])?'), Name.Builtin), + (_build_word_match(PERL6_BUILTINS, PERL6_IDENTIFIER_RANGE), Name.Builtin), + # copied from PerlLexer + (r'[$@%&][.^:?=!~]?' + PERL6_IDENTIFIER_RANGE + u'+(?:<<.*?>>|<.*?>|«.*?»)*', Name.Variable), + (r'\$[!/](?:<<.*?>>|<.*?>|«.*?»)*', Name.Variable.Global), + (r'::\?\w+', Name.Variable.Global), + (r'[$@%&]\*' + PERL6_IDENTIFIER_RANGE + u'+(?:<<.*?>>|<.*?>|«.*?»)*', Name.Variable.Global), + (r'\$(?:<.*?>)+', Name.Variable), + (r'(?:q|qq|Q)[a-zA-Z]?\s*(?P<adverbs>:[\w\s:]+)?\s*(?P<delimiter>(?P<first_char>[^0-9a-zA-Z:\s])(?P=first_char)*)', brackets_callback(String)), + # copied from PerlLexer + (r'0_?[0-7]+(_[0-7]+)*', Number.Oct), + (r'0x[0-9A-Fa-f]+(_[0-9A-Fa-f]+)*', Number.Hex), + (r'0b[01]+(_[01]+)*', Number.Bin), + (r'(?i)(\d*(_\d*)*\.\d+(_\d*)*|\d+(_\d*)*\.\d+(_\d*)*)(e[+-]?\d+)?', Number.Float), + (r'(?i)\d+(_\d*)*e[+-]?\d+(_\d*)*', Number.Float), + (r'\d+(_\d+)*', Number.Integer), + (r'(?<=~~)\s*/(?:\\\\|\\/|.)*?/', String.Regex), + (r'(?<=[=(,])\s*/(?:\\\\|\\/|.)*?/', String.Regex), + (r'm\w+(?=\()', Name), + (r'(?:m|ms|rx)\s*(?P<adverbs>:[\w\s:]+)?\s*(?P<delimiter>(?P<first_char>[^0-9a-zA-Z:\s])(?P=first_char)*)', brackets_callback(String.Regex)), + (r'(?:s|ss|tr)\s*(?::[\w\s:]+)?\s*/(?:\\\\|\\/|.)*?/(?:\\\\|\\/|.)*?/', String.Regex), + (r'<[^\s=].*?\S>', String), + (_build_word_match(PERL6_OPERATORS), Operator), + (r'[0-9a-zA-Z_]' + PERL6_IDENTIFIER_RANGE + '*', Name), + (r"'(\\\\|\\[^\\]|[^'\\])*'", String), + (r'"(\\\\|\\[^\\]|[^"\\])*"', String), + ], + 'root' : [ + include('common'), + (r'\{', opening_brace_callback), + (r'\}', closing_brace_callback), + (r'.+?', Text), + ], + 'pre-token' : [ + include('common'), + (r'\{', Text, ('#pop', 'token')), + (r'.+?', Text), + ], + 'token-sym-brackets' : [ + (r'(?P<delimiter>(?P<first_char>[' + ''.join(PERL6_BRACKETS.keys()) + '])(?P=first_char)*)', brackets_callback(Name), ('#pop', 'pre-token')), + (r'', Name, ('#pop', 'pre-token')), + ], + 'token': [ + (r'}', Text, '#pop'), + (r'(?<=:)(?:my|our|state|constant|temp|let).*?;', using(this)), + # make sure that quotes in character classes aren't treated as strings + (r'<(?:[-!?+.]\s*)?\[.*?\]>', String.Regex), + # make sure that '#' characters in quotes aren't treated as comments + (r"(?<!\\)'(\\\\|\\[^\\]|[^'\\])*'", String.Regex), + (r'(?<!\\)"(\\\\|\\[^\\]|[^"\\])*"', String.Regex), + (r'#.*?$', Comment.Singleline), + (r'\{', embedded_perl6_callback), + ('.+?', String.Regex), + ], + } + + def analyse_text(text): + def strip_pod(lines): + in_pod = False + stripped_lines = [] + + for line in lines: + if re.match(r'^=(?:end|cut)', line): + in_pod = False + elif re.match(r'^=\w+', line): + in_pod = True + elif not in_pod: + stripped_lines.append(line) + + return stripped_lines + + lines = text.splitlines() + lines = strip_pod(lines) + text = '\n'.join(lines) + + if shebang_matches(text, r'perl6|rakudo|niecza'): + return True + + if 'use v6' in text: + return 0.91 # 0.01 greater than Perl says for 'my $' + if re.search(r'[$@%]\*[A-Z]+', text): # Perl 6-style globals ($*OS) + return 0.91 + if re.search(r'[$@%]\?[A-Z]+', text): # Perl 6 compiler variables ($?PACKAGE) + return 0.91 + if re.search(r'[$@%][!.][A-Za-z0-9_-]+', text): # Perl 6 member variables + return 0.91 + + for line in text.splitlines(): + if re.match(r'\s*(?:my|our)?\s*module', line): # module declarations + return 0.91 + if re.match(r'\s*(?:my|our)?\s*role', line): # role declarations + return 0.91 + if re.match(r'\s*(?:my|our)?\s*class\b', line): # class declarations + return 0.91 + return False + + def __init__(self, **options): + super(Perl6Lexer, self).__init__(**options) + self.encoding = options.get('encoding', 'utf-8') diff --git a/pygments/lexers/asm.py b/pygments/lexers/asm.py index 7ff64bcc..3f67862c 100644 --- a/pygments/lexers/asm.py +++ b/pygments/lexers/asm.py @@ -25,7 +25,7 @@ class GasLexer(RegexLexer): For Gas (AT&T) assembly code. """ name = 'GAS' - aliases = ['gas'] + aliases = ['gas', 'asm'] filenames = ['*.s', '*.S'] mimetypes = ['text/x-gas'] @@ -244,7 +244,7 @@ class LlvmLexer(RegexLexer): r'|align|addrspace|section|alias|module|asm|sideeffect|gc|dbg' r'|ccc|fastcc|coldcc|x86_stdcallcc|x86_fastcallcc|arm_apcscc' - r'|arm_aapcscc|arm_aapcs_vfpcc' + r'|arm_aapcscc|arm_aapcs_vfpcc|ptx_device|ptx_kernel' r'|cc|c' diff --git a/pygments/lexers/compiled.py b/pygments/lexers/compiled.py index bd6750d8..c3b0909d 100644 --- a/pygments/lexers/compiled.py +++ b/pygments/lexers/compiled.py @@ -23,13 +23,14 @@ from pygments.scanner import Scanner from pygments.lexers.functional import OcamlLexer from pygments.lexers.jvm import JavaLexer, ScalaLexer -__all__ = ['CLexer', 'CppLexer', 'DLexer', 'DelphiLexer', 'ECLexer', 'DylanLexer', - 'ObjectiveCLexer', 'ObjectiveCppLexer', 'FortranLexer', 'GLShaderLexer', - 'PrologLexer', 'CythonLexer', 'ValaLexer', 'OocLexer', 'GoLexer', - 'FelixLexer', 'AdaLexer', 'Modula2Lexer', 'BlitzMaxLexer', - 'NimrodLexer', 'FantomLexer', 'RustLexer', 'CudaLexer', 'MonkeyLexer', +__all__ = ['CLexer', 'CppLexer', 'DLexer', 'DelphiLexer', 'ECLexer', + 'NesCLexer', 'DylanLexer', 'ObjectiveCLexer', 'ObjectiveCppLexer', + 'FortranLexer', 'GLShaderLexer', 'PrologLexer', 'CythonLexer', + 'ValaLexer', 'OocLexer', 'GoLexer', 'FelixLexer', 'AdaLexer', + 'Modula2Lexer', 'BlitzMaxLexer', 'BlitzBasicLexer', 'NimrodLexer', + 'FantomLexer', 'RustLexer', 'CudaLexer', 'MonkeyLexer', 'SwigLexer', 'DylanLidLexer', 'DylanConsoleLexer', 'CobolLexer', - 'CobolFreeformatLexer', 'LogosLexer'] + 'CobolFreeformatLexer', 'LogosLexer', 'ClayLexer'] class CFamilyLexer(RegexLexer): @@ -231,6 +232,63 @@ class CppLexer(CFamilyLexer): return 0.1 +class SwigLexer(CppLexer): + """ + For `SWIG <http://www.swig.org/>`_ source code. + + *New in Pygments 1.7.* + """ + name = 'SWIG' + aliases = ['Swig', 'swig'] + filenames = ['*.swg', '*.i'] + mimetypes = ['text/swig'] + priority = 0.04 # Lower than C/C++ and Objective C/C++ + + tokens = { + 'statements': [ + (r'(%[a-z_][a-z0-9_]*)', Name.Function), # SWIG directives + ('\$\**\&?[a-zA-Z0-9_]+', Name), # Special variables + (r'##*[a-zA-Z_][a-zA-Z0-9_]*', Comment.Preproc), # Stringification / additional preprocessor directives + inherit, + ], + } + + # This is a far from complete set of SWIG directives + swig_directives = ( + # Most common directives + '%apply', '%define', '%director', '%enddef', '%exception', '%extend', + '%feature', '%fragment', '%ignore', '%immutable', '%import', '%include', + '%inline', '%insert', '%module', '%newobject', '%nspace', '%pragma', + '%rename', '%shared_ptr', '%template', '%typecheck', '%typemap', + # Less common directives + '%arg', '%attribute', '%bang', '%begin', '%callback', '%catches', '%clear', + '%constant', '%copyctor', '%csconst', '%csconstvalue', '%csenum', + '%csmethodmodifiers', '%csnothrowexception', '%default', '%defaultctor', + '%defaultdtor', '%defined', '%delete', '%delobject', '%descriptor', + '%exceptionclass', '%exceptionvar', '%extend_smart_pointer', '%fragments', + '%header', '%ifcplusplus', '%ignorewarn', '%implicit', '%implicitconv', + '%init', '%javaconst', '%javaconstvalue', '%javaenum', '%javaexception', + '%javamethodmodifiers', '%kwargs', '%luacode', '%mutable', '%naturalvar', + '%nestedworkaround', '%perlcode', '%pythonabc', '%pythonappend', + '%pythoncallback', '%pythoncode', '%pythondynamic', '%pythonmaybecall', + '%pythonnondynamic', '%pythonprepend', '%refobject', '%shadow', '%sizeof', + '%trackobjects', '%types', '%unrefobject', '%varargs', '%warn', '%warnfilter') + + def analyse_text(text): + rv = 0.1 # Same as C/C++ + # Search for SWIG directives, which are conventionally at the beginning of + # a line. The probability of them being within a line is low, so let another + # lexer win in this case. + matches = re.findall(r'^\s*(%[a-z_][a-z0-9_]*)', text, re.M) + for m in matches: + if m in SwigLexer.swig_directives: + rv = 0.98 + break + else: + rv = 0.91 # Fraction higher than MatlabLexer + return rv + + class ECLexer(CLexer): """ For eC source code with preprocessor directives. @@ -266,6 +324,83 @@ class ECLexer(CLexer): } +class NesCLexer(CLexer): + """ + For `nesC <https://github.com/tinyos/nesc>`_ source code with preprocessor + directives. + + *New in Pygments 1.7.* + """ + name = 'nesC' + aliases = ['nesc'] + filenames = ['*.nc'] + mimetypes = ['text/x-nescsrc'] + + tokens = { + 'statements': [ + (r'(abstract|as|async|atomic|call|command|component|components|' + r'configuration|event|extends|generic|implementation|includes|' + r'interface|module|new|norace|post|provides|signal|task|uses)\b', + Keyword), + (r'(nx_struct|nx_union|nx_int8_t|nx_int16_t|nx_int32_t|nx_int64_t|' + r'nx_uint8_t|nx_uint16_t|nx_uint32_t|nx_uint64_t)\b', + Keyword.Type), + inherit, + ], + } + + +class ClayLexer(RegexLexer): + """ + For `Clay <http://claylabs.com/clay/>`_ source. + + *New in Pygments 1.7.* + """ + name = 'Clay' + filenames = ['*.clay'] + aliases = ['clay'] + mimetypes = ['text/x-clay'] + tokens = { + 'root': [ + (r'\s', Text), + (r'//.*?$', Comment.Singleline), + (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline), + (r'\b(public|private|import|as|record|variant|instance' + r'|define|overload|default|external|alias' + r'|rvalue|ref|forward|inline|noinline|forceinline' + r'|enum|var|and|or|not|if|else|goto|return|while' + r'|switch|case|break|continue|for|in|true|false|try|catch|throw' + r'|finally|onerror|staticassert|eval|when|newtype' + r'|__FILE__|__LINE__|__COLUMN__|__ARG__' + r')\b', Keyword), + (r'[~!%^&*+=|:<>/-]', Operator), + (r'[#(){}\[\],;.]', Punctuation), + (r'0x[0-9a-fA-F]+[LlUu]*', Number.Hex), + (r'\d+[LlUu]*', Number.Integer), + (r'\b(true|false)\b', Name.Builtin), + (r'(?i)[a-z_?][a-z_?0-9]*', Name), + (r'"""', String, 'tdqs'), + (r'"', String, 'dqs'), + ], + 'strings': [ + (r'(?i)\\(x[0-9a-f]{2}|.)', String.Escape), + (r'.', String), + ], + 'nl': [ + (r'\n', String), + ], + 'dqs': [ + (r'"', String, '#pop'), + include('strings'), + ], + 'tdqs': [ + (r'"""', String, '#pop'), + include('strings'), + include('nl'), + ], + } + + class DLexer(RegexLexer): """ For D source. @@ -1216,7 +1351,8 @@ def objective(baselexer): ('#pop', 'oc_classname')), (r'(@class|@protocol)(\s+)', bygroups(Keyword, Text), ('#pop', 'oc_forward_classname')), - (r'@[\[{(]', Punctuation), + # @ can also prefix other expressions like @{...} or @(...) + (r'@', Punctuation), inherit, ], 'oc_classname' : [ @@ -1523,7 +1659,7 @@ class CythonLexer(RegexLexer): """ name = 'Cython' - aliases = ['cython', 'pyx'] + aliases = ['cython', 'pyx', 'pyrex'] filenames = ['*.pyx', '*.pxd', '*.pxi'] mimetypes = ['text/x-cython', 'application/x-cython'] @@ -2582,6 +2718,88 @@ class BlitzMaxLexer(RegexLexer): } +class BlitzBasicLexer(RegexLexer): + """ + For `BlitzBasic <http://blitzbasic.com>`_ source code. + + *New in Pygments 1.7.* + """ + + name = 'BlitzBasic' + aliases = ['blitzbasic', 'b3d', 'bplus'] + filenames = ['*.bb', '*.decls'] + mimetypes = ['text/x-bb'] + + bb_vopwords = (r'\b(Shl|Shr|Sar|Mod|Or|And|Not|' + r'Abs|Sgn|Handle|Int|Float|Str|' + r'First|Last|Before|After)\b') + bb_sktypes = r'@{1,2}|[#$%]' + bb_name = r'[a-z][a-z0-9_]*' + bb_var = (r'(%s)(?:([ \t]*)(%s)|([ \t]*)([.])([ \t]*)(?:(%s)))?') % \ + (bb_name, bb_sktypes, bb_name) + + flags = re.MULTILINE | re.IGNORECASE + tokens = { + 'root': [ + # Text + (r'[ \t]+', Text), + # Comments + (r";.*?\n", Comment.Single), + # Data types + ('"', String.Double, 'string'), + # Numbers + (r'[0-9]+\.[0-9]*(?!\.)', Number.Float), + (r'\.[0-9]+(?!\.)', Number.Float), + (r'[0-9]+', Number.Integer), + (r'\$[0-9a-f]+', Number.Hex), + (r'\%[10]+', Number), # Binary + # Other + (r'(?:%s|([+\-*/~=<>^]))' % (bb_vopwords), Operator), + (r'[(),:\[\]\\]', Punctuation), + (r'\.([ \t]*)(%s)' % bb_name, Name.Label), + # Identifiers + (r'\b(New)\b([ \t]+)(%s)' % (bb_name), + bygroups(Keyword.Reserved, Text, Name.Class)), + (r'\b(Gosub|Goto)\b([ \t]+)(%s)' % (bb_name), + bygroups(Keyword.Reserved, Text, Name.Label)), + (r'\b(Object)\b([ \t]*)([.])([ \t]*)(%s)\b' % (bb_name), + bygroups(Operator, Text, Punctuation, Text, Name.Class)), + (r'\b%s\b([ \t]*)(\()' % bb_var, + bygroups(Name.Function, Text, Keyword.Type,Text, Punctuation, + Text, Name.Class, Text, Punctuation)), + (r'\b(Function)\b([ \t]+)%s' % bb_var, + bygroups(Keyword.Reserved, Text, Name.Function, Text, Keyword.Type, + Text, Punctuation, Text, Name.Class)), + (r'\b(Type)([ \t]+)(%s)' % (bb_name), + bygroups(Keyword.Reserved, Text, Name.Class)), + # Keywords + (r'\b(Pi|True|False|Null)\b', Keyword.Constant), + (r'\b(Local|Global|Const|Field|Dim)\b', Keyword.Declaration), + (r'\b(End|Return|Exit|' + r'Chr|Len|Asc|' + r'New|Delete|Insert|' + r'Include|' + r'Function|' + r'Type|' + r'If|Then|Else|ElseIf|EndIf|' + r'For|To|Next|Step|Each|' + r'While|Wend|' + r'Repeat|Until|Forever|' + r'Select|Case|Default|' + r'Goto|Gosub|Data|Read|Restore)\b', Keyword.Reserved), + # Final resolve (for variable names and such) +# (r'(%s)' % (bb_name), Name.Variable), + (bb_var, bygroups(Name.Variable, Text, Keyword.Type, + Text, Punctuation, Text, Name.Class)), + ], + 'string': [ + (r'""', String.Double), + (r'"C?', String.Double, '#pop'), + (r'[^"]+', String.Double), + ], + } + + class NimrodLexer(RegexLexer): """ For `Nimrod <http://nimrod-code.org/>`_ source code. diff --git a/pygments/lexers/functional.py b/pygments/lexers/functional.py index a082811b..77fe4723 100644 --- a/pygments/lexers/functional.py +++ b/pygments/lexers/functional.py @@ -719,7 +719,7 @@ class CommonLispLexer(RegexLexer): *New in Pygments 0.9.* """ name = 'Common Lisp' - aliases = ['common-lisp', 'cl'] + aliases = ['common-lisp', 'cl', 'lisp'] filenames = ['*.cl', '*.lisp', '*.el'] # use for Elisp too mimetypes = ['text/x-common-lisp'] @@ -808,6 +808,8 @@ class CommonLispLexer(RegexLexer): (r'"(\\.|\\\n|[^"\\])*"', String), # quoting (r":" + symbol, String.Symbol), + (r"::" + symbol, String.Symbol), + (r":#" + symbol, String.Symbol), (r"'" + symbol, String.Symbol), (r"'", Operator), (r"`", Operator), @@ -1026,7 +1028,7 @@ class LiterateHaskellLexer(Lexer): *New in Pygments 0.9.* """ name = 'Literate Haskell' - aliases = ['lhs', 'literate-haskell'] + aliases = ['lhs', 'literate-haskell', 'lhaskell'] filenames = ['*.lhs'] mimetypes = ['text/x-literate-haskell'] @@ -1663,9 +1665,10 @@ class OpaLexer(RegexLexer): # but if you color only real keywords, you might just # as well not color anything keywords = [ - 'and', 'as', 'begin', 'css', 'database', 'db', 'do', 'else', 'end', - 'external', 'forall', 'if', 'import', 'match', 'package', 'parser', - 'rec', 'server', 'then', 'type', 'val', 'with', 'xml_parser', + 'and', 'as', 'begin', 'case', 'client', 'css', 'database', 'db', 'do', + 'else', 'end', 'external', 'forall', 'function', 'if', 'import', + 'match', 'module', 'or', 'package', 'parser', 'rec', 'server', 'then', + 'type', 'val', 'with', 'xml_parser', ] # matches both stuff and `stuff` @@ -2399,7 +2402,7 @@ class ElixirConsoleLexer(Lexer): class KokaLexer(RegexLexer): """ - Lexer for the `Koka <http://research.microsoft.com/en-us/projects/koka/>`_ + Lexer for the `Koka <http://koka.codeplex.com>`_ language. *New in Pygments 1.6.* @@ -2411,7 +2414,7 @@ class KokaLexer(RegexLexer): mimetypes = ['text/x-koka'] keywords = [ - 'infix', 'infixr', 'infixl', 'prefix', 'postfix', + 'infix', 'infixr', 'infixl', 'type', 'cotype', 'rectype', 'alias', 'struct', 'con', 'fun', 'function', 'val', 'var', @@ -2450,7 +2453,12 @@ class KokaLexer(RegexLexer): sboundary = '(?!'+symbols+')' # name boundary: a keyword should not be followed by any of these - boundary = '(?![a-zA-Z0-9_\\-])' + boundary = '(?![\w/])' + + # koka token abstractions + tokenType = Name.Attribute + tokenTypeDef = Name.Class + tokenConstructor = Generic.Emph # main lexer tokens = { @@ -2458,41 +2466,51 @@ class KokaLexer(RegexLexer): include('whitespace'), # go into type mode - (r'::?' + sboundary, Keyword.Type, 'type'), - (r'alias' + boundary, Keyword, 'alias-type'), - (r'struct' + boundary, Keyword, 'struct-type'), - (r'(%s)' % '|'.join(typeStartKeywords) + boundary, Keyword, 'type'), + (r'::?' + sboundary, tokenType, 'type'), + (r'(alias)(\s+)([a-z]\w*)?', bygroups(Keyword, Text, tokenTypeDef), + 'alias-type'), + (r'(struct)(\s+)([a-z]\w*)?', bygroups(Keyword, Text, tokenTypeDef), + 'struct-type'), + ((r'(%s)' % '|'.join(typeStartKeywords)) + + r'(\s+)([a-z]\w*)?', bygroups(Keyword, Text, tokenTypeDef), + 'type'), # special sequences of tokens (we use ?: for non-capturing group as # required by 'bygroups') - (r'(module)(\s*)((?:interface)?)(\s*)' - r'((?:[a-z](?:[a-zA-Z0-9_]|\-[a-zA-Z])*\.)*' - r'[a-z](?:[a-zA-Z0-9_]|\-[a-zA-Z])*)', - bygroups(Keyword, Text, Keyword, Text, Name.Namespace)), - (r'(import)(\s+)((?:[a-z](?:[a-zA-Z0-9_]|\-[a-zA-Z])*\.)*[a-z]' - r'(?:[a-zA-Z0-9_]|\-[a-zA-Z])*)(\s*)((?:as)?)' - r'((?:[A-Z](?:[a-zA-Z0-9_]|\-[a-zA-Z])*)?)', - bygroups(Keyword, Text, Name.Namespace, Text, Keyword, - Name.Namespace)), + (r'(module)(\s+)(interface\s+)?((?:[a-z]\w*/)*[a-z]\w*)', + bygroups(Keyword, Text, Keyword, Name.Namespace)), + (r'(import)(\s+)((?:[a-z]\w*/)*[a-z]\w*)' + r'(?:(\s*)(=)(\s*)((?:qualified\s*)?)' + r'((?:[a-z]\w*/)*[a-z]\w*))?', + bygroups(Keyword, Text, Name.Namespace, Text, Keyword, Text, + Keyword, Name.Namespace)), + + (r'(^(?:(?:public|private)\s*)?(?:function|fun|val))' + r'(\s+)([a-z]\w*|\((?:' + symbols + r'|/)\))', + bygroups(Keyword, Text, Name.Function)), + (r'(^(?:(?:public|private)\s*)?external)(\s+)(inline\s+)?' + r'([a-z]\w*|\((?:' + symbols + r'|/)\))', + bygroups(Keyword, Text, Keyword, Name.Function)), # keywords (r'(%s)' % '|'.join(typekeywords) + boundary, Keyword.Type), (r'(%s)' % '|'.join(keywords) + boundary, Keyword), (r'(%s)' % '|'.join(builtin) + boundary, Keyword.Pseudo), - (r'::|:=|\->|[=\.:]' + sboundary, Keyword), - (r'\-' + sboundary, Generic.Strong), + (r'::?|:=|\->|[=\.]' + sboundary, Keyword), # names - (r'[A-Z]([a-zA-Z0-9_]|\-[a-zA-Z])*(?=\.)', Name.Namespace), - (r'[A-Z]([a-zA-Z0-9_]|\-[a-zA-Z])*(?!\.)', Name.Class), - (r'[a-z]([a-zA-Z0-9_]|\-[a-zA-Z])*', Name), - (r'_([a-zA-Z0-9_]|\-[a-zA-Z])*', Name.Variable), + (r'((?:[a-z]\w*/)*)([A-Z]\w*)', + bygroups(Name.Namespace, tokenConstructor)), + (r'((?:[a-z]\w*/)*)([a-z]\w*)', bygroups(Name.Namespace, Name)), + (r'((?:[a-z]\w*/)*)(\((?:' + symbols + r'|/)\))', + bygroups(Name.Namespace, Name)), + (r'_\w*', Name.Variable), # literal string (r'@"', String.Double, 'litstring'), # operators - (symbols, Operator), + (symbols + "|/(?![\*/])", Operator), (r'`', Operator), (r'[\{\}\(\)\[\];,]', Punctuation), @@ -2519,17 +2537,17 @@ class KokaLexer(RegexLexer): # type started by colon 'type': [ - (r'[\(\[<]', Keyword.Type, 'type-nested'), + (r'[\(\[<]', tokenType, 'type-nested'), include('type-content') ], # type nested in brackets: can contain parameters, comma etc. 'type-nested': [ - (r'[\)\]>]', Keyword.Type, '#pop'), - (r'[\(\[<]', Keyword.Type, 'type-nested'), - (r',', Keyword.Type), - (r'([a-z](?:[a-zA-Z0-9_]|\-[a-zA-Z])*)(\s*)(:)(?!:)', - bygroups(Name.Variable,Text,Keyword.Type)), # parameter name + (r'[\)\]>]', tokenType, '#pop'), + (r'[\(\[<]', tokenType, 'type-nested'), + (r',', tokenType), + (r'([a-z]\w*)(\s*)(:)(?!:)', + bygroups(Name, Text, tokenType)), # parameter name include('type-content') ], @@ -2538,23 +2556,23 @@ class KokaLexer(RegexLexer): include('whitespace'), # keywords - (r'(%s)' % '|'.join(typekeywords) + boundary, Keyword.Type), + (r'(%s)' % '|'.join(typekeywords) + boundary, Keyword), (r'(?=((%s)' % '|'.join(keywords) + boundary + '))', Keyword, '#pop'), # need to match because names overlap... # kinds - (r'[EPH]' + boundary, Keyword.Type), - (r'[*!]', Keyword.Type), + (r'[EPHVX]' + boundary, tokenType), # type names - (r'[A-Z]([a-zA-Z0-9_]|\-[a-zA-Z])*(?=\.)', Name.Namespace), - (r'[A-Z]([a-zA-Z0-9_]|\-[a-zA-Z])*(?!\.)', Name.Class), - (r'[a-z][0-9]*(?![a-zA-Z_\-])', Keyword.Type), # Generic.Emph - (r'_([a-zA-Z0-9_]|\-[a-zA-Z])*', Keyword.Type), # Generic.Emph - (r'[a-z]([a-zA-Z0-9_]|\-[a-zA-Z])*', Keyword.Type), + (r'[a-z][0-9]*(?![\w/])', tokenType ), + (r'_\w*', tokenType.Variable), # Generic.Emph + (r'((?:[a-z]\w*/)*)([A-Z]\w*)', + bygroups(Name.Namespace, tokenType)), + (r'((?:[a-z]\w*/)*)([a-z]\w+)', + bygroups(Name.Namespace, tokenType)), # type keyword operators - (r'::|\->|[\.:|]', Keyword.Type), + (r'::|\->|[\.:|]', tokenType), #catchall (r'', Text, '#pop') @@ -2562,6 +2580,7 @@ class KokaLexer(RegexLexer): # comments and literals 'whitespace': [ + (r'\n\s*#.*$', Comment.Preproc), (r'\s+', Text), (r'/\*', Comment.Multiline, 'comment'), (r'//.*$', Comment.Single) @@ -2588,11 +2607,10 @@ class KokaLexer(RegexLexer): (r'[\'\n]', String.Char, '#pop'), ], 'escape-sequence': [ - (r'\\[abfnrtv0\\\"\'\?]', String.Escape), + (r'\\[nrt\\\"\']', String.Escape), (r'\\x[0-9a-fA-F]{2}', String.Escape), (r'\\u[0-9a-fA-F]{4}', String.Escape), # Yes, \U literals are 6 hex digits. (r'\\U[0-9a-fA-F]{6}', String.Escape) ] } - diff --git a/pygments/lexers/jvm.py b/pygments/lexers/jvm.py index 717621e9..ed4d257c 100644 --- a/pygments/lexers/jvm.py +++ b/pygments/lexers/jvm.py @@ -888,11 +888,11 @@ class CeylonLexer(RegexLexer): (r'[^\S\n]+', Text), (r'//.*?\n', Comment.Single), (r'/\*.*?\*/', Comment.Multiline), - (r'(variable|shared|abstract|doc|by|formal|actual)', + (r'(variable|shared|abstract|doc|by|formal|actual|late|native)', Name.Decorator), (r'(break|case|catch|continue|default|else|finally|for|in|' - r'variable|if|return|switch|this|throw|try|while|is|exists|' - r'nonempty|then|outer)\b', Keyword), + r'variable|if|return|switch|this|throw|try|while|is|exists|dynamic|' + r'nonempty|then|outer|assert)\b', Keyword), (r'(abstracts|extends|satisfies|adapts|' r'super|given|of|out|assign|' r'transient|volatile)\b', Keyword.Declaration), @@ -900,16 +900,16 @@ class CeylonLexer(RegexLexer): Keyword.Type), (r'(package)(\s+)', bygroups(Keyword.Namespace, Text)), (r'(true|false|null)\b', Keyword.Constant), - (r'(class|interface|object)(\s+)', + (r'(class|interface|object|alias)(\s+)', bygroups(Keyword.Declaration, Text), 'class'), (r'(import)(\s+)', bygroups(Keyword.Namespace, Text), 'import'), (r'"(\\\\|\\"|[^"])*"', String), - (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Quoted), - (r"`\\.`|`[^\\]`|`\\u[0-9a-fA-F]{4}`", String.Char), - (r'(\.)([a-zA-Z_][a-zA-Z0-9_]*)', + (r"'\\.'|'[^\\]'|'\\\{#[0-9a-fA-F]{4}\}'", String.Char), + (r'".*``.*``.*"', String.Interpol), + (r'(\.)([a-z_][a-zA-Z0-9_]*)', bygroups(Operator, Name.Attribute)), (r'[a-zA-Z_][a-zA-Z0-9_]*:', Name.Label), - (r'[a-zA-Z_\$][a-zA-Z0-9_]*', Name), + (r'[a-zA-Z_][a-zA-Z0-9_]*', Name), (r'[~\^\*!%&\[\]\(\)\{\}<>\|+=:;,./?-]', Operator), (r'\d{1,3}(_\d{3})+\.\d{1,3}(_\d{3})+[kMGTPmunpf]?', Number.Float), (r'\d{1,3}(_\d{3})+\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', @@ -917,16 +917,19 @@ class CeylonLexer(RegexLexer): (r'[0-9][0-9]*\.\d{1,3}(_\d{3})+[kMGTPmunpf]?', Number.Float), (r'[0-9][0-9]*\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?', Number.Float), - (r'0x[0-9a-fA-F]+', Number.Hex), + (r'#([0-9a-fA-F]{4})(_[0-9a-fA-F]{4})+', Number.Hex), + (r'#[0-9a-fA-F]+', Number.Hex), + (r'\$([01]{4})(_[01]{4})+', Number.Integer), + (r'\$[01]+', Number.Integer), (r'\d{1,3}(_\d{3})+[kMGTP]?', Number.Integer), (r'[0-9]+[kMGTP]?', Number.Integer), (r'\n', Text) ], 'class': [ - (r'[a-zA-Z_][a-zA-Z0-9_]*', Name.Class, '#pop') + (r'[A-Za-z_][a-zA-Z0-9_]*', Name.Class, '#pop') ], 'import': [ - (r'[a-zA-Z0-9_.]+\w+ \{([a-zA-Z,]+|\.\.\.)\}', + (r'[a-z][a-zA-Z0-9_.]*', Name.Namespace, '#pop') ], } diff --git a/pygments/lexers/math.py b/pygments/lexers/math.py index 537c6d0e..f52e85e8 100644 --- a/pygments/lexers/math.py +++ b/pygments/lexers/math.py @@ -24,7 +24,7 @@ from pygments.lexers import _stan_builtins __all__ = ['JuliaLexer', 'JuliaConsoleLexer', 'MuPADLexer', 'MatlabLexer', 'MatlabSessionLexer', 'OctaveLexer', 'ScilabLexer', 'NumPyLexer', 'RConsoleLexer', 'SLexer', 'JagsLexer', 'BugsLexer', 'StanLexer', - 'IDLLexer', 'RdLexer'] + 'IDLLexer', 'RdLexer', 'IgorLexer'] class JuliaLexer(RegexLexer): @@ -59,7 +59,7 @@ class JuliaLexer(RegexLexer): (r'(begin|while|for|in|return|break|continue|' r'macro|quote|let|if|elseif|else|try|catch|end|' r'bitstype|ccall|do|using|module|import|export|' - r'importall|baremodule)\b', Keyword), + r'importall|baremodule|immutable)\b', Keyword), (r'(local|global|const)\b', Keyword.Declaration), (r'(Bool|Int|Int8|Int16|Int32|Int64|Uint|Uint8|Uint16|Uint32|Uint64' r'|Float32|Float64|Complex64|Complex128|Any|Nothing|None)\b', @@ -99,11 +99,17 @@ class JuliaLexer(RegexLexer): (r'[a-zA-Z_][a-zA-Z0-9_]*', Name), # numbers + (r'(\d+(_\d+)+\.\d*|\d*\.\d+(_\d+)+)([eEf][+-]?[0-9]+)?', Number.Float), (r'(\d+\.\d*|\d*\.\d+)([eEf][+-]?[0-9]+)?', Number.Float), + (r'\d+(_\d+)+[eEf][+-]?[0-9]+', Number.Float), (r'\d+[eEf][+-]?[0-9]+', Number.Float), + (r'0b[01]+(_[01]+)+', Number.Binary), (r'0b[01]+', Number.Binary), + (r'0o[0-7]+(_[0-7]+)+', Number.Oct), (r'0o[0-7]+', Number.Oct), + (r'0x[a-fA-F0-9]+(_[a-fA-F0-9]+)+', Number.Hex), (r'0x[a-fA-F0-9]+', Number.Hex), + (r'\d+(_\d+)+', Number.Integer), (r'\d+', Number.Integer) ], @@ -1294,8 +1300,11 @@ class JagsLexer(RegexLexer): return 0 class StanLexer(RegexLexer): - """ - Pygments Lexer for Stan models. + """Pygments Lexer for Stan models. + + The Stan modeling language is specified in the *Stan 1.3.0 + Modeling Language Manual* `pdf + <http://code.google.com/p/stan/downloads/detail?name=stan-reference-1.3.0.pdf>`_. *New in Pygments 1.6.* """ @@ -1304,13 +1313,6 @@ class StanLexer(RegexLexer): aliases = ['stan'] filenames = ['*.stan'] - _RESERVED = ('for', 'in', 'while', 'repeat', 'until', 'if', - 'then', 'else', 'true', 'false', 'T', - 'lower', 'upper', 'print') - - _TYPES = ('int', 'real', 'vector', 'simplex', 'ordered', 'row_vector', - 'matrix', 'corr_matrix', 'cov_matrix', 'positive_ordered') - tokens = { 'whitespace' : [ (r"\s+", Text), @@ -1334,20 +1336,21 @@ class StanLexer(RegexLexer): 'model', r'generated\s+quantities')), bygroups(Keyword.Namespace, Text, Punctuation)), # Reserved Words - (r'(%s)\b' % r'|'.join(_RESERVED), Keyword.Reserved), + (r'(%s)\b' % r'|'.join(_stan_builtins.KEYWORDS), Keyword), + # Truncation + (r'T(?=\s*\[)', Keyword), # Data types - (r'(%s)\b' % r'|'.join(_TYPES), Keyword.Type), + (r'(%s)\b' % r'|'.join(_stan_builtins.TYPES), Keyword.Type), # Punctuation - (r"[;:,\[\]()<>]", Punctuation), + (r"[;:,\[\]()]", Punctuation), # Builtin (r'(%s)(?=\s*\()' % r'|'.join(_stan_builtins.FUNCTIONS + _stan_builtins.DISTRIBUTIONS), Name.Builtin), - (r'(%s)(?=\s*\()' - % r'|'.join(_stan_builtins.CONSTANTS), Keyword.Constant), # Special names ending in __, like lp__ (r'[A-Za-z][A-Za-z0-9_]*__\b', Name.Builtin.Pseudo), + (r'(%s)\b' % r'|'.join(_stan_builtins.RESERVED), Keyword.Reserved), # Regular variable names (r'[A-Za-z][A-Za-z0-9_]*\b', Name), # Real Literals @@ -1359,7 +1362,7 @@ class StanLexer(RegexLexer): # SLexer makes these tokens Operators. (r'<-|~', Operator), # Infix and prefix operators (and = ) - (r"\+|-|\.?\*|\.?/|\\|'|=", Operator), + (r"\+|-|\.?\*|\.?/|\\|'|==?|!=?|<=?|>=?|\|\||&&", Operator), # Block delimiters (r'[{}]', Punctuation), ] @@ -1650,3 +1653,260 @@ class RdLexer(RegexLexer): (r'.', Text), ] } + + +class IgorLexer(RegexLexer): + """ + Pygments Lexer for Igor Pro procedure files (.ipf). + See http://www.wavemetrics.com/ and http://www.igorexchange.com/. + + *New in Pygments 1.7.* + """ + + name = 'Igor' + aliases = ['igor', 'igorpro'] + filenames = ['*.ipf'] + mimetypes = ['text/ipf'] + + flags = re.IGNORECASE + + flowControl = [ + 'if', 'else', 'elseif', 'endif', 'for', 'endfor', 'strswitch', 'switch', + 'case', 'endswitch', 'do', 'while', 'try', 'catch', 'endtry', 'break', + 'continue', 'return', + ] + types = [ + 'variable', 'string', 'constant', 'strconstant', 'NVAR', 'SVAR', 'WAVE', + 'STRUCT', 'ThreadSafe', 'function', 'end', 'static', 'macro', 'window', + 'graph', 'Structure', 'EndStructure', 'EndMacro', 'FuncFit', 'Proc', + 'Picture', 'Menu', 'SubMenu', 'Prompt', 'DoPrompt', + ] + operations = [ + 'Abort', 'AddFIFOData', 'AddFIFOVectData', 'AddMovieAudio', + 'AddMovieFrame', 'APMath', 'Append', 'AppendImage', + 'AppendLayoutObject', 'AppendMatrixContour', 'AppendText', + 'AppendToGraph', 'AppendToLayout', 'AppendToTable', 'AppendXYZContour', + 'AutoPositionWindow', 'BackgroundInfo', 'Beep', 'BoundingBall', + 'BrowseURL', 'BuildMenu', 'Button', 'cd', 'Chart', 'CheckBox', + 'CheckDisplayed', 'ChooseColor', 'Close', 'CloseMovie', 'CloseProc', + 'ColorScale', 'ColorTab2Wave', 'Concatenate', 'ControlBar', + 'ControlInfo', 'ControlUpdate', 'ConvexHull', 'Convolve', 'CopyFile', + 'CopyFolder', 'CopyScales', 'Correlate', 'CreateAliasShortcut', 'Cross', + 'CtrlBackground', 'CtrlFIFO', 'CtrlNamedBackground', 'Cursor', + 'CurveFit', 'CustomControl', 'CWT', 'Debugger', 'DebuggerOptions', + 'DefaultFont', 'DefaultGuiControls', 'DefaultGuiFont', 'DefineGuide', + 'DelayUpdate', 'DeleteFile', 'DeleteFolder', 'DeletePoints', + 'Differentiate', 'dir', 'Display', 'DisplayHelpTopic', + 'DisplayProcedure', 'DoAlert', 'DoIgorMenu', 'DoUpdate', 'DoWindow', + 'DoXOPIdle', 'DrawAction', 'DrawArc', 'DrawBezier', 'DrawLine', + 'DrawOval', 'DrawPICT', 'DrawPoly', 'DrawRect', 'DrawRRect', 'DrawText', + 'DSPDetrend', 'DSPPeriodogram', 'Duplicate', 'DuplicateDataFolder', + 'DWT', 'EdgeStats', 'Edit', 'ErrorBars', 'Execute', 'ExecuteScriptText', + 'ExperimentModified', 'Extract', 'FastGaussTransform', 'FastOp', + 'FBinRead', 'FBinWrite', 'FFT', 'FIFO2Wave', 'FIFOStatus', 'FilterFIR', + 'FilterIIR', 'FindLevel', 'FindLevels', 'FindPeak', 'FindPointsInPoly', + 'FindRoots', 'FindSequence', 'FindValue', 'FPClustering', 'fprintf', + 'FReadLine', 'FSetPos', 'FStatus', 'FTPDelete', 'FTPDownload', + 'FTPUpload', 'FuncFit', 'FuncFitMD', 'GetAxis', 'GetFileFolderInfo', + 'GetLastUserMenuInfo', 'GetMarquee', 'GetSelection', 'GetWindow', + 'GraphNormal', 'GraphWaveDraw', 'GraphWaveEdit', 'Grep', 'GroupBox', + 'Hanning', 'HideIgorMenus', 'HideInfo', 'HideProcedures', 'HideTools', + 'HilbertTransform', 'Histogram', 'IFFT', 'ImageAnalyzeParticles', + 'ImageBlend', 'ImageBoundaryToMask', 'ImageEdgeDetection', + 'ImageFileInfo', 'ImageFilter', 'ImageFocus', 'ImageGenerateROIMask', + 'ImageHistModification', 'ImageHistogram', 'ImageInterpolate', + 'ImageLineProfile', 'ImageLoad', 'ImageMorphology', 'ImageRegistration', + 'ImageRemoveBackground', 'ImageRestore', 'ImageRotate', 'ImageSave', + 'ImageSeedFill', 'ImageSnake', 'ImageStats', 'ImageThreshold', + 'ImageTransform', 'ImageUnwrapPhase', 'ImageWindow', 'IndexSort', + 'InsertPoints', 'Integrate', 'IntegrateODE', 'Interp3DPath', + 'Interpolate3D', 'KillBackground', 'KillControl', 'KillDataFolder', + 'KillFIFO', 'KillFreeAxis', 'KillPath', 'KillPICTs', 'KillStrings', + 'KillVariables', 'KillWaves', 'KillWindow', 'KMeans', 'Label', 'Layout', + 'Legend', 'LinearFeedbackShiftRegister', 'ListBox', 'LoadData', + 'LoadPackagePreferences', 'LoadPICT', 'LoadWave', 'Loess', + 'LombPeriodogram', 'Make', 'MakeIndex', 'MarkPerfTestTime', + 'MatrixConvolve', 'MatrixCorr', 'MatrixEigenV', 'MatrixFilter', + 'MatrixGaussJ', 'MatrixInverse', 'MatrixLinearSolve', + 'MatrixLinearSolveTD', 'MatrixLLS', 'MatrixLUBkSub', 'MatrixLUD', + 'MatrixMultiply', 'MatrixOP', 'MatrixSchur', 'MatrixSolve', + 'MatrixSVBkSub', 'MatrixSVD', 'MatrixTranspose', 'MeasureStyledText', + 'Modify', 'ModifyContour', 'ModifyControl', 'ModifyControlList', + 'ModifyFreeAxis', 'ModifyGraph', 'ModifyImage', 'ModifyLayout', + 'ModifyPanel', 'ModifyTable', 'ModifyWaterfall', 'MoveDataFolder', + 'MoveFile', 'MoveFolder', 'MoveString', 'MoveSubwindow', 'MoveVariable', + 'MoveWave', 'MoveWindow', 'NeuralNetworkRun', 'NeuralNetworkTrain', + 'NewDataFolder', 'NewFIFO', 'NewFIFOChan', 'NewFreeAxis', 'NewImage', + 'NewLayout', 'NewMovie', 'NewNotebook', 'NewPanel', 'NewPath', + 'NewWaterfall', 'Note', 'Notebook', 'NotebookAction', 'Open', + 'OpenNotebook', 'Optimize', 'ParseOperationTemplate', 'PathInfo', + 'PauseForUser', 'PauseUpdate', 'PCA', 'PlayMovie', 'PlayMovieAction', + 'PlaySnd', 'PlaySound', 'PopupContextualMenu', 'PopupMenu', + 'Preferences', 'PrimeFactors', 'Print', 'printf', 'PrintGraphs', + 'PrintLayout', 'PrintNotebook', 'PrintSettings', 'PrintTable', + 'Project', 'PulseStats', 'PutScrapText', 'pwd', 'Quit', + 'RatioFromNumber', 'Redimension', 'Remove', 'RemoveContour', + 'RemoveFromGraph', 'RemoveFromLayout', 'RemoveFromTable', 'RemoveImage', + 'RemoveLayoutObjects', 'RemovePath', 'Rename', 'RenameDataFolder', + 'RenamePath', 'RenamePICT', 'RenameWindow', 'ReorderImages', + 'ReorderTraces', 'ReplaceText', 'ReplaceWave', 'Resample', + 'ResumeUpdate', 'Reverse', 'Rotate', 'Save', 'SaveData', + 'SaveExperiment', 'SaveGraphCopy', 'SaveNotebook', + 'SavePackagePreferences', 'SavePICT', 'SaveTableCopy', + 'SetActiveSubwindow', 'SetAxis', 'SetBackground', 'SetDashPattern', + 'SetDataFolder', 'SetDimLabel', 'SetDrawEnv', 'SetDrawLayer', + 'SetFileFolderInfo', 'SetFormula', 'SetIgorHook', 'SetIgorMenuMode', + 'SetIgorOption', 'SetMarquee', 'SetProcessSleep', 'SetRandomSeed', + 'SetScale', 'SetVariable', 'SetWaveLock', 'SetWindow', 'ShowIgorMenus', + 'ShowInfo', 'ShowTools', 'Silent', 'Sleep', 'Slider', 'Smooth', + 'SmoothCustom', 'Sort', 'SoundInRecord', 'SoundInSet', + 'SoundInStartChart', 'SoundInStatus', 'SoundInStopChart', + 'SphericalInterpolate', 'SphericalTriangulate', 'SplitString', + 'sprintf', 'sscanf', 'Stack', 'StackWindows', + 'StatsAngularDistanceTest', 'StatsANOVA1Test', 'StatsANOVA2NRTest', + 'StatsANOVA2RMTest', 'StatsANOVA2Test', 'StatsChiTest', + 'StatsCircularCorrelationTest', 'StatsCircularMeans', + 'StatsCircularMoments', 'StatsCircularTwoSampleTest', + 'StatsCochranTest', 'StatsContingencyTable', 'StatsDIPTest', + 'StatsDunnettTest', 'StatsFriedmanTest', 'StatsFTest', + 'StatsHodgesAjneTest', 'StatsJBTest', 'StatsKendallTauTest', + 'StatsKSTest', 'StatsKWTest', 'StatsLinearCorrelationTest', + 'StatsLinearRegression', 'StatsMultiCorrelationTest', + 'StatsNPMCTest', 'StatsNPNominalSRTest', 'StatsQuantiles', + 'StatsRankCorrelationTest', 'StatsResample', 'StatsSample', + 'StatsScheffeTest', 'StatsSignTest', 'StatsSRTest', 'StatsTTest', + 'StatsTukeyTest', 'StatsVariancesTest', 'StatsWatsonUSquaredTest', + 'StatsWatsonWilliamsTest', 'StatsWheelerWatsonTest', + 'StatsWilcoxonRankTest', 'StatsWRCorrelationTest', 'String', + 'StructGet', 'StructPut', 'TabControl', 'Tag', 'TextBox', 'Tile', + 'TileWindows', 'TitleBox', 'ToCommandLine', 'ToolsGrid', + 'Triangulate3d', 'Unwrap', 'ValDisplay', 'Variable', 'WaveMeanStdv', + 'WaveStats', 'WaveTransform', 'wfprintf', 'WignerTransform', + 'WindowFunction', + ] + functions = [ + 'abs', 'acos', 'acosh', 'AiryA', 'AiryAD', 'AiryB', 'AiryBD', 'alog', + 'area', 'areaXY', 'asin', 'asinh', 'atan', 'atan2', 'atanh', + 'AxisValFromPixel', 'Besseli', 'Besselj', 'Besselk', 'Bessely', 'bessi', + 'bessj', 'bessk', 'bessy', 'beta', 'betai', 'BinarySearch', + 'BinarySearchInterp', 'binomial', 'binomialln', 'binomialNoise', 'cabs', + 'CaptureHistoryStart', 'ceil', 'cequal', 'char2num', 'chebyshev', + 'chebyshevU', 'CheckName', 'cmplx', 'cmpstr', 'conj', 'ContourZ', 'cos', + 'cosh', 'cot', 'CountObjects', 'CountObjectsDFR', 'cpowi', + 'CreationDate', 'csc', 'DataFolderExists', 'DataFolderRefsEqual', + 'DataFolderRefStatus', 'date2secs', 'datetime', 'DateToJulian', + 'Dawson', 'DDEExecute', 'DDEInitiate', 'DDEPokeString', 'DDEPokeWave', + 'DDERequestWave', 'DDEStatus', 'DDETerminate', 'deltax', 'digamma', + 'DimDelta', 'DimOffset', 'DimSize', 'ei', 'enoise', 'equalWaves', 'erf', + 'erfc', 'exists', 'exp', 'expInt', 'expNoise', 'factorial', 'fakedata', + 'faverage', 'faverageXY', 'FindDimLabel', 'FindListItem', 'floor', + 'FontSizeHeight', 'FontSizeStringWidth', 'FresnelCos', 'FresnelSin', + 'gamma', 'gammaInc', 'gammaNoise', 'gammln', 'gammp', 'gammq', 'Gauss', + 'Gauss1D', 'Gauss2D', 'gcd', 'GetDefaultFontSize', + 'GetDefaultFontStyle', 'GetKeyState', 'GetRTError', 'gnoise', + 'GrepString', 'hcsr', 'hermite', 'hermiteGauss', 'HyperG0F1', + 'HyperG1F1', 'HyperG2F1', 'HyperGNoise', 'HyperGPFQ', 'IgorVersion', + 'ilim', 'imag', 'Inf', 'Integrate1D', 'interp', 'Interp2D', 'Interp3D', + 'inverseERF', 'inverseERFC', 'ItemsInList', 'jlim', 'Laguerre', + 'LaguerreA', 'LaguerreGauss', 'leftx', 'LegendreA', 'limit', 'ln', + 'log', 'logNormalNoise', 'lorentzianNoise', 'magsqr', 'MandelbrotPoint', + 'MarcumQ', 'MatrixDet', 'MatrixDot', 'MatrixRank', 'MatrixTrace', 'max', + 'mean', 'min', 'mod', 'ModDate', 'NaN', 'norm', 'NumberByKey', + 'numpnts', 'numtype', 'NumVarOrDefault', 'NVAR_Exists', 'p2rect', + 'ParamIsDefault', 'pcsr', 'Pi', 'PixelFromAxisVal', 'pnt2x', + 'poissonNoise', 'poly', 'poly2D', 'PolygonArea', 'qcsr', 'r2polar', + 'real', 'rightx', 'round', 'sawtooth', 'ScreenResolution', 'sec', + 'SelectNumber', 'sign', 'sin', 'sinc', 'sinh', 'SphericalBessJ', + 'SphericalBessJD', 'SphericalBessY', 'SphericalBessYD', + 'SphericalHarmonics', 'sqrt', 'StartMSTimer', 'StatsBetaCDF', + 'StatsBetaPDF', 'StatsBinomialCDF', 'StatsBinomialPDF', + 'StatsCauchyCDF', 'StatsCauchyPDF', 'StatsChiCDF', 'StatsChiPDF', + 'StatsCMSSDCDF', 'StatsCorrelation', 'StatsDExpCDF', 'StatsDExpPDF', + 'StatsErlangCDF', 'StatsErlangPDF', 'StatsErrorPDF', 'StatsEValueCDF', + 'StatsEValuePDF', 'StatsExpCDF', 'StatsExpPDF', 'StatsFCDF', + 'StatsFPDF', 'StatsFriedmanCDF', 'StatsGammaCDF', 'StatsGammaPDF', + 'StatsGeometricCDF', 'StatsGeometricPDF', 'StatsHyperGCDF', + 'StatsHyperGPDF', 'StatsInvBetaCDF', 'StatsInvBinomialCDF', + 'StatsInvCauchyCDF', 'StatsInvChiCDF', 'StatsInvCMSSDCDF', + 'StatsInvDExpCDF', 'StatsInvEValueCDF', 'StatsInvExpCDF', + 'StatsInvFCDF', 'StatsInvFriedmanCDF', 'StatsInvGammaCDF', + 'StatsInvGeometricCDF', 'StatsInvKuiperCDF', 'StatsInvLogisticCDF', + 'StatsInvLogNormalCDF', 'StatsInvMaxwellCDF', 'StatsInvMooreCDF', + 'StatsInvNBinomialCDF', 'StatsInvNCChiCDF', 'StatsInvNCFCDF', + 'StatsInvNormalCDF', 'StatsInvParetoCDF', 'StatsInvPoissonCDF', + 'StatsInvPowerCDF', 'StatsInvQCDF', 'StatsInvQpCDF', + 'StatsInvRayleighCDF', 'StatsInvRectangularCDF', 'StatsInvSpearmanCDF', + 'StatsInvStudentCDF', 'StatsInvTopDownCDF', 'StatsInvTriangularCDF', + 'StatsInvUsquaredCDF', 'StatsInvVonMisesCDF', 'StatsInvWeibullCDF', + 'StatsKuiperCDF', 'StatsLogisticCDF', 'StatsLogisticPDF', + 'StatsLogNormalCDF', 'StatsLogNormalPDF', 'StatsMaxwellCDF', + 'StatsMaxwellPDF', 'StatsMedian', 'StatsMooreCDF', 'StatsNBinomialCDF', + 'StatsNBinomialPDF', 'StatsNCChiCDF', 'StatsNCChiPDF', 'StatsNCFCDF', + 'StatsNCFPDF', 'StatsNCTCDF', 'StatsNCTPDF', 'StatsNormalCDF', + 'StatsNormalPDF', 'StatsParetoCDF', 'StatsParetoPDF', 'StatsPermute', + 'StatsPoissonCDF', 'StatsPoissonPDF', 'StatsPowerCDF', + 'StatsPowerNoise', 'StatsPowerPDF', 'StatsQCDF', 'StatsQpCDF', + 'StatsRayleighCDF', 'StatsRayleighPDF', 'StatsRectangularCDF', + 'StatsRectangularPDF', 'StatsRunsCDF', 'StatsSpearmanRhoCDF', + 'StatsStudentCDF', 'StatsStudentPDF', 'StatsTopDownCDF', + 'StatsTriangularCDF', 'StatsTriangularPDF', 'StatsTrimmedMean', + 'StatsUSquaredCDF', 'StatsVonMisesCDF', 'StatsVonMisesNoise', + 'StatsVonMisesPDF', 'StatsWaldCDF', 'StatsWaldPDF', 'StatsWeibullCDF', + 'StatsWeibullPDF', 'StopMSTimer', 'str2num', 'stringCRC', 'stringmatch', + 'strlen', 'strsearch', 'StudentA', 'StudentT', 'sum', 'SVAR_Exists', + 'TagVal', 'tan', 'tanh', 'ThreadGroupCreate', 'ThreadGroupRelease', + 'ThreadGroupWait', 'ThreadProcessorCount', 'ThreadReturnValue', 'ticks', + 'trunc', 'Variance', 'vcsr', 'WaveCRC', 'WaveDims', 'WaveExists', + 'WaveMax', 'WaveMin', 'WaveRefsEqual', 'WaveType', 'WhichListItem', + 'WinType', 'WNoise', 'x', 'x2pnt', 'xcsr', 'y', 'z', 'zcsr', 'ZernikeR', + ] + functions += [ + 'AddListItem', 'AnnotationInfo', 'AnnotationList', 'AxisInfo', + 'AxisList', 'CaptureHistory', 'ChildWindowList', 'CleanupName', + 'ContourInfo', 'ContourNameList', 'ControlNameList', 'CsrInfo', + 'CsrWave', 'CsrXWave', 'CTabList', 'DataFolderDir', 'date', + 'DDERequestString', 'FontList', 'FuncRefInfo', 'FunctionInfo', + 'FunctionList', 'FunctionPath', 'GetDataFolder', 'GetDefaultFont', + 'GetDimLabel', 'GetErrMessage', 'GetFormula', + 'GetIndependentModuleName', 'GetIndexedObjName', 'GetIndexedObjNameDFR', + 'GetRTErrMessage', 'GetRTStackInfo', 'GetScrapText', 'GetUserData', + 'GetWavesDataFolder', 'GrepList', 'GuideInfo', 'GuideNameList', 'Hash', + 'IgorInfo', 'ImageInfo', 'ImageNameList', 'IndexedDir', 'IndexedFile', + 'JulianToDate', 'LayoutInfo', 'ListMatch', 'LowerStr', 'MacroList', + 'NameOfWave', 'note', 'num2char', 'num2istr', 'num2str', + 'OperationList', 'PadString', 'ParseFilePath', 'PathList', 'PICTInfo', + 'PICTList', 'PossiblyQuoteName', 'ProcedureText', 'RemoveByKey', + 'RemoveEnding', 'RemoveFromList', 'RemoveListItem', + 'ReplaceNumberByKey', 'ReplaceString', 'ReplaceStringByKey', + 'Secs2Date', 'Secs2Time', 'SelectString', 'SortList', + 'SpecialCharacterInfo', 'SpecialCharacterList', 'SpecialDirPath', + 'StringByKey', 'StringFromList', 'StringList', 'StrVarOrDefault', + 'TableInfo', 'TextFile', 'ThreadGroupGetDF', 'time', 'TraceFromPixel', + 'TraceInfo', 'TraceNameList', 'UniqueName', 'UnPadString', 'UpperStr', + 'VariableList', 'WaveInfo', 'WaveList', 'WaveName', 'WaveUnits', + 'WinList', 'WinName', 'WinRecreation', 'XWaveName', + 'ContourNameToWaveRef', 'CsrWaveRef', 'CsrXWaveRef', + 'ImageNameToWaveRef', 'NewFreeWave', 'TagWaveRef', 'TraceNameToWaveRef', + 'WaveRefIndexed', 'XWaveRefFromTrace', 'GetDataFolderDFR', + 'GetWavesDataFolderDFR', 'NewFreeDataFolder', 'ThreadGroupGetDFR', + ] + + tokens = { + 'root': [ + (r'//.*$', Comment.Single), + (r'"([^"\\]|\\.)*"', String), + # Flow Control. + (r'\b(%s)\b' % '|'.join(flowControl), Keyword), + # Types. + (r'\b(%s)\b' % '|'.join(types), Keyword.Type), + # Built-in operations. + (r'\b(%s)\b' % '|'.join(operations), Name.Class), + # Built-in functions. + (r'\b(%s)\b' % '|'.join(functions), Name.Function), + # Compiler directives. + (r'^#(include|pragma|define|ifdef|ifndef|endif)', + Name.Decorator), + (r'[^a-zA-Z"/]+', Text), + (r'.', Text), + ], + } diff --git a/pygments/lexers/other.py b/pygments/lexers/other.py index c8557922..10598fb4 100644 --- a/pygments/lexers/other.py +++ b/pygments/lexers/other.py @@ -14,7 +14,8 @@ import re from pygments.lexer import RegexLexer, include, bygroups, using, \ this, combined, ExtendedRegexLexer from pygments.token import Error, Punctuation, Literal, Token, \ - Text, Comment, Operator, Keyword, Name, String, Number, Generic + Text, Comment, Operator, Keyword, Name, String, Number, Generic, \ + Whitespace from pygments.util import get_bool_opt from pygments.lexers.web import HtmlLexer @@ -35,7 +36,7 @@ __all__ = ['BrainfuckLexer', 'BefungeLexer', 'RedcodeLexer', 'MOOCodeLexer', 'ECLLexer', 'UrbiscriptLexer', 'OpenEdgeLexer', 'BroLexer', 'MscgenLexer', 'KconfigLexer', 'VGLLexer', 'SourcePawnLexer', 'RobotFrameworkLexer', 'PuppetLexer', 'NSISLexer', 'RPMSpecLexer', - 'CbmBasicV2Lexer', 'AutoItLexer'] + 'CbmBasicV2Lexer', 'AutoItLexer', 'RexxLexer'] class ECLLexer(RegexLexer): @@ -245,7 +246,7 @@ class MOOCodeLexer(RegexLexer): """ name = 'MOOCode' filenames = ['*.moo'] - aliases = ['moocode'] + aliases = ['moocode', 'moo'] mimetypes = ['text/x-moocode'] tokens = { @@ -289,7 +290,7 @@ class SmalltalkLexer(RegexLexer): """ name = 'Smalltalk' filenames = ['*.st'] - aliases = ['smalltalk', 'squeak'] + aliases = ['smalltalk', 'squeak', 'st'] mimetypes = ['text/x-smalltalk'] tokens = { @@ -363,7 +364,7 @@ class SmalltalkLexer(RegexLexer): include('literals'), ], 'afterobject' : [ - (r'! !$', Keyword , '#pop'), # squeak chunk delimeter + (r'! !$', Keyword , '#pop'), # squeak chunk delimiter include('whitespaces'), (r'\b(ifTrue:|ifFalse:|whileTrue:|whileFalse:|timesRepeat:)', Name.Builtin, '#pop'), @@ -1397,8 +1398,6 @@ class RebolLexer(RegexLexer): (r';.*\n', Comment), (r'%"', Name.Decorator, 'stringFile'), (r'%[^(\^{^")\s\[\]]+', Name.Decorator), - (r'<[a-zA-Z0-9:._-]*>', Name.Tag), - (r'<[^(<>\s")]+', Name.Tag, 'tag'), (r'[+-]?([a-zA-Z]{1,3})?\$\d+(\.\d+)?', Number.Float), # money (r'[+-]?\d+\:\d+(\:\d+)?(\.\d+)?', String.Other), # time (r'\d+\-[0-9a-zA-Z]+\-\d+(\/\d+\:\d+(\:\d+)?' @@ -1415,6 +1414,8 @@ class RebolLexer(RegexLexer): (r'comment\s', Comment, 'comment'), (r'/[^(\^{^")\s/[\]]*', Name.Attribute), (r'([^(\^{^")\s/[\]]+)(?=[:({"\s/\[\]])', word_callback), + (r'<[a-zA-Z0-9:._-]*>', Name.Tag), + (r'<[^(<>\s")]+', Name.Tag, 'tag'), (r'([^(\^{^")\s]+)', Text), ], 'string': [ @@ -1961,11 +1962,11 @@ class AsymptoteLexer(RegexLexer): from pygments.lexers._asybuiltins import ASYFUNCNAME, ASYVARNAME for index, token, value in \ RegexLexer.get_tokens_unprocessed(self, text): - if token is Name and value in ASYFUNCNAME: - token = Name.Function - elif token is Name and value in ASYVARNAME: - token = Name.Variable - yield index, token, value + if token is Name and value in ASYFUNCNAME: + token = Name.Function + elif token is Name and value in ASYVARNAME: + token = Name.Variable + yield index, token, value class PostScriptLexer(RegexLexer): @@ -1979,7 +1980,7 @@ class PostScriptLexer(RegexLexer): *New in Pygments 1.4.* """ name = 'PostScript' - aliases = ['postscript'] + aliases = ['postscript', 'postscr'] filenames = ['*.ps', '*.eps'] mimetypes = ['application/postscript'] @@ -2067,7 +2068,7 @@ class AutohotkeyLexer(RegexLexer): *New in Pygments 1.4.* """ name = 'autohotkey' - aliases = ['ahk'] + aliases = ['ahk', 'autohotkey'] filenames = ['*.ahk', '*.ahkl'] mimetypes = ['text/x-autohotkey'] @@ -2352,7 +2353,7 @@ class ProtoBufLexer(RegexLexer): """ name = 'Protocol Buffer' - aliases = ['protobuf'] + aliases = ['protobuf', 'proto'] filenames = ['*.proto'] tokens = { @@ -2839,8 +2840,8 @@ class BroLexer(RegexLexer): (r'\\\n', Text), # Keywords (r'(add|alarm|break|case|const|continue|delete|do|else|enum|event' - r'|export|for|function|if|global|local|module|next' - r'|of|print|redef|return|schedule|type|when|while)\b', Keyword), + r'|export|for|function|if|global|hook|local|module|next' + r'|of|print|redef|return|schedule|switch|type|when|while)\b', Keyword), (r'(addr|any|bool|count|counter|double|file|int|interval|net' r'|pattern|port|record|set|string|subnet|table|time|timer' r'|vector)\b', Keyword.Type), @@ -3306,7 +3307,7 @@ class NSISLexer(RegexLexer): tokens = { 'root': [ (r'[;\#].*\n', Comment), - (r"'.*'", String.Single), + (r"'.*?'", String.Single), (r'"', String.Double, 'str_double'), (r'`', String.Backtick, 'str_backtick'), include('macro'), @@ -3457,7 +3458,7 @@ class RPMSpecLexer(RegexLexer): include('macro'), (r'(?i)^(Name|Version|Release|Epoch|Summary|Group|License|Packager|' r'Vendor|Icon|URL|Distribution|Prefix|Patch[0-9]*|Source[0-9]*|' - r'Requires\(?[a-z]*\)?|[a-z]+Req|Obsoletes|Provides|Conflicts|' + r'Requires\(?[a-z]*\)?|[a-z]+Req|Obsoletes|Suggests|Provides|Conflicts|' r'Build[a-z]+|[a-z]+Arch|Auto[a-z]+)(:)(.*)$', bygroups(Generic.Heading, Punctuation, using(this))), (r'^%description', Name.Decorator, 'description'), @@ -3467,7 +3468,7 @@ class RPMSpecLexer(RegexLexer): r'make(?:install)|ghost|patch[0-9]+|find_lang|exclude|verify)', Keyword), include('interpol'), - (r"'.*'", String.Single), + (r"'.*?'", String.Single), (r'"', String.Double, 'string'), (r'.', Text), ], @@ -3624,7 +3625,7 @@ class AutoItLexer(RegexLexer): (r'[a-zA-Z_#@$][a-zA-Z0-9_#@$]*', Name), (r'\\|\'', Text), (r'\`([\,\%\`abfnrtv\-\+;])', String.Escape), - (r'_\n', Text), # Line continuation + (r'_\n', Text), # Line continuation include('garbage'), ], 'commands': [ @@ -3665,3 +3666,113 @@ class AutoItLexer(RegexLexer): (r'[^\S\n]', Text), ], } + + +class RexxLexer(RegexLexer): + """ + `Rexx <http://www.rexxinfo.org/>`_ is a scripting language available for + a wide range of different platforms with its roots found on mainframe + systems. It is popular for I/O- and data based tasks and can act as glue + language to bind different applications together. + + *New in Pygments 1.7.* + """ + name = 'Rexx' + aliases = ['rexx', 'ARexx', 'arexx'] + filenames = ['*.rexx', '*.rex', '*.rx', '*.arexx'] + mimetypes = ['text/x-rexx'] + flags = re.IGNORECASE + + tokens = { + 'root': [ + (r'\s', Whitespace), + (r'/\*', Comment.Multiline, 'comment'), + (r'"', String, 'string_double'), + (r"'", String, 'string_single'), + (r'[0-9]+(\.[0-9]+)?(e[+-]?[0-9])?', Number), + (r'([a-z_][a-z0-9_]*)(\s*)(:)(\s*)(procedure)\b', + bygroups(Name.Function, Whitespace, Operator, Whitespace, + Keyword.Declaration)), + (r'([a-z_][a-z0-9_]*)(\s*)(:)', + bygroups(Name.Label, Whitespace, Operator)), + include('function'), + include('keyword'), + include('operator'), + (r'[a-z_][a-z0-9_]*', Text), + ], + 'function': [ + (r'(abbrev|abs|address|arg|b2x|bitand|bitor|bitxor|c2d|c2x|' + r'center|charin|charout|chars|compare|condition|copies|d2c|' + r'd2x|datatype|date|delstr|delword|digits|errortext|form|' + r'format|fuzz|insert|lastpos|left|length|linein|lineout|lines|' + r'max|min|overlay|pos|queued|random|reverse|right|sign|' + r'sourceline|space|stream|strip|substr|subword|symbol|time|' + r'trace|translate|trunc|value|verify|word|wordindex|' + r'wordlength|wordpos|words|x2b|x2c|x2d|xrange)(\s*)(\()', + bygroups(Name.Builtin, Whitespace, Operator)), + ], + 'keyword': [ + (r'(address|arg|by|call|do|drop|else|end|exit|for|forever|if|' + r'interpret|iterate|leave|nop|numeric|off|on|options|parse|' + r'pull|push|queue|return|say|select|signal|to|then|trace|until|' + r'while)\b', Keyword.Reserved), + ], + 'operator': [ + (ur'(-|//|/|\(|\)|\*\*|\*|\\<<|\\<|\\==|\\=|\\>>|\\>|\\|\|\||\||' + ur'&&|&|%|\+|<<=|<<|<=|<>|<|==|=|><|>=|>>=|>>|>|¬<<|¬<|¬==|¬=|' + ur'¬>>|¬>|¬|\.|,)', Operator), + ], + 'string_double': [ + (r'[^"\n]+', String), + (r'""', String), + (r'"', String, '#pop'), + (r'\n', Text, '#pop'), # Stray linefeed also terminates strings. + ], + 'string_single': [ + (r'[^\'\n]', String), + (r'\'\'', String), + (r'\'', String, '#pop'), + (r'\n', Text, '#pop'), # Stray linefeed also terminates strings. + ], + 'comment': [ + (r'[^*]+', Comment.Multiline), + (r'\*/', Comment.Multiline, '#pop'), + (r'\*', Comment.Multiline), + ] + } + + _c = lambda s: re.compile(s, re.MULTILINE) + _ADDRESS_COMMAND_PATTERN = _c(r'^\s*address\s+command\b') + _ADDRESS_PATTERN = _c(r'^\s*address\s+') + _DO_WHILE_PATTERN = _c(r'^\s*do\s+while\b') + _IF_THEN_DO_PATTERN = _c(r'^\s*if\b.+\bthen\s+do\s*$') + _PROCEDURE_PATTERN = _c(r'^\s*([a-z_][a-z0-9_]*)(\s*)(:)(\s*)(procedure)\b') + _ELSE_DO_PATTERN = _c(r'\belse\s+do\s*$') + _PARSE_ARG_PATTERN = _c(r'^\s*parse\s+(upper\s+)?(arg|value)\b') + PATTERNS_AND_WEIGHTS = ( + (_ADDRESS_COMMAND_PATTERN, 0.2), + (_ADDRESS_PATTERN, 0.05), + (_DO_WHILE_PATTERN, 0.1), + (_ELSE_DO_PATTERN, 0.1), + (_IF_THEN_DO_PATTERN, 0.1), + (_PROCEDURE_PATTERN, 0.5), + (_PARSE_ARG_PATTERN, 0.2), + ) + + def analyse_text(text): + """ + Check for inital comment and patterns that distinguish Rexx from other + C-like languages. + """ + if re.search(r'/\*\**\s*rexx', text, re.IGNORECASE): + # Header matches MVS Rexx requirements, this is certainly a Rexx + # script. + return 1.0 + elif text.startswith('/*'): + # Header matches general Rexx requirements; the source code might + # still be any language using C comments such as C++, C# or Java. + lowerText = text.lower() + result = sum(weight + for (pattern, weight) in RexxLexer.PATTERNS_AND_WEIGHTS + if pattern.search(lowerText)) + 0.01 + return min(result, 1.0) diff --git a/pygments/lexers/shell.py b/pygments/lexers/shell.py index b95faf93..78c5c996 100644 --- a/pygments/lexers/shell.py +++ b/pygments/lexers/shell.py @@ -67,9 +67,11 @@ class BashLexer(RegexLexer): 'data': [ (r'(?s)\$?"(\\\\|\\[0-7]+|\\.|[^"\\])*"', String.Double), (r"(?s)\$?'(\\\\|\\[0-7]+|\\.|[^'\\])*'", String.Single), - (r';', Text), + (r';', Punctuation), + (r'&', Punctuation), + (r'\|', Punctuation), (r'\s+', Text), - (r'[^=\s\[\]{}()$"\'`\\<]+', Text), + (r'[^=\s\[\]{}()$"\'`\\<&|;]+', Text), (r'\d+(?= |\Z)', Number), (r'\$#?(\w+|.)', Name.Variable), (r'<', Text), @@ -206,7 +208,7 @@ class BatchLexer(RegexLexer): *New in Pygments 0.7.* """ name = 'Batchfile' - aliases = ['bat'] + aliases = ['bat', 'dosbatch', 'winbatch'] filenames = ['*.bat', '*.cmd'] mimetypes = ['application/x-dos-batch'] @@ -329,8 +331,8 @@ class PowerShellLexer(RegexLexer): *New in Pygments 1.5.* """ name = 'PowerShell' - aliases = ['powershell', 'posh', 'ps1'] - filenames = ['*.ps1'] + aliases = ['powershell', 'posh', 'ps1', 'psm1'] + filenames = ['*.ps1','*.psm1'] mimetypes = ['text/x-powershell'] flags = re.DOTALL | re.IGNORECASE | re.MULTILINE @@ -342,7 +344,7 @@ class PowerShellLexer(RegexLexer): 'dynamicparam do default continue cmdletbinding break begin alias \\? ' '% #script #private #local #global mandatory parametersetname position ' 'valuefrompipeline valuefrompipelinebypropertyname ' - 'valuefromremainingarguments helpmessage try catch').split() + 'valuefromremainingarguments helpmessage try catch throw').split() operators = ( 'and as band bnot bor bxor casesensitive ccontains ceq cge cgt cle ' @@ -368,12 +370,15 @@ class PowerShellLexer(RegexLexer): tokens = { 'root': [ + # we need to count pairs of parentheses for correct highlight + # of '$(...)' blocks in strings + (r'\(', Punctuation, 'child'), (r'\s+', Text), (r'^(\s*#[#\s]*)(\.(?:%s))([^\n]*$)' % '|'.join(commenthelp), bygroups(Comment, String.Doc, Comment)), (r'#[^\n]*?$', Comment), (r'(<|<)#', Comment.Multiline, 'multline'), - (r'@"\n.*?\n"@', String.Heredoc), + (r'@"\n', String.Heredoc, 'heredoc-double'), (r"@'\n.*?\n'@", String.Heredoc), # escaped syntax (r'`[\'"$@-]', Punctuation), @@ -387,7 +392,11 @@ class PowerShellLexer(RegexLexer): (r'\[[a-z_\[][a-z0-9_. `,\[\]]*\]', Name.Constant), # .net [type]s (r'-[a-z_][a-z0-9_]*', Name), (r'\w+', Name), - (r'[.,{}\[\]$()=+*/\\&%!~?^`|<>-]', Punctuation), + (r'[.,;@{}\[\]$()=+*/\\&%!~?^`|<>-]|::', Punctuation), + ], + 'child': [ + (r'\)', Punctuation, '#pop'), + include('root'), ], 'multline': [ (r'[^#&.]+', Comment.Multiline), @@ -396,15 +405,17 @@ class PowerShellLexer(RegexLexer): (r'[#&.]', Comment.Multiline), ], 'string': [ + (r"`[0abfnrtv'\"\$]", String.Escape), (r'[^$`"]+', String.Double), - (r'\$\(', String.Interpol, 'interpol'), - (r'`"|""', String.Double), + (r'\$\(', Punctuation, 'child'), + (r'""', String.Double), (r'[`$]', String.Double), (r'"', String.Double, '#pop'), ], - 'interpol': [ - (r'[^$)]+', String.Interpol), - (r'\$\(', String.Interpol, '#push'), - (r'\)', String.Interpol, '#pop'), + 'heredoc-double': [ + (r'\n"@', String.Heredoc, '#pop'), + (r'\$\(', Punctuation, 'child'), + (r'[^@\n]+"]', String.Heredoc), + (r".", String.Heredoc), ] } diff --git a/pygments/lexers/templates.py b/pygments/lexers/templates.py index b3e70d05..63fc5f37 100644 --- a/pygments/lexers/templates.py +++ b/pygments/lexers/templates.py @@ -766,7 +766,7 @@ class CheetahHtmlLexer(DelegatingLexer): """ name = 'HTML+Cheetah' - aliases = ['html+cheetah', 'html+spitfire'] + aliases = ['html+cheetah', 'html+spitfire', 'htmlcheetah'] mimetypes = ['text/html+cheetah', 'text/html+spitfire'] def __init__(self, **options): @@ -1258,7 +1258,7 @@ class HtmlDjangoLexer(DelegatingLexer): """ name = 'HTML+Django/Jinja' - aliases = ['html+django', 'html+jinja'] + aliases = ['html+django', 'html+jinja', 'htmldjango'] alias_filenames = ['*.html', '*.htm', '*.xhtml'] mimetypes = ['text/html+django', 'text/html+jinja'] @@ -1657,7 +1657,7 @@ class LassoHtmlLexer(DelegatingLexer): super(LassoHtmlLexer, self).__init__(HtmlLexer, LassoLexer, **options) def analyse_text(text): - rv = LassoLexer.analyse_text(text) + rv = LassoLexer.analyse_text(text) - 0.01 if re.search(r'<\w+>', text, re.I): rv += 0.2 if html_doctype_matches(text): @@ -1683,9 +1683,9 @@ class LassoXmlLexer(DelegatingLexer): super(LassoXmlLexer, self).__init__(XmlLexer, LassoLexer, **options) def analyse_text(text): - rv = LassoLexer.analyse_text(text) + rv = LassoLexer.analyse_text(text) - 0.01 if looks_like_xml(text): - rv += 0.5 + rv += 0.4 return rv @@ -1707,8 +1707,8 @@ class LassoCssLexer(DelegatingLexer): super(LassoCssLexer, self).__init__(CssLexer, LassoLexer, **options) def analyse_text(text): - rv = LassoLexer.analyse_text(text) - if re.search(r'\w+:.+;', text): + rv = LassoLexer.analyse_text(text) - 0.05 + if re.search(r'\w+:.+?;', text): rv += 0.1 if 'padding:' in text: rv += 0.1 @@ -1736,7 +1736,7 @@ class LassoJavascriptLexer(DelegatingLexer): **options) def analyse_text(text): - rv = LassoLexer.analyse_text(text) + rv = LassoLexer.analyse_text(text) - 0.05 if 'function' in text: rv += 0.2 return rv diff --git a/pygments/lexers/text.py b/pygments/lexers/text.py index 5e340893..f3feec83 100644 --- a/pygments/lexers/text.py +++ b/pygments/lexers/text.py @@ -25,7 +25,7 @@ __all__ = ['IniLexer', 'PropertiesLexer', 'SourcesListLexer', 'BaseMakefileLexer 'RstLexer', 'VimLexer', 'GettextLexer', 'SquidConfLexer', 'DebianControlLexer', 'DarcsPatchLexer', 'YamlLexer', 'LighttpdConfLexer', 'NginxConfLexer', 'CMakeLexer', 'HttpLexer', - 'PyPyLogLexer', 'RegeditLexer', 'HxmlLexer'] + 'PyPyLogLexer', 'RegeditLexer', 'HxmlLexer', 'EbnfLexer'] class IniLexer(RegexLexer): @@ -34,7 +34,7 @@ class IniLexer(RegexLexer): """ name = 'INI' - aliases = ['ini', 'cfg'] + aliases = ['ini', 'cfg', 'dosini'] filenames = ['*.ini', '*.cfg'] mimetypes = ['text/x-ini'] @@ -106,7 +106,7 @@ class PropertiesLexer(RegexLexer): """ name = 'Properties' - aliases = ['properties'] + aliases = ['properties', 'jproperties'] filenames = ['*.properties'] mimetypes = ['text/x-java-properties'] @@ -128,7 +128,7 @@ class SourcesListLexer(RegexLexer): """ name = 'Debian Sourcelist' - aliases = ['sourceslist', 'sources.list'] + aliases = ['sourceslist', 'sources.list', 'debsources'] filenames = ['sources.list'] mimetype = ['application/x-debian-sourceslist'] @@ -1053,7 +1053,7 @@ class DebianControlLexer(RegexLexer): *New in Pygments 0.9.* """ name = 'Debian Control file' - aliases = ['control'] + aliases = ['control', 'debcontrol'] filenames = ['control'] tokens = { @@ -1709,12 +1709,12 @@ class HttpLexer(RegexLexer): tokens = { 'root': [ - (r'(GET|POST|PUT|DELETE|HEAD|OPTIONS|TRACE)( +)([^ ]+)( +)' - r'(HTTPS?)(/)(1\.[01])(\r?\n|$)', + (r'(GET|POST|PUT|DELETE|HEAD|OPTIONS|TRACE|PATCH)( +)([^ ]+)( +)' + r'(HTTP)(/)(1\.[01])(\r?\n|$)', bygroups(Name.Function, Text, Name.Namespace, Text, Keyword.Reserved, Operator, Number, Text), 'headers'), - (r'(HTTPS?)(/)(1\.[01])( +)(\d{3})( +)([^\r\n]+)(\r?\n|$)', + (r'(HTTP)(/)(1\.[01])( +)(\d{3})( +)([^\r\n]+)(\r?\n|$)', bygroups(Keyword.Reserved, Operator, Number, Text, Number, Text, Name.Exception, Text), 'headers'), @@ -1841,3 +1841,53 @@ class HxmlLexer(RegexLexer): (r'#.*', Comment.Single) ] } + + +class EbnfLexer(RegexLexer): + """ + Lexer for `ISO/IEC 14977 EBNF + <http://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form>`_ + grammars. + + *New in Pygments 1.7.* + """ + + name = 'EBNF' + aliases = ['ebnf'] + filenames = ['*.ebnf'] + mimetypes = ['text/x-ebnf'] + + tokens = { + 'root': [ + include('whitespace'), + include('comment_start'), + include('identifier'), + (r'=', Operator, 'production'), + ], + 'production': [ + include('whitespace'), + include('comment_start'), + include('identifier'), + (r'"[^"]*"', String.Double), + (r"'[^']*'", String.Single), + (r'(\?[^?]*\?)', Name.Entity), + (r'[\[\]{}(),|]', Punctuation), + (r'-', Operator), + (r';', Punctuation, '#pop'), + ], + 'whitespace': [ + (r'\s+', Text), + ], + 'comment_start': [ + (r'\(\*', Comment.Multiline, 'comment'), + ], + 'comment': [ + (r'[^*)]', Comment.Multiline), + include('comment_start'), + (r'\*\)', Comment.Multiline, '#pop'), + (r'[*)]', Comment.Multiline), + ], + 'identifier': [ + (r'([a-zA-Z][a-zA-Z0-9 \-]*)', Keyword), + ], + } diff --git a/pygments/lexers/web.py b/pygments/lexers/web.py index 24942007..7d2c812a 100644 --- a/pygments/lexers/web.py +++ b/pygments/lexers/web.py @@ -67,7 +67,7 @@ class JavascriptLexer(RegexLexer): (r'[{(\[;,]', Punctuation, 'slashstartsregex'), (r'[})\].]', Punctuation), (r'(for|in|while|do|break|return|continue|switch|case|default|if|else|' - r'throw|try|catch|finally|new|delete|typeof|instanceof|void|' + r'throw|try|catch|finally|new|delete|typeof|instanceof|void|yield|' r'this)\b', Keyword, 'slashstartsregex'), (r'(var|let|with|function)\b', Keyword.Declaration, 'slashstartsregex'), (r'(abstract|boolean|byte|char|class|const|debugger|double|enum|export|' @@ -1125,227 +1125,840 @@ class MxmlLexer(RegexLexer): } -class HaxeLexer(RegexLexer): +class HaxeLexer(ExtendedRegexLexer): """ - For haXe source code (http://haxe.org/). + For Haxe source code (http://haxe.org/). *New in Pygments 1.3.* """ - name = 'haXe' - aliases = ['hx', 'haXe'] - filenames = ['*.hx'] - mimetypes = ['text/haxe'] + name = 'Haxe' + aliases = ['hx', 'Haxe', 'haxe', 'haXe', 'hxsl'] + filenames = ['*.hx', '*.hxsl'] + mimetypes = ['text/haxe', 'text/x-haxe', 'text/x-hx'] + + # keywords extracted from lexer.mll in the haxe compiler source + keyword = (r'(?:function|class|static|var|if|else|while|do|for|' + r'break|return|continue|extends|implements|import|' + r'switch|case|default|public|private|try|untyped|' + r'catch|new|this|throw|extern|enum|in|interface|' + r'cast|override|dynamic|typedef|package|' + r'inline|using|null|true|false|abstract)\b') + + # idtype in lexer.mll + typeid = r'_*[A-Z][_a-zA-Z0-9]*' + + # combined ident and dollar and idtype + ident = r'(?:_*[a-z][_a-zA-Z0-9]*|_+[0-9][_a-zA-Z0-9]*|' + typeid + \ + '|_+|\$[_a-zA-Z0-9]+)' + + binop = (r'(?:%=|&=|\|=|\^=|\+=|\-=|\*=|/=|<<=|>\s*>\s*=|>\s*>\s*>\s*=|==|' + r'!=|<=|>\s*=|&&|\|\||<<|>>>|>\s*>|\.\.\.|<|>|%|&|\||\^|\+|\*|' + r'/|\-|=>|=)') - ident = r'(?:[a-zA-Z_][a-zA-Z0-9_]*)' - typeid = r'(?:(?:[a-z0-9_\.])*[A-Z_][A-Za-z0-9_]*)' - key_prop = r'(?:default|null|never)' - key_decl_mod = r'(?:public|private|override|static|inline|extern|dynamic)' + # ident except keywords + ident_no_keyword = r'(?!' + keyword + ')' + ident flags = re.DOTALL | re.MULTILINE + preproc_stack = [] + + def preproc_callback(self, match, ctx): + proc = match.group(2) + + if proc == 'if': + # store the current stack + self.preproc_stack.append(ctx.stack[:]) + elif proc in ['else', 'elseif']: + # restore the stack back to right before #if + if self.preproc_stack: ctx.stack = self.preproc_stack[-1][:] + elif proc == 'end': + # remove the saved stack of previous #if + if self.preproc_stack: self.preproc_stack.pop() + + # #if and #elseif should follow by an expr + if proc in ['if', 'elseif']: + ctx.stack.append('preproc-expr') + + # #error can be optionally follow by the error msg + if proc in ['error']: + ctx.stack.append('preproc-error') + + yield match.start(), Comment.Preproc, '#' + proc + ctx.pos = match.end() + + tokens = { 'root': [ - include('whitespace'), - include('comments'), - (key_decl_mod, Keyword.Declaration), - include('enumdef'), - include('typedef'), - include('classdef'), - include('imports'), + include('spaces'), + include('meta'), + (r'(?:package)\b', Keyword.Namespace, ('semicolon', 'package')), + (r'(?:import)\b', Keyword.Namespace, ('semicolon', 'import')), + (r'(?:using)\b', Keyword.Namespace, ('semicolon', 'using')), + (r'(?:extern|private)\b', Keyword.Declaration), + (r'(?:abstract)\b', Keyword.Declaration, 'abstract'), + (r'(?:class|interface)\b', Keyword.Declaration, 'class'), + (r'(?:enum)\b', Keyword.Declaration, 'enum'), + (r'(?:typedef)\b', Keyword.Declaration, 'typedef'), + + # top-level expression + # although it is not supported in haxe, but it is common to write + # expression in web pages the positive lookahead here is to prevent + # an infinite loop at the EOF + (r'(?=.)', Text, 'expr-statement'), + ], + + # space/tab/comment/preproc + 'spaces': [ + (r'\s+', Text), + (r'//[^\n\r]*', Comment.Single), + (r'/\*.*?\*/', Comment.Multiline), + (r'(#)(if|elseif|else|end|error)\b', preproc_callback), ], - # General constructs - 'comments': [ - (r'//.*?\n', Comment.Single), - (r'/\*.*?\*/', Comment.Multiline), - (r'#[^\n]*', Comment.Preproc), + 'string-single-interpol': [ + (r'\$\{', String.Interpol, ('string-interpol-close', 'expr')), + (r'\$\$', String.Escape), + (r'\$(?=' + ident + ')', String.Interpol, 'ident'), + include('string-single'), ], - 'whitespace': [ - include('comments'), - (r'\s+', Text), + + 'string-single': [ + (r"'", String.Single, '#pop'), + (r'\\.', String.Escape), + (r'.', String.Single), ], - 'codekeywords': [ - (r'\b(if|else|while|do|for|in|break|continue|' - r'return|switch|case|try|catch|throw|null|trace|' - r'new|this|super|untyped|cast|callback|here)\b', - Keyword.Reserved), + + 'string-double': [ + (r'"', String.Double, '#pop'), + (r'\\.', String.Escape), + (r'.', String.Double), ], - 'literals': [ - (r'0[xX][0-9a-fA-F]+', Number.Hex), - (r'[0-9]+', Number.Integer), - (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), - (r"'(\\\\|\\'|[^'])*'", String.Single), - (r'"(\\\\|\\"|[^"])*"', String.Double), - (r'~/([^\n])*?/[gisx]*', String.Regex), - (r'\b(true|false|null)\b', Keyword.Constant), - ], - 'codeblock': [ - include('whitespace'), - include('new'), - include('case'), - include('anonfundef'), - include('literals'), - include('vardef'), - include('codekeywords'), - (r'[();,\[\]]', Punctuation), - (r'(?:=|\+=|-=|\*=|/=|%=|&=|\|=|\^=|<<=|>>=|>>>=|\|\||&&|' - r'\.\.\.|==|!=|>|<|>=|<=|\||&|\^|<<|>>>|>>|\+|\-|\*|/|%|' - r'!|\+\+|\-\-|~|\.|\?|\:)', - Operator), - (ident, Name), - - (r'}', Punctuation,'#pop'), - (r'{', Punctuation,'#push'), - ], - - # Instance/Block level constructs - 'propertydef': [ - (r'(\()(' + key_prop + ')(,)(' + key_prop + ')(\))', - bygroups(Punctuation, Keyword.Reserved, Punctuation, - Keyword.Reserved, Punctuation)), + + 'string-interpol-close': [ + (r'\$'+ident, String.Interpol), + (r'\}', String.Interpol, '#pop'), ], - 'new': [ - (r'\bnew\b', Keyword, 'typedecl'), + + 'package': [ + include('spaces'), + (ident, Name.Namespace), + (r'\.', Punctuation, 'import-ident'), + (r'', Text, '#pop'), ], - 'case': [ - (r'\b(case)(\s+)(' + ident + ')(\s*)(\()', - bygroups(Keyword.Reserved, Text, Name, Text, Punctuation), - 'funargdecl'), + + 'import': [ + include('spaces'), + (ident, Name.Namespace), + (r'\*', Keyword), # wildcard import + (r'\.', Punctuation, 'import-ident'), + (r'in', Keyword.Namespace, 'ident'), + (r'', Text, '#pop'), ], - 'vardef': [ - (r'\b(var)(\s+)(' + ident + ')', - bygroups(Keyword.Declaration, Text, Name.Variable), 'vardecl'), + + 'import-ident': [ + include('spaces'), + (r'\*', Keyword, '#pop'), # wildcard import + (ident, Name.Namespace, '#pop'), ], - 'vardecl': [ - include('whitespace'), - include('typelabel'), - (r'=', Operator,'#pop'), - (r';', Punctuation,'#pop'), + + 'using': [ + include('spaces'), + (ident, Name.Namespace), + (r'\.', Punctuation, 'import-ident'), + (r'', Text, '#pop'), + ], + + 'preproc-error': [ + (r'\s+', Comment.Preproc), + (r"'", String.Single, ('#pop', 'string-single')), + (r'"', String.Double, ('#pop', 'string-double')), + (r'', Text, '#pop'), ], - 'instancevardef': [ - (key_decl_mod,Keyword.Declaration), - (r'\b(var)(\s+)(' + ident + ')', - bygroups(Keyword.Declaration, Text, Name.Variable.Instance), - 'instancevardecl'), + + 'preproc-expr': [ + (r'\s+', Comment.Preproc), + (r'\!', Comment.Preproc), + (r'\(', Comment.Preproc, ('#pop', 'preproc-parenthesis')), + + (ident, Comment.Preproc, '#pop'), + (r"'", String.Single, ('#pop', 'string-single')), + (r'"', String.Double, ('#pop', 'string-double')), ], - 'instancevardecl': [ - include('vardecl'), - include('propertydef'), + + 'preproc-parenthesis': [ + (r'\s+', Comment.Preproc), + (r'\)', Comment.Preproc, '#pop'), + ('', Text, 'preproc-expr-in-parenthesis'), ], - 'anonfundef': [ - (r'\bfunction\b', Keyword.Declaration, 'fundecl'), + 'preproc-expr-chain': [ + (r'\s+', Comment.Preproc), + (binop, Comment.Preproc, ('#pop', 'preproc-expr-in-parenthesis')), + (r'', Text, '#pop'), ], - 'instancefundef': [ - (key_decl_mod, Keyword.Declaration), - (r'\b(function)(\s+)(' + ident + ')', - bygroups(Keyword.Declaration, Text, Name.Function), 'fundecl'), + + # same as 'preproc-expr' but able to chain 'preproc-expr-chain' + 'preproc-expr-in-parenthesis': [ + (r'\s+', Comment.Preproc), + (r'\!', Comment.Preproc), + (r'\(', Comment.Preproc, + ('#pop', 'preproc-expr-chain', 'preproc-parenthesis')), + + (ident, Comment.Preproc, ('#pop', 'preproc-expr-chain')), + (r"'", String.Single, + ('#pop', 'preproc-expr-chain', 'string-single')), + (r'"', String.Double, + ('#pop', 'preproc-expr-chain', 'string-double')), ], - 'fundecl': [ - include('whitespace'), - include('typelabel'), - include('generictypedecl'), - (r'\(',Punctuation,'funargdecl'), - (r'(?=[a-zA-Z0-9_])',Text,'#pop'), - (r'{',Punctuation,('#pop','codeblock')), - (r';',Punctuation,'#pop'), - ], - 'funargdecl': [ - include('whitespace'), - (ident, Name.Variable), - include('typelabel'), - include('literals'), - (r'=', Operator), + + 'abstract' : [ + include('spaces'), + (r'', Text, ('#pop', 'abstract-body', 'abstract-relation', + 'abstract-opaque', 'type-param-constraint', 'type-name')), + ], + + 'abstract-body' : [ + include('spaces'), + (r'\{', Punctuation, ('#pop', 'class-body')), + ], + + 'abstract-opaque' : [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'parenthesis-close', 'type')), + (r'', Text, '#pop'), + ], + + 'abstract-relation': [ + include('spaces'), + (r'(?:to|from)', Keyword.Declaration, 'type'), (r',', Punctuation), + (r'', Text, '#pop'), + ], + + 'meta': [ + include('spaces'), + (r'@', Name.Decorator, ('meta-body', 'meta-ident', 'meta-colon')), + ], + + # optional colon + 'meta-colon': [ + include('spaces'), + (r':', Name.Decorator, '#pop'), + (r'', Text, '#pop'), + ], + + # same as 'ident' but set token as Name.Decorator instead of Name + 'meta-ident': [ + include('spaces'), + (ident, Name.Decorator, '#pop'), + ], + + 'meta-body': [ + include('spaces'), + (r'\(', Name.Decorator, ('#pop', 'meta-call')), + (r'', Text, '#pop'), + ], + + 'meta-call': [ + include('spaces'), + (r'\)', Name.Decorator, '#pop'), + (r'', Text, ('#pop', 'meta-call-sep', 'expr')), + ], + + 'meta-call-sep': [ + include('spaces'), + (r'\)', Name.Decorator, '#pop'), + (r',', Punctuation, ('#pop', 'meta-call')), + ], + + 'typedef': [ + include('spaces'), + (r'', Text, ('#pop', 'typedef-body', 'type-param-constraint', + 'type-name')), + ], + + 'typedef-body': [ + include('spaces'), + (r'=', Operator, ('#pop', 'optional-semicolon', 'type')), + ], + + 'enum': [ + include('spaces'), + (r'', Text, ('#pop', 'enum-body', 'bracket-open', + 'type-param-constraint', 'type-name')), + ], + + 'enum-body': [ + include('spaces'), + include('meta'), + (r'\}', Punctuation, '#pop'), + (ident_no_keyword, Name, ('enum-member', 'type-param-constraint')), + ], + + 'enum-member': [ + include('spaces'), + (r'\(', Punctuation, + ('#pop', 'semicolon', 'flag', 'function-param')), + (r'', Punctuation, ('#pop', 'semicolon', 'flag')), + ], + + 'class': [ + include('spaces'), + (r'', Text, ('#pop', 'class-body', 'bracket-open', 'extends', + 'type-param-constraint', 'type-name')), + ], + + 'extends': [ + include('spaces'), + (r'(?:extends|implements)\b', Keyword.Declaration, 'type'), + (r',', Punctuation), # the comma is made optional here, since haxe2 + # requires the comma but haxe3 does not allow it + (r'', Text, '#pop'), + ], + + 'bracket-open': [ + include('spaces'), + (r'\{', Punctuation, '#pop'), + ], + + 'bracket-close': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + ], + + 'class-body': [ + include('spaces'), + include('meta'), + (r'\}', Punctuation, '#pop'), + (r'(?:static|public|private|override|dynamic|inline|macro)\b', + Keyword.Declaration), + (r'', Text, 'class-member'), + ], + + 'class-member': [ + include('spaces'), + (r'(var)\b', Keyword.Declaration, + ('#pop', 'optional-semicolon', 'prop')), + (r'(function)\b', Keyword.Declaration, + ('#pop', 'optional-semicolon', 'class-method')), + ], + + # local function, anonymous or not + 'function-local': [ + include('spaces'), + (r'(' + ident_no_keyword + ')?', Name.Function, + ('#pop', 'expr', 'flag', 'function-param', + 'parenthesis-open', 'type-param-constraint')), + ], + + 'optional-expr': [ + include('spaces'), + include('expr'), + (r'', Text, '#pop'), + ], + + 'class-method': [ + include('spaces'), + (ident, Name.Function, ('#pop', 'optional-expr', 'flag', + 'function-param', 'parenthesis-open', + 'type-param-constraint')), + ], + + # function arguments + 'function-param': [ + include('spaces'), + (r'\)', Punctuation, '#pop'), (r'\?', Punctuation), + (ident_no_keyword, Name, + ('#pop', 'function-param-sep', 'assign', 'flag')), + ], + + 'function-param-sep': [ + include('spaces'), (r'\)', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'function-param')), ], - 'typelabel': [ - (r':', Punctuation, 'type'), + # class property + # eg. var prop(default, null):String; + 'prop': [ + include('spaces'), + (ident_no_keyword, Name, ('#pop', 'assign', 'flag', 'prop-get-set')), ], - 'typedecl': [ - include('whitespace'), - (typeid, Name.Class), - (r'<', Punctuation, 'generictypedecl'), - (r'(?=[{}()=,a-z])', Text,'#pop'), + + 'prop-get-set': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'parenthesis-close', + 'prop-get-set-opt', 'comma', 'prop-get-set-opt')), + (r'', Text, '#pop'), + ], + + 'prop-get-set-opt': [ + include('spaces'), + (r'(?:default|null|never|dynamic|get|set)\b', Keyword, '#pop'), + (ident_no_keyword, Text, '#pop'), #custom getter/setter + ], + + 'expr-statement': [ + include('spaces'), + # makes semicolon optional here, just to avoid checking the last + # one is bracket or not. + (r'', Text, ('#pop', 'optional-semicolon', 'expr')), + ], + + 'expr': [ + include('spaces'), + (r'@', Name.Decorator, ('#pop', 'optional-expr', 'meta-body', + 'meta-ident', 'meta-colon')), + (r'(?:\+\+|\-\-|~(?!/)|!|\-)', Operator), + (r'\(', Punctuation, ('#pop', 'expr-chain', 'parenthesis')), + (r'(?:inline)\b', Keyword.Declaration), + (r'(?:function)\b', Keyword.Declaration, ('#pop', 'expr-chain', + 'function-local')), + (r'\{', Punctuation, ('#pop', 'expr-chain', 'bracket')), + (r'(?:true|false|null)\b', Keyword.Constant, ('#pop', 'expr-chain')), + (r'(?:this)\b', Keyword, ('#pop', 'expr-chain')), + (r'(?:cast)\b', Keyword, ('#pop', 'expr-chain', 'cast')), + (r'(?:try)\b', Keyword, ('#pop', 'catch', 'expr')), + (r'(?:var)\b', Keyword.Declaration, ('#pop', 'var')), + (r'(?:new)\b', Keyword, ('#pop', 'expr-chain', 'new')), + (r'(?:switch)\b', Keyword, ('#pop', 'switch')), + (r'(?:if)\b', Keyword, ('#pop', 'if')), + (r'(?:do)\b', Keyword, ('#pop', 'do')), + (r'(?:while)\b', Keyword, ('#pop', 'while')), + (r'(?:for)\b', Keyword, ('#pop', 'for')), + (r'(?:untyped|throw)\b', Keyword), + (r'(?:return)\b', Keyword, ('#pop', 'optional-expr')), + (r'(?:macro)\b', Keyword, ('#pop', 'macro')), + (r'(?:continue|break)\b', Keyword, '#pop'), + (r'(?:\$\s*[a-z]\b|\$(?!'+ident+'))', Name, ('#pop', 'dollar')), + (ident_no_keyword, Name, ('#pop', 'expr-chain')), + + # Float + (r'\.[0-9]+', Number.Float, ('#pop', 'expr-chain')), + (r'[0-9]+[eE][\+\-]?[0-9]+', Number.Float, ('#pop', 'expr-chain')), + (r'[0-9]+\.[0-9]*[eE][\+\-]?[0-9]+', Number.Float, ('#pop', 'expr-chain')), + (r'[0-9]+\.[0-9]+', Number.Float, ('#pop', 'expr-chain')), + (r'[0-9]+\.(?!' + ident + '|\.\.)', Number.Float, ('#pop', 'expr-chain')), + + # Int + (r'0x[0-9a-fA-F]+', Number.Hex, ('#pop', 'expr-chain')), + (r'[0-9]+', Number.Integer, ('#pop', 'expr-chain')), + + # String + (r"'", String.Single, ('#pop', 'expr-chain', 'string-single-interpol')), + (r'"', String.Double, ('#pop', 'expr-chain', 'string-double')), + + # EReg + (r'~/(\\\\|\\/|[^/\n])*/[gimsu]*', String.Regex, ('#pop', 'expr-chain')), + + # Array + (r'\[', Punctuation, ('#pop', 'expr-chain', 'array-decl')), + ], + + 'expr-chain': [ + include('spaces'), + (r'(?:\+\+|\-\-)', Operator), + (binop, Operator, ('#pop', 'expr')), + (r'(?:in)\b', Keyword, ('#pop', 'expr')), + (r'\?', Operator, ('#pop', 'expr', 'ternary', 'expr')), + (r'(\.)(' + ident_no_keyword + ')', bygroups(Punctuation, Name)), + (r'\[', Punctuation, 'array-access'), + (r'\(', Punctuation, 'call'), + (r'', Text, '#pop'), + ], + + # macro reification + 'macro': [ + include('spaces'), + (r':', Punctuation, ('#pop', 'type')), + (r'', Text, ('#pop', 'expr')), + ], + + # cast can be written as "cast expr" or "cast(expr, type)" + 'cast': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'parenthesis-close', + 'cast-type', 'expr')), + (r'', Text, ('#pop', 'expr')), + ], + + # optionally give a type as the 2nd argument of cast() + 'cast-type': [ + include('spaces'), + (r',', Punctuation, ('#pop', 'type')), + (r'', Text, '#pop'), + ], + + 'catch': [ + include('spaces'), + (r'(?:catch)\b', Keyword, ('expr', 'function-param', + 'parenthesis-open')), + (r'', Text, '#pop'), + ], + + # do-while loop + 'do': [ + include('spaces'), + (r'', Punctuation, ('#pop', 'do-while', 'expr')), + ], + + # the while after do + 'do-while': [ + include('spaces'), + (r'(?:while)\b', Keyword, ('#pop', 'parenthesis', + 'parenthesis-open')), + ], + + 'while': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'expr', 'parenthesis')), + ], + + 'for': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'expr', 'parenthesis')), ], + + 'if': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'else', 'optional-semicolon', 'expr', + 'parenthesis')), + ], + + 'else': [ + include('spaces'), + (r'(?:else)\b', Keyword, ('#pop', 'expr')), + (r'', Text, '#pop'), + ], + + 'switch': [ + include('spaces'), + (r'', Text, ('#pop', 'switch-body', 'bracket-open', 'expr')), + ], + + 'switch-body': [ + include('spaces'), + (r'(?:case|default)\b', Keyword, ('case-block', 'case')), + (r'\}', Punctuation, '#pop'), + ], + + 'case': [ + include('spaces'), + (r':', Punctuation, '#pop'), + (r'', Text, ('#pop', 'case-sep', 'case-guard', 'expr')), + ], + + 'case-sep': [ + include('spaces'), + (r':', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'case')), + ], + + 'case-guard': [ + include('spaces'), + (r'(?:if)\b', Keyword, ('#pop', 'parenthesis', 'parenthesis-open')), + (r'', Text, '#pop'), + ], + + # optional multiple expr under a case + 'case-block': [ + include('spaces'), + (r'(?!(?:case|default)\b|\})', Keyword, 'expr-statement'), + (r'', Text, '#pop'), + ], + + 'new': [ + include('spaces'), + (r'', Text, ('#pop', 'call', 'parenthesis-open', 'type')), + ], + + 'array-decl': [ + include('spaces'), + (r'\]', Punctuation, '#pop'), + (r'', Text, ('#pop', 'array-decl-sep', 'expr')), + ], + + 'array-decl-sep': [ + include('spaces'), + (r'\]', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'array-decl')), + ], + + 'array-access': [ + include('spaces'), + (r'', Text, ('#pop', 'array-access-close', 'expr')), + ], + + 'array-access-close': [ + include('spaces'), + (r'\]', Punctuation, '#pop'), + ], + + 'comma': [ + include('spaces'), + (r',', Punctuation, '#pop'), + ], + + 'colon': [ + include('spaces'), + (r':', Punctuation, '#pop'), + ], + + 'semicolon': [ + include('spaces'), + (r';', Punctuation, '#pop'), + ], + + 'optional-semicolon': [ + include('spaces'), + (r';', Punctuation, '#pop'), + (r'', Text, '#pop'), + ], + + # identity that CAN be a Haxe keyword + 'ident': [ + include('spaces'), + (ident, Name, '#pop'), + ], + + 'dollar': [ + include('spaces'), + (r'\{', Keyword, ('#pop', 'bracket-close', 'expr')), + (r'', Text, ('#pop', 'expr-chain')), + ], + + 'type-name': [ + include('spaces'), + (typeid, Name, '#pop'), + ], + + 'type-full-name': [ + include('spaces'), + (r'\.', Punctuation, 'ident'), + (r'', Text, '#pop'), + ], + 'type': [ - include('whitespace'), - (typeid, Name.Class), - (r'<', Punctuation, 'generictypedecl'), - (r'->', Keyword.Type), - (r'(?=[{}(),;=])', Text, '#pop'), + include('spaces'), + (r'\?', Punctuation), + (ident, Name, ('#pop', 'type-check', 'type-full-name')), + (r'\{', Punctuation, ('#pop', 'type-check', 'type-struct')), + (r'\(', Punctuation, ('#pop', 'type-check', 'type-parenthesis')), ], - 'generictypedecl': [ - include('whitespace'), - (typeid, Name.Class), - (r'<', Punctuation, '#push'), + + 'type-parenthesis': [ + include('spaces'), + (r'', Text, ('#pop', 'parenthesis-close', 'type')), + ], + + 'type-check': [ + include('spaces'), + (r'->', Punctuation, ('#pop', 'type')), + (r'<(?!=)', Punctuation, 'type-param'), + (r'', Text, '#pop'), + ], + + 'type-struct': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + (r'\?', Punctuation), + (r'>', Punctuation, ('comma', 'type')), + (ident_no_keyword, Name, ('#pop', 'type-struct-sep', 'type', 'colon')), + include('class-body'), + ], + + 'type-struct-sep': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'type-struct')), + ], + + # type-param can be a normal type or a constant literal... + 'type-param-type': [ + # Float + (r'\.[0-9]+', Number.Float, '#pop'), + (r'[0-9]+[eE][\+\-]?[0-9]+', Number.Float, '#pop'), + (r'[0-9]+\.[0-9]*[eE][\+\-]?[0-9]+', Number.Float, '#pop'), + (r'[0-9]+\.[0-9]+', Number.Float, '#pop'), + (r'[0-9]+\.(?!' + ident + '|\.\.)', Number.Float, '#pop'), + + # Int + (r'0x[0-9a-fA-F]+', Number.Hex, '#pop'), + (r'[0-9]+', Number.Integer, '#pop'), + + # String + (r"'", String.Single, ('#pop', 'string-single')), + (r'"', String.Double, ('#pop', 'string-double')), + + # EReg + (r'~/(\\\\|\\/|[^/\n])*/[gim]*', String.Regex, '#pop'), + + # Array + (r'\[', Operator, ('#pop', 'array-decl')), + + include('type'), + ], + + # type-param part of a type + # ie. the <A,B> path in Map<A,B> + 'type-param': [ + include('spaces'), + (r'', Text, ('#pop', 'type-param-sep', 'type-param-type')), + ], + + 'type-param-sep': [ + include('spaces'), (r'>', Punctuation, '#pop'), - (r',', Punctuation), + (r',', Punctuation, ('#pop', 'type-param')), ], - # Top level constructs - 'imports': [ - (r'(package|import|using)(\s+)([^;]+)(;)', - bygroups(Keyword.Namespace, Text, Name.Namespace,Punctuation)), + # optional type-param that may include constraint + # ie. <T:Constraint, T2:(ConstraintA,ConstraintB)> + 'type-param-constraint': [ + include('spaces'), + (r'<(?!=)', Punctuation, ('#pop', 'type-param-constraint-sep', + 'type-param-constraint-flag', 'type-name')), + (r'', Text, '#pop'), ], - 'typedef': [ - (r'typedef', Keyword.Declaration, ('typedefprebody', 'typedecl')), + + 'type-param-constraint-sep': [ + include('spaces'), + (r'>', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'type-param-constraint-sep', + 'type-param-constraint-flag', 'type-name')), ], - 'typedefprebody': [ - include('whitespace'), - (r'(=)(\s*)({)', bygroups(Punctuation, Text, Punctuation), - ('#pop', 'typedefbody')), + + # the optional constraint inside type-param + 'type-param-constraint-flag': [ + include('spaces'), + (r':', Punctuation, ('#pop', 'type-param-constraint-flag-type')), + (r'', Text, '#pop'), ], - 'enumdef': [ - (r'enum', Keyword.Declaration, ('enumdefprebody', 'typedecl')), + + 'type-param-constraint-flag-type': [ + include('spaces'), + (r'\(', Punctuation, ('#pop', 'type-param-constraint-flag-type-sep', + 'type')), + (r'', Text, ('#pop', 'type')), ], - 'enumdefprebody': [ - include('whitespace'), - (r'{', Punctuation, ('#pop','enumdefbody')), + + 'type-param-constraint-flag-type-sep': [ + include('spaces'), + (r'\)', Punctuation, '#pop'), + (r',', Punctuation, 'type'), ], - 'classdef': [ - (r'class', Keyword.Declaration, ('classdefprebody', 'typedecl')), + + # a parenthesis expr that contain exactly one expr + 'parenthesis': [ + include('spaces'), + (r'', Text, ('#pop', 'parenthesis-close', 'expr')), ], - 'classdefprebody': [ - include('whitespace'), - (r'(extends|implements)', Keyword.Declaration,'typedecl'), - (r'{', Punctuation, ('#pop', 'classdefbody')), + + 'parenthesis-open': [ + include('spaces'), + (r'\(', Punctuation, '#pop'), ], - 'interfacedef': [ - (r'interface', Keyword.Declaration, - ('interfacedefprebody', 'typedecl')), + + 'parenthesis-close': [ + include('spaces'), + (r'\)', Punctuation, '#pop'), ], - 'interfacedefprebody': [ - include('whitespace'), - (r'(extends)', Keyword.Declaration, 'typedecl'), - (r'{', Punctuation, ('#pop', 'classdefbody')), - ], - - 'typedefbody': [ - include('whitespace'), - include('instancevardef'), - include('instancefundef'), - (r'>', Punctuation, 'typedecl'), - (r',', Punctuation), - (r'}', Punctuation, '#pop'), - ], - 'enumdefbody': [ - include('whitespace'), - (ident, Name.Variable.Instance), - (r'\(', Punctuation, 'funargdecl'), - (r';', Punctuation), - (r'}', Punctuation, '#pop'), - ], - 'classdefbody': [ - include('whitespace'), - include('instancevardef'), - include('instancefundef'), - (r'}', Punctuation, '#pop'), - include('codeblock'), + + 'var': [ + include('spaces'), + (ident_no_keyword, Text, ('#pop', 'var-sep', 'assign', 'flag')), ], + + # optional more var decl. + 'var-sep': [ + include('spaces'), + (r',', Punctuation, ('#pop', 'var')), + (r'', Text, '#pop'), + ], + + # optional assignment + 'assign': [ + include('spaces'), + (r'=', Operator, ('#pop', 'expr')), + (r'', Text, '#pop'), + ], + + # optional type flag + 'flag': [ + include('spaces'), + (r':', Punctuation, ('#pop', 'type')), + (r'', Text, '#pop'), + ], + + # colon as part of a ternary operator (?:) + 'ternary': [ + include('spaces'), + (r':', Operator, '#pop'), + ], + + # function call + 'call': [ + include('spaces'), + (r'\)', Punctuation, '#pop'), + (r'', Text, ('#pop', 'call-sep', 'expr')), + ], + + # after a call param + 'call-sep': [ + include('spaces'), + (r'\)', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'call')), + ], + + # bracket can be block or object + 'bracket': [ + include('spaces'), + (r'(?!(?:\$\s*[a-z]\b|\$(?!'+ident+')))' + ident_no_keyword, Name, + ('#pop', 'bracket-check')), + (r"'", String.Single, ('#pop', 'bracket-check', 'string-single')), + (r'"', String.Double, ('#pop', 'bracket-check', 'string-double')), + (r'', Text, ('#pop', 'block')), + ], + + 'bracket-check': [ + include('spaces'), + (r':', Punctuation, ('#pop', 'object-sep', 'expr')), #is object + (r'', Text, ('#pop', 'block', 'optional-semicolon', 'expr-chain')), #is block + ], + + # code block + 'block': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + (r'', Text, 'expr-statement'), + ], + + # object in key-value pairs + 'object': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + (r'', Text, ('#pop', 'object-sep', 'expr', 'colon', 'ident-or-string')) + ], + + # a key of an object + 'ident-or-string': [ + include('spaces'), + (ident_no_keyword, Name, '#pop'), + (r"'", String.Single, ('#pop', 'string-single')), + (r'"', String.Double, ('#pop', 'string-double')), + ], + + # after a key-value pair in object + 'object-sep': [ + include('spaces'), + (r'\}', Punctuation, '#pop'), + (r',', Punctuation, ('#pop', 'object')), + ], + + + } def analyse_text(text): @@ -1797,7 +2410,7 @@ class CoffeeScriptLexer(RegexLexer): """ name = 'CoffeeScript' - aliases = ['coffee-script', 'coffeescript'] + aliases = ['coffee-script', 'coffeescript', 'coffee'] filenames = ['*.coffee'] mimetypes = ['text/coffeescript'] @@ -3106,8 +3719,8 @@ class LassoLexer(RegexLexer): Additional options accepted: `builtinshighlighting` - If given and ``True``, highlight builtin tags, types, traits, and - methods (default: ``True``). + If given and ``True``, highlight builtin types, traits, methods, and + members (default: ``True``). `requiredelimiters` If given and ``True``, only highlight code between delimiters as Lasso (default: ``False``). @@ -3186,13 +3799,15 @@ class LassoLexer(RegexLexer): # names (r'\$[a-z_][\w.]*', Name.Variable), - (r'#[a-z_][\w.]*|#\d+', Name.Variable.Instance), + (r'#([a-z_][\w.]*|\d+)', Name.Variable.Instance), (r"(\.)('[a-z_][\w.]*')", bygroups(Name.Builtin.Pseudo, Name.Variable.Class)), - (r"(self)(->)('[a-z_][\w.]*')", + (r"(self)(\s*->\s*)('[a-z_][\w.]*')", bygroups(Name.Builtin.Pseudo, Operator, Name.Variable.Class)), (r'(\.\.?)([a-z_][\w.]*)', - bygroups(Name.Builtin.Pseudo, Name.Other)), + bygroups(Name.Builtin.Pseudo, Name.Other.Member)), + (r'(->\\?\s*|&\s*)([a-z_][\w.]*)', + bygroups(Operator, Name.Other.Member)), (r'(self|inherited|global|void)\b', Name.Builtin.Pseudo), (r'-[a-z_][\w.]*', Name.Attribute), (r'(::\s*)([a-z_][\w.]*)', bygroups(Punctuation, Name.Label)), @@ -3207,15 +3822,13 @@ class LassoLexer(RegexLexer): r'Error_UpdateError)\b', Name.Exception), # definitions - (r'(define)(\s+)([a-z_][\w.]*)(\s*)(=>)(\s*)(type|trait|thread)\b', - bygroups(Keyword.Declaration, Text, Name.Class, Text, Operator, - Text, Keyword)), - (r'(define)(\s+)([a-z_][\w.]*)(->)([a-z_][\w.]*=?|[-+*/%<>]|==)', + (r'(define)(\s+)([a-z_][\w.]*)(\s*=>\s*)(type|trait|thread)\b', + bygroups(Keyword.Declaration, Text, Name.Class, Operator, Keyword)), + (r'(define)(\s+)([a-z_][\w.]*)(\s*->\s*)([a-z_][\w.]*=?|[-+*/%<>]|==)', bygroups(Keyword.Declaration, Text, Name.Class, Operator, - Name.Function), 'signature'), + Name.Function), 'signature'), (r'(define)(\s+)([a-z_][\w.]*)', - bygroups(Keyword.Declaration, Text, Name.Function), - 'signature'), + bygroups(Keyword.Declaration, Text, Name.Function), 'signature'), (r'(public|protected|private|provide)(\s+)(([a-z_][\w.]*=?|' r'[-+*/%<>]|==)(?=\s*\())', bygroups(Keyword, Text, Name.Function), 'signature'), @@ -3224,14 +3837,13 @@ class LassoLexer(RegexLexer): # keywords (r'(true|false|none|minimal|full|all)\b', Keyword.Constant), - (r'(local|var|variable|data)\b', Keyword.Declaration), + (r'(local|var|variable|data(?=\s))\b', Keyword.Declaration), (r'(array|date|decimal|duration|integer|map|pair|string|tag|xml|' - r'null)\b', Keyword.Type), + r'null|list|queue|set|stack|staticarray)\b', Keyword.Type), (r'([a-z_][\w.]*)(\s+)(in)\b', bygroups(Name, Text, Keyword)), (r'(let|into)(\s+)([a-z_][\w.]*)', bygroups(Keyword, Text, Name)), (r'require\b', Keyword, 'requiresection'), - (r'(/?)(Namespace_Using)\b', - bygroups(Punctuation, Keyword.Namespace)), + (r'(/?)(Namespace_Using)\b', bygroups(Punctuation, Keyword.Namespace)), (r'(/?)(Cache|Database_Names|Database_SchemaNames|' r'Database_TableNames|Define_Tag|Define_Type|Email_Batch|' r'Encode_Set|HTML_Comment|Handle|Handle_Error|Header|If|Inline|' @@ -3249,16 +3861,13 @@ class LassoLexer(RegexLexer): r'frozen|group|handle_failure|import|in|into|join|let|match|max|' r'min|on|order|parent|protected|provide|public|require|skip|' r'split_thread|sum|take|thread|to|trait|type|where|with|yield)\b', - bygroups(Punctuation, Keyword)), + bygroups(Punctuation, Keyword)), # other - (r'(([a-z_][\w.]*=?|[-+*/%<>]|==)(?=\s*\([^)]*\)\s*=>))', - Name.Function, 'signature'), + (r',', Punctuation, 'commamember'), (r'(and|or|not)\b', Operator.Word), - (r'([a-z_][\w.]*)(\s*)(::\s*)([a-z_][\w.]*)(\s*)(=)', - bygroups(Name, Text, Punctuation, Name.Label, Text, Operator)), - (r'((?<!->)[a-z_][\w.]*)(\s*)(=(?!=))', - bygroups(Name, Text, Operator)), + (r'([a-z_][\w.]*)(\s*::\s*)?([a-z_][\w.]*)?(\s*=(?!=))', + bygroups(Name, Punctuation, Name.Label, Operator)), (r'(/?)([\w.]+)', bygroups(Punctuation, Name.Other)), (r'(=)(bw|ew|cn|lte?|gte?|n?eq|ft|n?rx)\b', bygroups(Operator, Operator.Word)), @@ -3310,6 +3919,13 @@ class LassoLexer(RegexLexer): (r'[(,]', Punctuation), include('whitespacecomments'), ], + 'commamember': [ + (r'(([a-z_][\w.]*=?|[-+*/%<>]|==)' + r'(?=\s*(\(([^()]*\([^()]*\))*[^)]*\)\s*)?(::[\w.\s]+)?=>))', + Name.Function, 'signature'), + include('whitespacecomments'), + (r'', Text, '#pop'), + ], } def __init__(self, **options): @@ -3319,10 +3935,13 @@ class LassoLexer(RegexLexer): options, 'requiredelimiters', False) self._builtins = set() + self._members = set() if self.builtinshighlighting: - from pygments.lexers._lassobuiltins import BUILTINS + from pygments.lexers._lassobuiltins import BUILTINS, MEMBERS for key, value in BUILTINS.iteritems(): self._builtins.update(value) + for key, value in MEMBERS.iteritems(): + self._members.update(value) RegexLexer.__init__(self, **options) def get_tokens_unprocessed(self, text): @@ -3331,22 +3950,22 @@ class LassoLexer(RegexLexer): stack.append('delimiters') for index, token, value in \ RegexLexer.get_tokens_unprocessed(self, text, stack): - if token is Name.Other: - if value.lower() in self._builtins: - yield index, Name.Builtin, value - continue + if (token is Name.Other and value.lower() in self._builtins or + token is Name.Other.Member and value.lower() in self._members): + yield index, Name.Builtin, value + continue yield index, token, value def analyse_text(text): rv = 0.0 if 'bin/lasso9' in text: rv += 0.8 - if re.search(r'<\?(=|lasso)', text, re.I): + if re.search(r'<\?(=|lasso)|\A\[', text, re.I): rv += 0.4 if re.search(r'local\(', text, re.I): rv += 0.4 - if re.search(r'\[\n|\?>', text): - rv += 0.4 + if '?>' in text: + rv += 0.1 return rv diff --git a/pygments/modeline.py b/pygments/modeline.py new file mode 100644 index 00000000..cba1cab2 --- /dev/null +++ b/pygments/modeline.py @@ -0,0 +1,40 @@ +# -*- coding: utf-8 -*- +""" + pygments.modeline + ~~~~~~~~~~~~~~~~~ + + A simple modeline parser (based on pymodeline). + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +__all__ = ['get_filetype_from_buffer'] + +modeline_re = re.compile(r''' + (?: vi | vim | ex ) (?: [<=>]? \d* )? : + .* (?: ft | filetype | syn | syntax ) = ( [^:\s]+ ) +''', re.VERBOSE) + +def get_filetype_from_line(l): + m = modeline_re.search(l) + if m: + return m.group(1) + +def get_filetype_from_buffer(buf, max_lines=5): + """ + Scan the buffer for modelines and return filetype if one is found. + """ + lines = buf.splitlines() + for l in lines[-1:-max_lines-1:-1]: + ret = get_filetype_from_line(l) + if ret: + return ret + for l in lines[max_lines:0:-1]: + ret = get_filetype_from_line(l) + if ret: + return ret + + return None diff --git a/tests/examplefiles/Get-CommandDefinitionHtml.ps1 b/tests/examplefiles/Get-CommandDefinitionHtml.ps1 new file mode 100644 index 00000000..b181955f --- /dev/null +++ b/tests/examplefiles/Get-CommandDefinitionHtml.ps1 @@ -0,0 +1,66 @@ +
+function Get-CommandDefinitionHtml {
+
+ # this tells powershell to allow advanced features,
+ # like the [validatenotnullorempty()] attribute below.
+ [CmdletBinding()]
+ param(
+ [ValidateNotNullOrEmpty()]
+ [string]$name
+ )
+
+ $command = get-command $name
+
+ # Look mom! I'm a cmdlet!
+ $PSCmdlet.WriteVerbose("Dumping HTML for " + $command)
+
+@"
+ <html>
+ <head>
+ <title>$($command.name)</title>
+ </head>
+ <body>
+ <table border="1">
+$(
+ $command.parametersets | % {
+@"
+
+ <tr>
+ <td>$($_.name)</td>
+ <td>
+ <table border="1">
+ <tr>
+ <th colspan="8">Parameters</th>
+
+$(
+ $count = 0
+ $_.parameters | % {
+ if (0 -eq ($count % 8)) {
+@'
+ </tr>
+ <tr>
+'@
+ }
+@"
+ <td>$($_.name)</td>
+"@
+ $count++
+ }
+)
+ </tr>
+ </table>
+ </td>
+ </tr>
+"@
+ }
+)
+ </table>
+ </body>
+ </html>
+"@
+}
+
+Get-CommandDefinitionHtml get-item > out.html
+
+# show in browser
+invoke-item out.html
diff --git a/tests/examplefiles/IPDispatchC.nc b/tests/examplefiles/IPDispatchC.nc new file mode 100644 index 00000000..95a61a2c --- /dev/null +++ b/tests/examplefiles/IPDispatchC.nc @@ -0,0 +1,104 @@ +/* + * "Copyright (c) 2008-2011 The Regents of the University of California. + * All rights reserved." + * + * Permission to use, copy, modify, and distribute this software and its + * documentation for any purpose, without fee, and without written agreement is + * hereby granted, provided that the above copyright notice, the following + * two paragraphs and the author appear in all copies of this software. + * + * IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR + * DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT + * OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF + * CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS + * ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATION TO + * PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS." + * + */ + +/** + * + * + */ +#include "IPDispatch.h" +#include "BlipStatistics.h" + +configuration IPDispatchC { + provides { + interface SplitControl; + interface IPLower; + interface BlipStatistics<ip_statistics_t>; + } +} implementation { + + components MainC; + components NoLedsC as LedsC; + + /* IPDispatchP wiring -- fragment rassembly and lib6lowpan bindings */ + components IPDispatchP; + components CC2420RadioC as MessageC; + components ReadLqiC; + components new TimerMilliC(); + + SplitControl = IPDispatchP.SplitControl; + IPLower = IPDispatchP; + BlipStatistics = IPDispatchP; + + IPDispatchP.Boot -> MainC; +/* #else */ +/* components ResourceSendP; */ +/* ResourceSendP.SubSend -> MessageC; */ +/* ResourceSendP.Resource -> MessageC.SendResource[unique("RADIO_SEND_RESOURCE")]; */ +/* IPDispatchP.Ieee154Send -> ResourceSendP.Ieee154Send; */ +/* #endif */ + IPDispatchP.RadioControl -> MessageC; + + IPDispatchP.BarePacket -> MessageC.BarePacket; + IPDispatchP.Ieee154Send -> MessageC.BareSend; + IPDispatchP.Ieee154Receive -> MessageC.BareReceive; + +#ifdef LOW_POWER_LISTENING + IPDispatchP.LowPowerListening -> MessageC; +#endif + MainC.SoftwareInit -> IPDispatchP.Init; + + IPDispatchP.PacketLink -> MessageC; + IPDispatchP.ReadLqi -> ReadLqiC; + IPDispatchP.Leds -> LedsC; + IPDispatchP.ExpireTimer -> TimerMilliC; + + components new PoolC(message_t, N_FRAGMENTS) as FragPool; + components new PoolC(struct send_entry, N_FRAGMENTS) as SendEntryPool; + components new QueueC(struct send_entry *, N_FRAGMENTS); + components new PoolC(struct send_info, N_CONCURRENT_SENDS) as SendInfoPool; + + IPDispatchP.FragPool -> FragPool; + IPDispatchP.SendEntryPool -> SendEntryPool; + IPDispatchP.SendInfoPool -> SendInfoPool; + IPDispatchP.SendQueue -> QueueC; + + components IPNeighborDiscoveryP; + IPDispatchP.NeighborDiscovery -> IPNeighborDiscoveryP; + +/* components ICMPResponderC; */ +/* #ifdef BLIP_MULTICAST */ +/* components MulticastP; */ +/* components new TrickleTimerMilliC(2, 30, 2, 1); */ +/* IP = MulticastP.IP; */ + +/* MainC.SoftwareInit -> MulticastP.Init; */ +/* MulticastP.MulticastRx -> IPDispatchP.Multicast; */ +/* MulticastP.HopHeader -> IPExtensionP.HopByHopExt[0]; */ +/* MulticastP.TrickleTimer -> TrickleTimerMilliC.TrickleTimer[0]; */ +/* MulticastP.IPExtensions -> IPDispatchP; */ +/* #endif */ + +#ifdef DELUGE + components NWProgC; +#endif + +} diff --git a/tests/examplefiles/IPDispatchP.nc b/tests/examplefiles/IPDispatchP.nc new file mode 100644 index 00000000..628f39a0 --- /dev/null +++ b/tests/examplefiles/IPDispatchP.nc @@ -0,0 +1,671 @@ +/* + * "Copyright (c) 2008 The Regents of the University of California. + * All rights reserved." + * + * Permission to use, copy, modify, and distribute this software and its + * documentation for any purpose, without fee, and without written agreement is + * hereby granted, provided that the above copyright notice, the following + * two paragraphs and the author appear in all copies of this software. + * + * IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR + * DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT + * OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF + * CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS + * ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATION TO + * PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS." + * + */ + +#include <lib6lowpan/blip-tinyos-includes.h> +#include <lib6lowpan/6lowpan.h> +#include <lib6lowpan/lib6lowpan.h> +#include <lib6lowpan/ip.h> +#include <lib6lowpan/in_cksum.h> +#include <lib6lowpan/ip_malloc.h> + +#include "blip_printf.h" +#include "IPDispatch.h" +#include "BlipStatistics.h" +#include "table.h" + +/* + * Provides IP layer reception to applications on motes. + * + * @author Stephen Dawson-Haggerty <stevedh@cs.berkeley.edu> + */ + +module IPDispatchP { + provides { + interface SplitControl; + // interface for protocols not requiring special hand-holding + interface IPLower; + + interface BlipStatistics<ip_statistics_t>; + + } + uses { + interface Boot; + + + /* link-layer wiring */ + interface SplitControl as RadioControl; + + interface Packet as BarePacket; + interface Send as Ieee154Send; + interface Receive as Ieee154Receive; + + /* context lookup */ + interface NeighborDiscovery; + + interface ReadLqi; + interface PacketLink; + interface LowPowerListening; + + /* buffers for outgoing fragments */ + interface Pool<message_t> as FragPool; + interface Pool<struct send_info> as SendInfoPool; + interface Pool<struct send_entry> as SendEntryPool; + interface Queue<struct send_entry *> as SendQueue; + + /* expire reconstruction */ + interface Timer<TMilli> as ExpireTimer; + + interface Leds; + + } + provides interface Init; +} implementation { + +#define HAVE_LOWPAN_EXTERN_MATCH_CONTEXT +int lowpan_extern_read_context(struct in6_addr *addr, int context) { + return call NeighborDiscovery.getContext(context, addr); +} + +int lowpan_extern_match_context(struct in6_addr *addr, uint8_t *ctx_id) { + return call NeighborDiscovery.matchContext(addr, ctx_id); +} + + // generally including source files like this is a no-no. I'm doing + // this in the hope that the optimizer will do a better job when + // they're part of a component. +#include <lib6lowpan/ieee154_header.c> +#include <lib6lowpan/lib6lowpan.c> +#include <lib6lowpan/lib6lowpan_4944.c> +#include <lib6lowpan/lib6lowpan_frag.c> + + enum { + S_RUNNING, + S_STOPPED, + S_STOPPING, + }; + uint8_t state = S_STOPPED; + bool radioBusy; + uint8_t current_local_label = 0; + ip_statistics_t stats; + + // this in theory could be arbitrarily large; however, it needs to + // be large enough to hold all active reconstructions, and any tags + // which we are dropping. It's important to keep dropped tags + // around for a while, or else there are pathological situations + // where you continually allocate buffers for packets which will + // never complete. + + //////////////////////////////////////// + // + // + + table_t recon_cache; + + // table of packets we are currently receiving fragments from, that + // are destined to us + struct lowpan_reconstruct recon_data[N_RECONSTRUCTIONS]; + + // + // + //////////////////////////////////////// + + // task void sendTask(); + + void reconstruct_clear(void *ent) { + struct lowpan_reconstruct *recon = (struct lowpan_reconstruct *)ent; + memclr((uint8_t *)&recon->r_meta, sizeof(struct ip6_metadata)); + recon->r_timeout = T_UNUSED; + recon->r_buf = NULL; + } + + struct send_info *getSendInfo() { + struct send_info *ret = call SendInfoPool.get(); + if (ret == NULL) return ret; + ret->_refcount = 1; + ret->upper_data = NULL; + ret->failed = FALSE; + ret->link_transmissions = 0; + ret->link_fragments = 0; + ret->link_fragment_attempts = 0; + return ret; + } +#define SENDINFO_INCR(X) ((X)->_refcount)++ +void SENDINFO_DECR(struct send_info *si) { + if (--(si->_refcount) == 0) { + call SendInfoPool.put(si); + } +} + + command error_t SplitControl.start() { + return call RadioControl.start(); + } + + command error_t SplitControl.stop() { + if (!radioBusy) { + state = S_STOPPED; + return call RadioControl.stop(); + } else { + // if there's a packet in the radio, wait for it to exit before + // stopping + state = S_STOPPING; + return SUCCESS; + } + } + + event void RadioControl.startDone(error_t error) { +#ifdef LPL_SLEEP_INTERVAL + call LowPowerListening.setLocalWakeupInterval(LPL_SLEEP_INTERVAL); +#endif + + if (error == SUCCESS) { + call Leds.led2Toggle(); + call ExpireTimer.startPeriodic(FRAG_EXPIRE_TIME); + state = S_RUNNING; + radioBusy = FALSE; + } + + signal SplitControl.startDone(error); + } + + event void RadioControl.stopDone(error_t error) { + signal SplitControl.stopDone(error); + } + + command error_t Init.init() { + // ip_malloc_init needs to be in init, not booted, because + // context for coap is initialised in init + ip_malloc_init(); + return SUCCESS; + } + + event void Boot.booted() { + call BlipStatistics.clear(); + + /* set up our reconstruction cache */ + table_init(&recon_cache, recon_data, sizeof(struct lowpan_reconstruct), N_RECONSTRUCTIONS); + table_map(&recon_cache, reconstruct_clear); + + call SplitControl.start(); + } + + /* + * Receive-side code. + */ + void deliver(struct lowpan_reconstruct *recon) { + struct ip6_hdr *iph = (struct ip6_hdr *)recon->r_buf; + + // printf("deliver [%i]: ", recon->r_bytes_rcvd); + // printf_buf(recon->r_buf, recon->r_bytes_rcvd); + + /* the payload length field is always compressed, have to put it back here */ + iph->ip6_plen = htons(recon->r_bytes_rcvd - sizeof(struct ip6_hdr)); + signal IPLower.recv(iph, (void *)(iph + 1), &recon->r_meta); + + // printf("ip_free(%p)\n", recon->r_buf); + ip_free(recon->r_buf); + recon->r_timeout = T_UNUSED; + recon->r_buf = NULL; + } + + /* + * Bulletproof recovery logic is very important to make sure we + * don't get wedged with no free buffers. + * + * The table is managed as follows: + * - unused entries are marked T_UNUSED + * - entries which + * o have a buffer allocated + * o have had a fragment reception before we fired + * are marked T_ACTIVE + * - entries which have not had a fragment reception during the last timer period + * and were active are marked T_ZOMBIE + * - zombie receptions are deleted: their buffer is freed and table entry marked unused. + * - when a fragment is dropped, it is entered into the table as T_FAILED1. + * no buffer is allocated + * - when the timer fires, T_FAILED1 entries are aged to T_FAILED2. + * - T_FAILED2 entries are deleted. Incomming fragments with tags + * that are marked either FAILED1 or FAILED2 are dropped; this + * prevents us from allocating a buffer for a packet which we + * have already dropped fragments from. + * + */ + void reconstruct_age(void *elt) { + struct lowpan_reconstruct *recon = (struct lowpan_reconstruct *)elt; + if (recon->r_timeout != T_UNUSED) + printf("recon src: 0x%x tag: 0x%x buf: %p recvd: %i/%i\n", + recon->r_source_key, recon->r_tag, recon->r_buf, + recon->r_bytes_rcvd, recon->r_size); + switch (recon->r_timeout) { + case T_ACTIVE: + recon->r_timeout = T_ZOMBIE; break; // age existing receptions + case T_FAILED1: + recon->r_timeout = T_FAILED2; break; // age existing receptions + case T_ZOMBIE: + case T_FAILED2: + // deallocate the space for reconstruction + printf("timing out buffer: src: %i tag: %i\n", recon->r_source_key, recon->r_tag); + if (recon->r_buf != NULL) { + printf("ip_free(%p)\n", recon->r_buf); + ip_free(recon->r_buf); + } + recon->r_timeout = T_UNUSED; + recon->r_buf = NULL; + break; + } + } + + void ip_print_heap() { +#ifdef PRINTFUART_ENABLED + bndrt_t *cur = (bndrt_t *)heap; + while (((uint8_t *)cur) - heap < IP_MALLOC_HEAP_SIZE) { + printf ("heap region start: %p length: %u used: %u\n", + cur, (*cur & IP_MALLOC_LEN), (*cur & IP_MALLOC_INUSE) >> 15); + cur = (bndrt_t *)(((uint8_t *)cur) + ((*cur) & IP_MALLOC_LEN)); + } +#endif + } + + event void ExpireTimer.fired() { + table_map(&recon_cache, reconstruct_age); + + + printf("Frag pool size: %i\n", call FragPool.size()); + printf("SendInfo pool size: %i\n", call SendInfoPool.size()); + printf("SendEntry pool size: %i\n", call SendEntryPool.size()); + printf("Forward queue length: %i\n", call SendQueue.size()); + ip_print_heap(); + printfflush(); + } + + /* + * allocate a structure for recording information about incomming fragments. + */ + + struct lowpan_reconstruct *get_reconstruct(uint16_t key, uint16_t tag) { + struct lowpan_reconstruct *ret = NULL; + int i; + + // printf("get_reconstruct: %x %i\n", key, tag); + + for (i = 0; i < N_RECONSTRUCTIONS; i++) { + struct lowpan_reconstruct *recon = (struct lowpan_reconstruct *)&recon_data[i]; + + if (recon->r_tag == tag && + recon->r_source_key == key) { + + if (recon->r_timeout > T_UNUSED) { + recon->r_timeout = T_ACTIVE; + ret = recon; + goto done; + + } else if (recon->r_timeout < T_UNUSED) { + // if we have already tried and failed to get a buffer, we + // need to drop remaining fragments. + ret = NULL; + goto done; + } + } + if (recon->r_timeout == T_UNUSED) + ret = recon; + } + done: + // printf("got%p\n", ret); + return ret; + } + + event message_t *Ieee154Receive.receive(message_t *msg, void *msg_payload, uint8_t len) { + struct packed_lowmsg lowmsg; + struct ieee154_frame_addr frame_address; + uint8_t *buf = msg_payload; + + // printf(" -- RECEIVE -- len : %i\n", len); + + BLIP_STATS_INCR(stats.rx_total); + + /* unpack the 802.15.4 address fields */ + buf = unpack_ieee154_hdr(msg_payload, &frame_address); + len -= buf - (uint8_t *)msg_payload; + + /* unpack and 6lowpan headers */ + lowmsg.data = buf; + lowmsg.len = len; + lowmsg.headers = getHeaderBitmap(&lowmsg); + if (lowmsg.headers == LOWMSG_NALP) { + goto fail; + } + + if (hasFrag1Header(&lowmsg) || hasFragNHeader(&lowmsg)) { + // start reassembly + int rv; + struct lowpan_reconstruct *recon; + uint16_t tag, source_key; + + source_key = ieee154_hashaddr(&frame_address.ieee_src); + getFragDgramTag(&lowmsg, &tag); + recon = get_reconstruct(source_key, tag); + if (!recon) { + goto fail; + } + + /* fill in metadata: on fragmented packets, it applies to the + first fragment only */ + memcpy(&recon->r_meta.sender, &frame_address.ieee_src, + sizeof(ieee154_addr_t)); + recon->r_meta.lqi = call ReadLqi.readLqi(msg); + recon->r_meta.rssi = call ReadLqi.readRssi(msg); + + if (hasFrag1Header(&lowmsg)) { + if (recon->r_buf != NULL) goto fail; + rv = lowpan_recon_start(&frame_address, recon, buf, len); + } else { + rv = lowpan_recon_add(recon, buf, len); + } + + if (rv < 0) { + recon->r_timeout = T_FAILED1; + goto fail; + } else { + // printf("start recon buf: %p\n", recon->r_buf); + recon->r_timeout = T_ACTIVE; + recon->r_source_key = source_key; + recon->r_tag = tag; + } + + if (recon->r_size == recon->r_bytes_rcvd) { + deliver(recon); + } + + } else { + /* no fragmentation, just deliver it */ + int rv; + struct lowpan_reconstruct recon; + + /* fill in metadata */ + memcpy(&recon.r_meta.sender, &frame_address.ieee_src, + sizeof(ieee154_addr_t)); + recon.r_meta.lqi = call ReadLqi.readLqi(msg); + recon.r_meta.rssi = call ReadLqi.readRssi(msg); + + buf = getLowpanPayload(&lowmsg); + if ((rv = lowpan_recon_start(&frame_address, &recon, buf, len)) < 0) { + goto fail; + } + + if (recon.r_size == recon.r_bytes_rcvd) { + deliver(&recon); + } else { + // printf("ip_free(%p)\n", recon.r_buf); + ip_free(recon.r_buf); + } + } + goto done; + fail: + BLIP_STATS_INCR(stats.rx_drop); + done: + return msg; + } + + + /* + * Send-side functionality + */ + task void sendTask() { + struct send_entry *s_entry; + + // printf("sendTask() - sending\n"); + + if (radioBusy || state != S_RUNNING) return; + if (call SendQueue.empty()) return; + // this does not dequeue + s_entry = call SendQueue.head(); + +#ifdef LPL_SLEEP_INTERVAL + call LowPowerListening.setRemoteWakeupInterval(s_entry->msg, + call LowPowerListening.getLocalWakeupInterval()); +#endif + + if (s_entry->info->failed) { + dbg("Drops", "drops: sendTask: dropping failed fragment\n"); + goto fail; + } + + if ((call Ieee154Send.send(s_entry->msg, + call BarePacket.payloadLength(s_entry->msg))) != SUCCESS) { + dbg("Drops", "drops: sendTask: send failed\n"); + goto fail; + } else { + radioBusy = TRUE; + } + + return; + fail: + printf("SEND FAIL\n"); + post sendTask(); + BLIP_STATS_INCR(stats.tx_drop); + + // deallocate the memory associated with this request. + // other fragments associated with this packet will get dropped. + s_entry->info->failed = TRUE; + SENDINFO_DECR(s_entry->info); + call FragPool.put(s_entry->msg); + call SendEntryPool.put(s_entry); + call SendQueue.dequeue(); + } + + + /* + * it will pack the message into the fragment pool and enqueue + * those fragments for sending + * + * it will set + * - payload length + * - version, traffic class and flow label + * + * the source and destination IP addresses must be set by higher + * layers. + */ + command error_t IPLower.send(struct ieee154_frame_addr *frame_addr, + struct ip6_packet *msg, + void *data) { + struct lowpan_ctx ctx; + struct send_info *s_info; + struct send_entry *s_entry; + message_t *outgoing; + + int frag_len = 1; + error_t rc = SUCCESS; + + if (state != S_RUNNING) { + return EOFF; + } + + /* set version to 6 in case upper layers forgot */ + msg->ip6_hdr.ip6_vfc &= ~IPV6_VERSION_MASK; + msg->ip6_hdr.ip6_vfc |= IPV6_VERSION; + + ctx.tag = current_local_label++; + ctx.offset = 0; + + s_info = getSendInfo(); + if (s_info == NULL) { + rc = ERETRY; + goto cleanup_outer; + } + s_info->upper_data = data; + + while (frag_len > 0) { + s_entry = call SendEntryPool.get(); + outgoing = call FragPool.get(); + + if (s_entry == NULL || outgoing == NULL) { + if (s_entry != NULL) + call SendEntryPool.put(s_entry); + if (outgoing != NULL) + call FragPool.put(outgoing); + // this will cause any fragments we have already enqueued to + // be dropped by the send task. + s_info->failed = TRUE; + printf("drops: IP send: no fragments\n"); + rc = ERETRY; + goto done; + } + + call BarePacket.clear(outgoing); + frag_len = lowpan_frag_get(call Ieee154Send.getPayload(outgoing, 0), + call BarePacket.maxPayloadLength(), + msg, + frame_addr, + &ctx); + if (frag_len < 0) { + printf(" get frag error: %i\n", frag_len); + } + + printf("fragment length: %i offset: %i\n", frag_len, ctx.offset); + call BarePacket.setPayloadLength(outgoing, frag_len); + + if (frag_len <= 0) { + call FragPool.put(outgoing); + call SendEntryPool.put(s_entry); + goto done; + } + + if (call SendQueue.enqueue(s_entry) != SUCCESS) { + BLIP_STATS_INCR(stats.encfail); + s_info->failed = TRUE; + printf("drops: IP send: enqueue failed\n"); + goto done; + } + + s_info->link_fragments++; + s_entry->msg = outgoing; + s_entry->info = s_info; + + /* configure the L2 */ + if (frame_addr->ieee_dst.ieee_mode == IEEE154_ADDR_SHORT && + frame_addr->ieee_dst.i_saddr == IEEE154_BROADCAST_ADDR) { + call PacketLink.setRetries(s_entry->msg, 0); + } else { + call PacketLink.setRetries(s_entry->msg, BLIP_L2_RETRIES); + } + call PacketLink.setRetryDelay(s_entry->msg, BLIP_L2_DELAY); + + SENDINFO_INCR(s_info);} + + // printf("got %i frags\n", s_info->link_fragments); + done: + BLIP_STATS_INCR(stats.sent); + SENDINFO_DECR(s_info); + post sendTask(); + cleanup_outer: + return rc; + } + + event void Ieee154Send.sendDone(message_t *msg, error_t error) { + struct send_entry *s_entry = call SendQueue.head(); + + radioBusy = FALSE; + + // printf("sendDone: %p %i\n", msg, error); + + if (state == S_STOPPING) { + call RadioControl.stop(); + state = S_STOPPED; + goto done; + } + + s_entry->info->link_transmissions += (call PacketLink.getRetries(msg)); + s_entry->info->link_fragment_attempts++; + + if (!call PacketLink.wasDelivered(msg)) { + printf("sendDone: was not delivered! (%i tries)\n", + call PacketLink.getRetries(msg)); + s_entry->info->failed = TRUE; + signal IPLower.sendDone(s_entry->info); +/* if (s_entry->info->policy.dest[0] != 0xffff) */ +/* dbg("Drops", "drops: sendDone: frag was not delivered\n"); */ + // need to check for broadcast frames + // BLIP_STATS_INCR(stats.tx_drop); + } else if (s_entry->info->link_fragment_attempts == + s_entry->info->link_fragments) { + signal IPLower.sendDone(s_entry->info); + } + + done: + // kill off any pending fragments + SENDINFO_DECR(s_entry->info); + call FragPool.put(s_entry->msg); + call SendEntryPool.put(s_entry); + call SendQueue.dequeue(); + + post sendTask(); + } + +#if 0 + command struct tlv_hdr *IPExtensions.findTlv(struct ip6_ext *ext, uint8_t tlv_val) { + int len = ext->len - sizeof(struct ip6_ext); + struct tlv_hdr *tlv = (struct tlv_hdr *)(ext + 1); + while (len > 0) { + if (tlv->type == tlv_val) return tlv; + if (tlv->len == 0) return NULL; + tlv = (struct tlv_hdr *)(((uint8_t *)tlv) + tlv->len); + len -= tlv->len; + } + return NULL; + } +#endif + + + /* + * BlipStatistics interface + */ + command void BlipStatistics.get(ip_statistics_t *statistics) { +#ifdef BLIP_STATS_IP_MEM + stats.fragpool = call FragPool.size(); + stats.sendinfo = call SendInfoPool.size(); + stats.sendentry= call SendEntryPool.size(); + stats.sndqueue = call SendQueue.size(); + stats.heapfree = ip_malloc_freespace(); + printf("frag: %i sendinfo: %i sendentry: %i sendqueue: %i heap: %i\n", + stats.fragpool, + stats.sendinfo, + stats.sendentry, + stats.sndqueue, + stats.heapfree); +#endif + memcpy(statistics, &stats, sizeof(ip_statistics_t)); + + } + + command void BlipStatistics.clear() { + memclr((uint8_t *)&stats, sizeof(ip_statistics_t)); + } + +/* default event void IP.recv[uint8_t nxt_hdr](struct ip6_hdr *iph, */ +/* void *payload, */ +/* struct ip_metadata *meta) { */ +/* } */ + +/* default event void Multicast.recv[uint8_t scope](struct ip6_hdr *iph, */ +/* void *payload, */ +/* struct ip_metadata *meta) { */ +/* } */ +} diff --git a/tests/examplefiles/RoleQ.pm6 b/tests/examplefiles/RoleQ.pm6 new file mode 100644 index 00000000..9b66bde4 --- /dev/null +++ b/tests/examplefiles/RoleQ.pm6 @@ -0,0 +1,23 @@ +role q { + token stopper { \' } + + token escape:sym<\\> { <sym> <item=.backslash> } + + token backslash:sym<qq> { <?before 'q'> <quote=.LANG('MAIN','quote')> } + token backslash:sym<\\> { <text=.sym> } + token backslash:sym<stopper> { <text=.stopper> } + + token backslash:sym<miscq> { {} . } + + method tweak_q($v) { self.panic("Too late for :q") } + method tweak_qq($v) { self.panic("Too late for :qq") } +} + +role qq does b1 does c1 does s1 does a1 does h1 does f1 { + token stopper { \" } + token backslash:sym<unrec> { {} (\w) { self.throw_unrecog_backslash_seq: $/[0].Str } } + token backslash:sym<misc> { \W } + + method tweak_q($v) { self.panic("Too late for :q") } + method tweak_qq($v) { self.panic("Too late for :qq") } +} diff --git a/tests/examplefiles/example.ceylon b/tests/examplefiles/example.ceylon index b136b995..04223c56 100644 --- a/tests/examplefiles/example.ceylon +++ b/tests/examplefiles/example.ceylon @@ -1,33 +1,52 @@ +import ceylon.language { parseInteger } + doc "A top-level function, with multi-line documentation." -void topLevel(String? a, Integer b=5, String... seqs) { +void topLevel(String? a, Integer b=5, String* seqs) { function nested(String s) { print(s[1..2]); return true; } - for (s in seqs.filter((String x) x.size > 2)) { + for (s in seqs.filter((String x) => x.size > 2)) { nested(s); } - value uppers = seqs.sequence[].uppercased; - String|Nothing z = a; - Sequence<Integer> ints = { 1, 2, 3, 4, 5 }; + value uppers = seqs.map((String x) { + return x.uppercased; + }); + String|Null z = a; + {Integer+} ints = { 1, 2, 3, 4, 5 }; + value numbers = [ 1, #ffff, #ffff_ffff, $10101010, $1010_1010_1010_1010, + 123_456_789 ]; + value chars = ['a', '\{#ffff}' ]; } -shared class Example<Element>(name, element) satisfies Comparable<Example<Element>> +shared class Example_1<Element>(name, element) satisfies Comparable<Example_1<Element>> given Element satisfies Comparable<Element> { shared String name; shared Element element; + shared [Integer,String] tuple = [1, "2"]; + shared late String lastName; + variable Integer cnt = 0; + + shared Integer count => cnt; + assign count { + assert(count >= cnt); + cnt = count; + } - shared actual Comparison compare(Example<Element> other) { + shared actual Comparison compare(Example_1<Element> other) { return element <=> other.element; } shared actual String string { - return "Example with " + element.string; + return "Example with ``element.string``"; } } -Example<Integer> instance = Example { - name = "Named args call"; +Example_1<Integer> instance = Example_1 { element = 5; + name = "Named args call \{#0060}"; }; + +object example1 extends Example_1<Integer>("object", 5) { +}
\ No newline at end of file diff --git a/tests/examplefiles/example.clay b/tests/examplefiles/example.clay new file mode 100644 index 00000000..784752c6 --- /dev/null +++ b/tests/examplefiles/example.clay @@ -0,0 +1,33 @@ + +/// @section StringLiteralRef + +record StringLiteralRef ( + sizep : Pointer[SizeT], +); + + +/// @section predicates + +overload ContiguousSequence?(#StringLiteralRef) : Bool = true; +[s when StringLiteral?(s)] +overload ContiguousSequence?(#Static[s]) : Bool = true; + + + +/// @section size, begin, end, index + +forceinline overload size(a:StringLiteralRef) = a.sizep^; + +forceinline overload begin(a:StringLiteralRef) : Pointer[Char] = Pointer[Char](a.sizep + 1); +forceinline overload end(a:StringLiteralRef) = begin(a) + size(a); + +[I when Integer?(I)] +forceinline overload index(a:StringLiteralRef, i:I) : ByRef[Char] { + assert["boundsChecks"](i >= 0 and i < size(a), "StringLiteralRef index out of bounds"); + return ref (begin(a) + i)^; +} + +foo() = """ +long\tlong +story +""" diff --git a/tests/examplefiles/example.hx b/tests/examplefiles/example.hx new file mode 100644 index 00000000..fd93bb49 --- /dev/null +++ b/tests/examplefiles/example.hx @@ -0,0 +1,142 @@ +/** + * This is not really a valid Haxe file, but just an demo... + */ + +package; +package net.onthewings; + +import net.onthewings.Test; +import net.onthewings.*; + +using Lambda; +using net.onthewings.Test; + +#if flash8 +// Haxe code specific for flash player 8 +#elseif flash +// Haxe code specific for flash platform (any version) +#elseif js +// Haxe code specific for javascript plaform +#elseif neko +// Haxe code specific for neko plaform +#else +// do something else + #error // will display an error "Not implemented on this platform" + #error "Custom error message" // will display an error "Custom error message" +#end + +0; // Int +-134; // Int +0xFF00; // Int + +123.0; // Float +.14179; // Float +13e50; // Float +-1e-99; // Float + +"hello"; // String +"hello \"world\" !"; // String +'hello "world" !'; // String + +true; // Bool +false; // Bool + +null; // Unknown<0> + +~/[a-z]+/i; // EReg : regular expression + +var point = { "x" : 1, "y" : -5 }; + +{ + var x; + var y = 3; + var z : String; + var w : String = ""; + var a, b : Bool, c : Int = 0; +} + +//haxe3 pattern matching +switch(e.expr) { + case EConst(CString(s)) if (StringTools.startsWith(s, "foo")): + "1"; + case EConst(CString(s)) if (StringTools.startsWith(s, "bar")): + "2"; + case EConst(CInt(i)) if (switch(Std.parseInt(i) * 2) { case 4: true; case _: false; }): + "3"; + case EConst(_): + "4"; + case _: + "5"; +} + +switch [true, 1, "foo"] { + case [true, 1, "foo"]: "0"; + case [true, 1, _]: "1"; + case _: "_"; +} + + +class Test <T:Void->Void> { + private function new():Void { + inline function innerFun(a:Int, b:Int):Int { + return readOnlyField = a + b; + } + + _innerFun(1, 2.3); + } + + static public var instance(get,null):Test; + static function get_instance():Test { + return instance != null ? instance : instance = new Test(); + } +} + +@:native("Test") private class Test2 {} + +extern class Ext {} + +@:macro class M { + @:macro static function test(e:Array<Expr>):ExprOf<String> { + return macro "ok"; + } +} + +enum Color { + Red; + Green; + Blue; + Grey( v : Int ); + Rgb( r : Int, g : Int, b : Int ); + Alpha( a : Int, col : Color ); +} + +class Colors { + static function toInt( c : Color ) : Int { + return switch( c ) { + case Red: 0xFF0000; + case Green: 0x00FF00; + case Blue: 0x0000FF; + case Grey(v): (v << 16) | (v << 8) | v; + case Rgb(r,g,b): (r << 16) | (g << 8) | b; + case Alpha(a,c): (a << 24) | (toInt(c) & 0xFFFFFF); + } + } +} + +class EvtQueue<T : (Event, EventDispatcher)> { + var evt : T; +} + +typedef DS = Dynamic<String>; +typedef Pt = { + var x:Float; + var y:Float; + @:optional var z:Float; /* optional z */ + function add(pt:Pt):Void; +} +typedef Pt2 = { + x:Float, + y:Float, + ?z:Float, //optional z + add : Point -> Void, +}
\ No newline at end of file diff --git a/tests/examplefiles/example.rexx b/tests/examplefiles/example.rexx new file mode 100644 index 00000000..ec4da5ad --- /dev/null +++ b/tests/examplefiles/example.rexx @@ -0,0 +1,50 @@ +/* REXX example. */ + +/* Some basic constructs. */ +almost_pi = 0.1415 + 3 +if almost_pi < 3 then + say 'huh?' +else do + say 'almost_pi=' almost_pi || " - ok" +end +x = '"' || "'" || '''' || """" /* quotes */ + +/* A comment + * spawning multiple + lines. /* / */ + +/* Built-in functions. */ +line = 'line containing some short text' +say WordPos(line, 'some') +say Word(line, 4) + +/* Labels and procedures. */ +some_label : + +divide: procedure + parse arg some other + return some / other + +call divide(5, 2) + +/* Loops */ +do i = 1 to 5 + do j = -3 to -9 by -3 + say i '+' j '=' i + j + end j +end i + +do forever + leave +end + +/* Print a text file on MVS. */ +ADDRESS TSO +"ALLOC F(TEXTFILE) DSN('some.text.dsn') SHR REU" +"EXECIO * DISKR TEXTFILE ( FINIS STEM LINES." +"FREE F(TEXTFILE)" +I = 1 +DO WHILE I <= LINES.0 + SAY ' LINE ' I ' : ' LINES.I + I = I + 1 +END diff --git a/tests/examplefiles/example.stan b/tests/examplefiles/example.stan index 5723403c..e936f54a 100644 --- a/tests/examplefiles/example.stan +++ b/tests/examplefiles/example.stan @@ -6,92 +6,103 @@ It is not a real model and will not compile # also a comment // also a comment data { - // valid name - int abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_abc; - // all types should be highlighed - int a3; - real foo[2]; - vector[3] bar; - row_vector[3] baz; - matrix[3,3] qux; - simplex[3] quux; - ordered[3] corge; - positive_ordered[3] wibble; - corr_matrix[3] grault; - cov_matrix[3] garply; - - real<lower=-1,upper=1> foo1; - real<lower=0> foo2; - real<upper=0> foo3; - - // bad names - // includes . - // real foo.; - // beings with number - //real 0foo; - // begins with _ - //real _foo; + // valid name + int abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_abc; + // all types should be highlighed + int a3; + real foo[2]; + vector[3] bar; + row_vector[3] baz; + matrix[3,3] qux; + simplex[3] quux; + ordered[3] corge; + positive_ordered[3] wibble; + corr_matrix[3] grault; + cov_matrix[3] garply; + + real<lower=-1,upper=1> foo1; + real<lower=0> foo2; + real<upper=0> foo3; } transformed data { - real xyzzy; - int thud; - row_vector grault2; - matrix qux2; - - // all floating point literals should be recognized - // all operators should be recognized - // paren should be recognized; - xyzzy <- 1234.5687 + .123 - (2.7e3 / 2E-5 * 135e-5); - // integer literal - thud <- -12309865; - // ./ and .* should be recognized as operators - grault2 <- grault .* garply ./ garply; - // ' and \ should be regognized as operators - qux2 <- qux' \ bar; - + real xyzzy; + int thud; + row_vector grault2; + matrix qux2; + + // all floating point literals should be recognized + // all operators should be recognized + // paren should be recognized; + xyzzy <- 1234.5687 + .123 - (2.7e3 / 2E-5 * 135e-5); + // integer literal + thud <- -12309865; + // ./ and .* should be recognized as operators + grault2 <- grault .* garply ./ garply; + // ' and \ should be regognized as operators + qux2 <- qux' \ bar; + } parameters { - real fred; - real plugh; - + real fred; + real plugh; } transformed parameters { } model { - // ~, <- are operators, - // T may be be recognized - // normal is a function - fred ~ normal(0, 1) T(-0.5, 0.5); - // interior block - { - real tmp; - // for, in should be highlighted - for (i in 1:10) { - tmp <- tmp + 0.1; - } - } - // lp__ should be highlighted - // normal_log as a function - lp__ <- lp__ + normal_log(plugh, 0, 1); + // ~, <- are operators, + // T may be be recognized + // normal is a function + fred ~ normal(0, 1) T(-0.5, 0.5); + real tmp; + // C++ reserved + real public; + + // control structures + for (i in 1:10) { + tmp <- tmp + 0.1; + } + tmp <- 0.0; + while (tmp < 5.0) { + tmp <- tmp + 1; + } + if (tmp > 0.0) { + print(tmp); + } else { + print(tmp); + } - // print statement and string literal - print("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_~@#$%^&*`'-+={}[].,;: "); - print("Hello, world!"); - print(""); + // operators + tmp || tmp; + tmp && tmp; + tmp == tmp; + tmp != tmp; + tmp < tmp; + tmp <= tmp; + tmp > tmp; + tmp >= tmp; + tmp + tmp; + tmp - tmp; + tmp * tmp; + tmp / tmp; + tmp .* tmp; + tmp ./ tmp; + ! tmp; + - tmp; + + tmp; + tmp '; + // lp__ should be highlighted + // normal_log as a function + lp__ <- lp__ + normal_log(plugh, 0, 1); + + // print statement and string literal + print("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_~@#$%^&*`'-+={}[].,;: "); + print("Hello, world!"); + print(""); + } generated quantities { - real bar1; - bar1 <- foo + 1; + real bar1; + bar1 <- foo + 1; } -## Baddness -//foo <- 2.0; -//foo ~ normal(0, 1); -//not_a_block { -//} - -/* -what happens with this? -*/ -// */ diff --git a/tests/examplefiles/garcia-wachs.kk b/tests/examplefiles/garcia-wachs.kk index f766e051..91a01fbe 100644 --- a/tests/examplefiles/garcia-wachs.kk +++ b/tests/examplefiles/garcia-wachs.kk @@ -1,9 +1,25 @@ -/* This is an example in the Koka Language of the Garcia-Wachs algorithm */
-module garcia-wachs
+// Koka language test module
-public fun main()
-{
- test().print
+// This module implements the GarsiaWachs algorithm.
+// It is an adaptation of the algorithm in ML as described by JeanChristophe Filli�tre:
+// in ''A functional implementation of the GarsiaWachs algorithm. (functional pearl). ML workshop 2008, pages 91--96''.
+// See: http://www.lri.fr/~filliatr/publis/gwWml08.pdf
+//
+// The algorithm is interesting since it uses mutable references shared between a list and tree but the
+// side effects are not observable from outside. Koka automatically infers that the final algorithm is pure.
+// Note: due to a current limitation in the divergence analysis, koka cannot yet infer that mutually recursive
+// definitions in "insert" and "extract" are terminating and the final algorithm still has a divergence effect.
+// However, koka does infer that no other effect (i.e. an exception due to a partial match) can occur.
+module garcsiaWachs
+
+import test = qualified std/flags
+
+# pre processor test
+
+public function main() {
+ wlist = Cons1(('a',3), [('b',2),('c',1),('d',4),('e',5)])
+ tree = wlist.garsiaWachs()
+ tree.show.println()
}
//----------------------------------------------------
@@ -14,10 +30,9 @@ public type tree<a> { con Node(left :tree<a>, right :tree<a>)
}
-fun show( t : tree<char> ) : string
-{
+function show( t : tree<char> ) : string {
match(t) {
- Leaf(c) -> Core.show(c)
+ Leaf(c) -> core/show(c)
Node(l,r) -> "Node(" + show(l) + "," + show(r) + ")"
}
}
@@ -30,23 +45,21 @@ public type list1<a> { Cons1( head : a, tail : list<a> )
}
-
-fun map( xs, f ) {
+function map( xs, f ) {
val Cons1(y,ys) = xs
- return Cons1(f(y), Core.map(ys,f))
+ return Cons1(f(y), core/map(ys,f))
}
-fun zip( xs :list1<a>, ys :list1<b> ) : list1<(a,b)> {
+function zip( xs :list1<a>, ys :list1<b> ) : list1<(a,b)> {
Cons1( (xs.head, ys.head), zip(xs.tail, ys.tail))
}
-
//----------------------------------------------------
// Phase 1
//----------------------------------------------------
-fun insert( after : list<(tree<a>,int)>, t : (tree<a>,int), before : list<(tree<a>,int)> ) : div tree<a>
+function insert( after : list<(tree<a>,int)>, t : (tree<a>,int), before : list<(tree<a>,int)> ) : div tree<a>
{
match(before) {
Nil -> extract( [], Cons1(t,after) )
@@ -60,7 +73,7 @@ fun insert( after : list<(tree<a>,int)>, t : (tree<a>,int), before : list<(tree< }
}
-fun extract( before : list<(tree<a>,int)>, after : list1<(tree<a>,int)> ) : div tree<a>
+function extract( before : list<(tree<a>,int)>, after : list1<(tree<a>,int)> ) : div tree<a>
{
val Cons1((t1,w1) as x, xs ) = after
match(xs) {
@@ -75,25 +88,24 @@ fun extract( before : list<(tree<a>,int)>, after : list1<(tree<a>,int)> ) : div }
}
-
-
-fun balance( xs : list1<(tree<a>,int)> ) : div tree<a>
-{
+function balance( xs : list1<(tree<a>,int)> ) : div tree<a> {
extract( [], xs )
}
-fun mark( depth :int, t :tree<(a,ref<h,int>)> ) : <write<h>> ()
-{
+//----------------------------------------------------
+// Phase 2
+//----------------------------------------------------
+
+function mark( depth :int, t :tree<(a,ref<h,int>)> ) : <write<h>> () {
match(t) {
Leaf((_,d)) -> d := depth
Node(l,r) -> { mark(depth+1,l); mark(depth+1,r) }
}
}
-
-fun build( depth :int, xs :list1<(a,ref<h,int>)> ) : <read<h>,div> (tree<a>,list<(a,ref<h,int>)>)
+function build( depth :int, xs :list1<(a,ref<h,int>)> ) : <read<h>,div> (tree<a>,list<(a,ref<h,int>)>)
{
- if (!xs.head.snd == depth) return (Leaf(xs.head.fst), xs.tail)
+ if (!(xs.head.snd) == depth) return (Leaf(xs.head.fst), xs.tail)
l = build(depth+1, xs)
match(l.snd) {
@@ -105,13 +117,11 @@ fun build( depth :int, xs :list1<(a,ref<h,int>)> ) : <read<h>,div> (tree<a>,list }
}
-public fun test() {
- wlist = Cons1(('a',3), [('b',2),('c',1),('d',4),('e',5)])
- tree = wlist.garciawachs()
- tree.show()
-}
+//----------------------------------------------------
+// Main
+//----------------------------------------------------
-public fun garciawachs( xs : list1<(a,int)> ) : div tree<a>
+public function garsiaWachs( xs : list1<(a,int)> ) : div tree<a>
{
refs = xs.map(fst).map( fun(x) { (x, ref(0)) } )
wleafs = zip( refs.map(Leaf), xs.map(snd) )
diff --git a/tests/examplefiles/grammar-test.p6 b/tests/examplefiles/grammar-test.p6 new file mode 100644 index 00000000..28107f3e --- /dev/null +++ b/tests/examplefiles/grammar-test.p6 @@ -0,0 +1,22 @@ +token pod_formatting_code { + $<code>=<[A..Z]> + '<' { $*POD_IN_FORMATTINGCODE := 1 } + $<content>=[ <!before '>'> <pod_string_character> ]+ + '>' { $*POD_IN_FORMATTINGCODE := 0 } +} + +token pod_string { + <pod_string_character>+ +} + +token something:sym«<» { + <!> +} + +token name { + <!> +} + +token comment:sym<#> { + '#' {} \N* +} diff --git a/tests/examplefiles/objc_example.m b/tests/examplefiles/objc_example.m index 7b49c29e..67b33022 100644 --- a/tests/examplefiles/objc_example.m +++ b/tests/examplefiles/objc_example.m @@ -29,4 +29,4 @@ NSArray *a = @[ @"1", @"2" ]; NSDictionary *d = @{ @"key": @"value" }; NSNumber *n1 = @( 1 ); -NSNumber *n2 = @( [a length] );
\ No newline at end of file +NSNumber *n2 = @( [a length] ); diff --git a/tests/examplefiles/py3tb_test.py3tb b/tests/examplefiles/py3tb_test.py3tb new file mode 100644 index 00000000..706a540f --- /dev/null +++ b/tests/examplefiles/py3tb_test.py3tb @@ -0,0 +1,4 @@ + File "<stdin>", line 1 + 1+ + ^ +SyntaxError: invalid syntax diff --git a/tests/examplefiles/swig_java.swg b/tests/examplefiles/swig_java.swg new file mode 100644 index 00000000..6126a55e --- /dev/null +++ b/tests/examplefiles/swig_java.swg @@ -0,0 +1,1329 @@ +/* ----------------------------------------------------------------------------- + * java.swg + * + * Java typemaps + * ----------------------------------------------------------------------------- */ + +%include <javahead.swg> + +/* The jni, jtype and jstype typemaps work together and so there should be one of each. + * The jni typemap contains the JNI type used in the JNI (C/C++) code. + * The jtype typemap contains the Java type used in the JNI intermediary class. + * The jstype typemap contains the Java type used in the Java proxy classes, type wrapper classes and module class. */ + +/* Fragments */ +%fragment("SWIG_PackData", "header") { +/* Pack binary data into a string */ +SWIGINTERN char * SWIG_PackData(char *c, void *ptr, size_t sz) { + static const char hex[17] = "0123456789abcdef"; + register const unsigned char *u = (unsigned char *) ptr; + register const unsigned char *eu = u + sz; + for (; u != eu; ++u) { + register unsigned char uu = *u; + *(c++) = hex[(uu & 0xf0) >> 4]; + *(c++) = hex[uu & 0xf]; + } + return c; +} +} + +%fragment("SWIG_UnPackData", "header") { +/* Unpack binary data from a string */ +SWIGINTERN const char * SWIG_UnpackData(const char *c, void *ptr, size_t sz) { + register unsigned char *u = (unsigned char *) ptr; + register const unsigned char *eu = u + sz; + for (; u != eu; ++u) { + register char d = *(c++); + register unsigned char uu; + if ((d >= '0') && (d <= '9')) + uu = ((d - '0') << 4); + else if ((d >= 'a') && (d <= 'f')) + uu = ((d - ('a'-10)) << 4); + else + return (char *) 0; + d = *(c++); + if ((d >= '0') && (d <= '9')) + uu |= (d - '0'); + else if ((d >= 'a') && (d <= 'f')) + uu |= (d - ('a'-10)); + else + return (char *) 0; + *u = uu; + } + return c; +} +} + +/* Primitive types */ +%typemap(jni) bool, const bool & "jboolean" +%typemap(jni) char, const char & "jchar" +%typemap(jni) signed char, const signed char & "jbyte" +%typemap(jni) unsigned char, const unsigned char & "jshort" +%typemap(jni) short, const short & "jshort" +%typemap(jni) unsigned short, const unsigned short & "jint" +%typemap(jni) int, const int & "jint" +%typemap(jni) unsigned int, const unsigned int & "jlong" +%typemap(jni) long, const long & "jint" +%typemap(jni) unsigned long, const unsigned long & "jlong" +%typemap(jni) long long, const long long & "jlong" +%typemap(jni) unsigned long long, const unsigned long long & "jobject" +%typemap(jni) float, const float & "jfloat" +%typemap(jni) double, const double & "jdouble" +%typemap(jni) void "void" + +%typemap(jtype) bool, const bool & "boolean" +%typemap(jtype) char, const char & "char" +%typemap(jtype) signed char, const signed char & "byte" +%typemap(jtype) unsigned char, const unsigned char & "short" +%typemap(jtype) short, const short & "short" +%typemap(jtype) unsigned short, const unsigned short & "int" +%typemap(jtype) int, const int & "int" +%typemap(jtype) unsigned int, const unsigned int & "long" +%typemap(jtype) long, const long & "int" +%typemap(jtype) unsigned long, const unsigned long & "long" +%typemap(jtype) long long, const long long & "long" +%typemap(jtype) unsigned long long, const unsigned long long & "java.math.BigInteger" +%typemap(jtype) float, const float & "float" +%typemap(jtype) double, const double & "double" +%typemap(jtype) void "void" + +%typemap(jstype) bool, const bool & "boolean" +%typemap(jstype) char, const char & "char" +%typemap(jstype) signed char, const signed char & "byte" +%typemap(jstype) unsigned char, const unsigned char & "short" +%typemap(jstype) short, const short & "short" +%typemap(jstype) unsigned short, const unsigned short & "int" +%typemap(jstype) int, const int & "int" +%typemap(jstype) unsigned int, const unsigned int & "long" +%typemap(jstype) long, const long & "int" +%typemap(jstype) unsigned long, const unsigned long & "long" +%typemap(jstype) long long, const long long & "long" +%typemap(jstype) unsigned long long, const unsigned long long & "java.math.BigInteger" +%typemap(jstype) float, const float & "float" +%typemap(jstype) double, const double & "double" +%typemap(jstype) void "void" + +%typemap(jni) char *, char *&, char[ANY], char[] "jstring" +%typemap(jtype) char *, char *&, char[ANY], char[] "String" +%typemap(jstype) char *, char *&, char[ANY], char[] "String" + +/* JNI types */ +%typemap(jni) jboolean "jboolean" +%typemap(jni) jchar "jchar" +%typemap(jni) jbyte "jbyte" +%typemap(jni) jshort "jshort" +%typemap(jni) jint "jint" +%typemap(jni) jlong "jlong" +%typemap(jni) jfloat "jfloat" +%typemap(jni) jdouble "jdouble" +%typemap(jni) jstring "jstring" +%typemap(jni) jobject "jobject" +%typemap(jni) jbooleanArray "jbooleanArray" +%typemap(jni) jcharArray "jcharArray" +%typemap(jni) jbyteArray "jbyteArray" +%typemap(jni) jshortArray "jshortArray" +%typemap(jni) jintArray "jintArray" +%typemap(jni) jlongArray "jlongArray" +%typemap(jni) jfloatArray "jfloatArray" +%typemap(jni) jdoubleArray "jdoubleArray" +%typemap(jni) jobjectArray "jobjectArray" + +%typemap(jtype) jboolean "boolean" +%typemap(jtype) jchar "char" +%typemap(jtype) jbyte "byte" +%typemap(jtype) jshort "short" +%typemap(jtype) jint "int" +%typemap(jtype) jlong "long" +%typemap(jtype) jfloat "float" +%typemap(jtype) jdouble "double" +%typemap(jtype) jstring "String" +%typemap(jtype) jobject "Object" +%typemap(jtype) jbooleanArray "boolean[]" +%typemap(jtype) jcharArray "char[]" +%typemap(jtype) jbyteArray "byte[]" +%typemap(jtype) jshortArray "short[]" +%typemap(jtype) jintArray "int[]" +%typemap(jtype) jlongArray "long[]" +%typemap(jtype) jfloatArray "float[]" +%typemap(jtype) jdoubleArray "double[]" +%typemap(jtype) jobjectArray "Object[]" + +%typemap(jstype) jboolean "boolean" +%typemap(jstype) jchar "char" +%typemap(jstype) jbyte "byte" +%typemap(jstype) jshort "short" +%typemap(jstype) jint "int" +%typemap(jstype) jlong "long" +%typemap(jstype) jfloat "float" +%typemap(jstype) jdouble "double" +%typemap(jstype) jstring "String" +%typemap(jstype) jobject "Object" +%typemap(jstype) jbooleanArray "boolean[]" +%typemap(jstype) jcharArray "char[]" +%typemap(jstype) jbyteArray "byte[]" +%typemap(jstype) jshortArray "short[]" +%typemap(jstype) jintArray "int[]" +%typemap(jstype) jlongArray "long[]" +%typemap(jstype) jfloatArray "float[]" +%typemap(jstype) jdoubleArray "double[]" +%typemap(jstype) jobjectArray "Object[]" + +/* Non primitive types */ +%typemap(jni) SWIGTYPE "jlong" +%typemap(jtype) SWIGTYPE "long" +%typemap(jstype) SWIGTYPE "$&javaclassname" + +%typemap(jni) SWIGTYPE [] "jlong" +%typemap(jtype) SWIGTYPE [] "long" +%typemap(jstype) SWIGTYPE [] "$javaclassname" + +%typemap(jni) SWIGTYPE * "jlong" +%typemap(jtype) SWIGTYPE * "long" +%typemap(jstype) SWIGTYPE * "$javaclassname" + +%typemap(jni) SWIGTYPE & "jlong" +%typemap(jtype) SWIGTYPE & "long" +%typemap(jstype) SWIGTYPE & "$javaclassname" + +/* pointer to a class member */ +%typemap(jni) SWIGTYPE (CLASS::*) "jstring" +%typemap(jtype) SWIGTYPE (CLASS::*) "String" +%typemap(jstype) SWIGTYPE (CLASS::*) "$javaclassname" + +/* The following are the in, out, freearg, argout typemaps. These are the JNI code generating typemaps for converting from Java to C and visa versa. */ + +/* primitive types */ +%typemap(in) bool +%{ $1 = $input ? true : false; %} + +%typemap(directorout) bool +%{ $result = $input ? true : false; %} + +%typemap(javadirectorin) bool "$jniinput" +%typemap(javadirectorout) bool "$javacall" + +%typemap(in) char, + signed char, + unsigned char, + short, + unsigned short, + int, + unsigned int, + long, + unsigned long, + long long, + float, + double +%{ $1 = ($1_ltype)$input; %} + +%typemap(directorout) char, + signed char, + unsigned char, + short, + unsigned short, + int, + unsigned int, + long, + unsigned long, + long long, + float, + double +%{ $result = ($1_ltype)$input; %} + +%typemap(directorin, descriptor="Z") bool "$input = (jboolean) $1;" +%typemap(directorin, descriptor="C") char "$input = (jint) $1;" +%typemap(directorin, descriptor="B") signed char "$input = (jbyte) $1;" +%typemap(directorin, descriptor="S") unsigned char "$input = (jshort) $1;" +%typemap(directorin, descriptor="S") short "$input = (jshort) $1;" +%typemap(directorin, descriptor="I") unsigned short "$input = (jint) $1;" +%typemap(directorin, descriptor="I") int "$input = (jint) $1;" +%typemap(directorin, descriptor="J") unsigned int "$input = (jlong) $1;" +%typemap(directorin, descriptor="I") long "$input = (jint) $1;" +%typemap(directorin, descriptor="J") unsigned long "$input = (jlong) $1;" +%typemap(directorin, descriptor="J") long long "$input = (jlong) $1;" +%typemap(directorin, descriptor="F") float "$input = (jfloat) $1;" +%typemap(directorin, descriptor="D") double "$input = (jdouble) $1;" + +%typemap(javadirectorin) char, + signed char, + unsigned char, + short, + unsigned short, + int, + unsigned int, + long, + unsigned long, + long long, + float, + double + "$jniinput" + +%typemap(javadirectorout) char, + signed char, + unsigned char, + short, + unsigned short, + int, + unsigned int, + long, + unsigned long, + long long, + float, + double + "$javacall" + +%typemap(out) bool %{ $result = (jboolean)$1; %} +%typemap(out) char %{ $result = (jchar)$1; %} +%typemap(out) signed char %{ $result = (jbyte)$1; %} +%typemap(out) unsigned char %{ $result = (jshort)$1; %} +%typemap(out) short %{ $result = (jshort)$1; %} +%typemap(out) unsigned short %{ $result = (jint)$1; %} +%typemap(out) int %{ $result = (jint)$1; %} +%typemap(out) unsigned int %{ $result = (jlong)$1; %} +%typemap(out) long %{ $result = (jint)$1; %} +%typemap(out) unsigned long %{ $result = (jlong)$1; %} +%typemap(out) long long %{ $result = (jlong)$1; %} +%typemap(out) float %{ $result = (jfloat)$1; %} +%typemap(out) double %{ $result = (jdouble)$1; %} + +/* unsigned long long */ +/* Convert from BigInteger using the toByteArray member function */ +%typemap(in) unsigned long long { + jclass clazz; + jmethodID mid; + jbyteArray ba; + jbyte* bae; + jsize sz; + int i; + + if (!$input) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "BigInteger null"); + return $null; + } + clazz = JCALL1(GetObjectClass, jenv, $input); + mid = JCALL3(GetMethodID, jenv, clazz, "toByteArray", "()[B"); + ba = (jbyteArray)JCALL2(CallObjectMethod, jenv, $input, mid); + bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + sz = JCALL1(GetArrayLength, jenv, ba); + $1 = 0; + for(i=0; i<sz; i++) { + $1 = ($1 << 8) | ($1_type)(unsigned char)bae[i]; + } + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); +} + +%typemap(directorout) unsigned long long { + jclass clazz; + jmethodID mid; + jbyteArray ba; + jbyte* bae; + jsize sz; + int i; + + if (!$input) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "BigInteger null"); + return $null; + } + clazz = JCALL1(GetObjectClass, jenv, $input); + mid = JCALL3(GetMethodID, jenv, clazz, "toByteArray", "()[B"); + ba = (jbyteArray)JCALL2(CallObjectMethod, jenv, $input, mid); + bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + sz = JCALL1(GetArrayLength, jenv, ba); + $result = 0; + for(i=0; i<sz; i++) { + $result = ($result << 8) | ($1_type)(unsigned char)bae[i]; + } + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); +} + + +/* Convert to BigInteger - byte array holds number in 2's complement big endian format */ +%typemap(out) unsigned long long { + jbyteArray ba = JCALL1(NewByteArray, jenv, 9); + jbyte* bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + jclass clazz = JCALL1(FindClass, jenv, "java/math/BigInteger"); + jmethodID mid = JCALL3(GetMethodID, jenv, clazz, "<init>", "([B)V"); + jobject bigint; + int i; + + bae[0] = 0; + for(i=1; i<9; i++ ) { + bae[i] = (jbyte)($1>>8*(8-i)); + } + + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); + bigint = JCALL3(NewObject, jenv, clazz, mid, ba); + $result = bigint; +} + +/* Convert to BigInteger (see out typemap) */ +%typemap(directorin, descriptor="Ljava/math/BigInteger;") unsigned long long, const unsigned long long & { + jbyteArray ba = JCALL1(NewByteArray, jenv, 9); + jbyte* bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + jclass clazz = JCALL1(FindClass, jenv, "java/math/BigInteger"); + jmethodID mid = JCALL3(GetMethodID, jenv, clazz, "<init>", "([B)V"); + jobject bigint; + int swig_i; + + bae[0] = 0; + for(swig_i=1; swig_i<9; swig_i++ ) { + bae[swig_i] = (jbyte)($1>>8*(8-swig_i)); + } + + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); + bigint = JCALL3(NewObject, jenv, clazz, mid, ba); + $input = bigint; +} + +%typemap(javadirectorin) unsigned long long "$jniinput" +%typemap(javadirectorout) unsigned long long "$javacall" + +/* char * - treat as String */ +%typemap(in, noblock=1) char * { + $1 = 0; + if ($input) { + $1 = ($1_ltype)JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!$1) return $null; + } +} + +%typemap(directorout, noblock=1, warning=SWIGWARN_TYPEMAP_DIRECTOROUT_PTR_MSG) char * { + $1 = 0; + if ($input) { + $result = ($1_ltype)JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!$result) return $null; + } +} + +%typemap(directorin, descriptor="Ljava/lang/String;", noblock=1) char * { + $input = 0; + if ($1) { + $input = JCALL1(NewStringUTF, jenv, (const char *)$1); + if (!$input) return $null; + } +} + +%typemap(freearg, noblock=1) char * { if ($1) JCALL2(ReleaseStringUTFChars, jenv, $input, (const char *)$1); } +%typemap(out, noblock=1) char * { if ($1) $result = JCALL1(NewStringUTF, jenv, (const char *)$1); } +%typemap(javadirectorin) char * "$jniinput" +%typemap(javadirectorout) char * "$javacall" + +/* char *& - treat as String */ +%typemap(in, noblock=1) char *& ($*1_ltype temp = 0) { + $1 = 0; + if ($input) { + temp = ($*1_ltype)JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!temp) return $null; + } + $1 = &temp; +} +%typemap(freearg, noblock=1) char *& { if ($1 && *$1) JCALL2(ReleaseStringUTFChars, jenv, $input, (const char *)*$1); } +%typemap(out, noblock=1) char *& { if (*$1) $result = JCALL1(NewStringUTF, jenv, (const char *)*$1); } + +%typemap(out) void "" +%typemap(javadirectorin) void "$jniinput" +%typemap(javadirectorout) void "$javacall" +%typemap(directorin, descriptor="V") void "" + +/* primitive types by reference */ +%typemap(in) const bool & ($*1_ltype temp) +%{ temp = $input ? true : false; + $1 = &temp; %} + +%typemap(directorout,warning=SWIGWARN_TYPEMAP_THREAD_UNSAFE_MSG) const bool & +%{ static $*1_ltype temp; + temp = $input ? true : false; + $result = &temp; %} + +%typemap(javadirectorin) const bool & "$jniinput" +%typemap(javadirectorout) const bool & "$javacall" + +%typemap(in) const char & ($*1_ltype temp), + const signed char & ($*1_ltype temp), + const unsigned char & ($*1_ltype temp), + const short & ($*1_ltype temp), + const unsigned short & ($*1_ltype temp), + const int & ($*1_ltype temp), + const unsigned int & ($*1_ltype temp), + const long & ($*1_ltype temp), + const unsigned long & ($*1_ltype temp), + const long long & ($*1_ltype temp), + const float & ($*1_ltype temp), + const double & ($*1_ltype temp) +%{ temp = ($*1_ltype)$input; + $1 = &temp; %} + +%typemap(directorout,warning=SWIGWARN_TYPEMAP_THREAD_UNSAFE_MSG) const char &, + const signed char &, + const unsigned char &, + const short &, + const unsigned short &, + const int &, + const unsigned int &, + const long &, + const unsigned long &, + const long long &, + const float &, + const double & +%{ static $*1_ltype temp; + temp = ($*1_ltype)$input; + $result = &temp; %} + +%typemap(directorin, descriptor="Z") const bool & "$input = (jboolean)$1;" +%typemap(directorin, descriptor="C") const char & "$input = (jchar)$1;" +%typemap(directorin, descriptor="B") const signed char & "$input = (jbyte)$1;" +%typemap(directorin, descriptor="S") const unsigned char & "$input = (jshort)$1;" +%typemap(directorin, descriptor="S") const short & "$input = (jshort)$1;" +%typemap(directorin, descriptor="I") const unsigned short & "$input = (jint)$1;" +%typemap(directorin, descriptor="I") const int & "$input = (jint)$1;" +%typemap(directorin, descriptor="J") const unsigned int & "$input = (jlong)$1;" +%typemap(directorin, descriptor="I") const long & "$input = (jint)$1;" +%typemap(directorin, descriptor="J") const unsigned long & "$input = (jlong)$1;" +%typemap(directorin, descriptor="J") const long long & "$input = (jlong)$1;" +%typemap(directorin, descriptor="F") const float & "$input = (jfloat)$1;" +%typemap(directorin, descriptor="D") const double & "$input = (jdouble)$1;" + +%typemap(javadirectorin) const char & ($*1_ltype temp), + const signed char & ($*1_ltype temp), + const unsigned char & ($*1_ltype temp), + const short & ($*1_ltype temp), + const unsigned short & ($*1_ltype temp), + const int & ($*1_ltype temp), + const unsigned int & ($*1_ltype temp), + const long & ($*1_ltype temp), + const unsigned long & ($*1_ltype temp), + const long long & ($*1_ltype temp), + const float & ($*1_ltype temp), + const double & ($*1_ltype temp) + "$jniinput" + +%typemap(javadirectorout) const char & ($*1_ltype temp), + const signed char & ($*1_ltype temp), + const unsigned char & ($*1_ltype temp), + const short & ($*1_ltype temp), + const unsigned short & ($*1_ltype temp), + const int & ($*1_ltype temp), + const unsigned int & ($*1_ltype temp), + const long & ($*1_ltype temp), + const unsigned long & ($*1_ltype temp), + const long long & ($*1_ltype temp), + const float & ($*1_ltype temp), + const double & ($*1_ltype temp) + "$javacall" + + +%typemap(out) const bool & %{ $result = (jboolean)*$1; %} +%typemap(out) const char & %{ $result = (jchar)*$1; %} +%typemap(out) const signed char & %{ $result = (jbyte)*$1; %} +%typemap(out) const unsigned char & %{ $result = (jshort)*$1; %} +%typemap(out) const short & %{ $result = (jshort)*$1; %} +%typemap(out) const unsigned short & %{ $result = (jint)*$1; %} +%typemap(out) const int & %{ $result = (jint)*$1; %} +%typemap(out) const unsigned int & %{ $result = (jlong)*$1; %} +%typemap(out) const long & %{ $result = (jint)*$1; %} +%typemap(out) const unsigned long & %{ $result = (jlong)*$1; %} +%typemap(out) const long long & %{ $result = (jlong)*$1; %} +%typemap(out) const float & %{ $result = (jfloat)*$1; %} +%typemap(out) const double & %{ $result = (jdouble)*$1; %} + +/* const unsigned long long & */ +/* Similar to unsigned long long */ +%typemap(in) const unsigned long long & ($*1_ltype temp) { + jclass clazz; + jmethodID mid; + jbyteArray ba; + jbyte* bae; + jsize sz; + int i; + + if (!$input) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "BigInteger null"); + return $null; + } + clazz = JCALL1(GetObjectClass, jenv, $input); + mid = JCALL3(GetMethodID, jenv, clazz, "toByteArray", "()[B"); + ba = (jbyteArray)JCALL2(CallObjectMethod, jenv, $input, mid); + bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + sz = JCALL1(GetArrayLength, jenv, ba); + $1 = &temp; + temp = 0; + for(i=0; i<sz; i++) { + temp = (temp << 8) | ($*1_ltype)(unsigned char)bae[i]; + } + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); +} + +%typemap(directorout,warning=SWIGWARN_TYPEMAP_THREAD_UNSAFE_MSG) const unsigned long long & { + static $*1_ltype temp; + jclass clazz; + jmethodID mid; + jbyteArray ba; + jbyte* bae; + jsize sz; + int i; + + if (!$input) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "BigInteger null"); + return $null; + } + clazz = JCALL1(GetObjectClass, jenv, $input); + mid = JCALL3(GetMethodID, jenv, clazz, "toByteArray", "()[B"); + ba = (jbyteArray)JCALL2(CallObjectMethod, jenv, $input, mid); + bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + sz = JCALL1(GetArrayLength, jenv, ba); + $result = &temp; + temp = 0; + for(i=0; i<sz; i++) { + temp = (temp << 8) | ($*1_ltype)(unsigned char)bae[i]; + } + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); +} + +%typemap(out) const unsigned long long & { + jbyteArray ba = JCALL1(NewByteArray, jenv, 9); + jbyte* bae = JCALL2(GetByteArrayElements, jenv, ba, 0); + jclass clazz = JCALL1(FindClass, jenv, "java/math/BigInteger"); + jmethodID mid = JCALL3(GetMethodID, jenv, clazz, "<init>", "([B)V"); + jobject bigint; + int i; + + bae[0] = 0; + for(i=1; i<9; i++ ) { + bae[i] = (jbyte)(*$1>>8*(8-i)); + } + + JCALL3(ReleaseByteArrayElements, jenv, ba, bae, 0); + bigint = JCALL3(NewObject, jenv, clazz, mid, ba); + $result = bigint; +} + +%typemap(javadirectorin) const unsigned long long & "$jniinput" +%typemap(javadirectorout) const unsigned long long & "$javacall" + +/* Default handling. Object passed by value. Convert to a pointer */ +%typemap(in) SWIGTYPE ($&1_type argp) +%{ argp = *($&1_ltype*)&$input; + if (!argp) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "Attempt to dereference null $1_type"); + return $null; + } + $1 = *argp; %} + +%typemap(directorout) SWIGTYPE ($&1_type argp) +%{ argp = *($&1_ltype*)&$input; + if (!argp) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "Unexpected null return for type $1_type"); + return $null; + } + $result = *argp; %} + +%typemap(out) SWIGTYPE +#ifdef __cplusplus +%{ *($&1_ltype*)&$result = new $1_ltype((const $1_ltype &)$1); %} +#else +{ + $&1_ltype $1ptr = ($&1_ltype) malloc(sizeof($1_ltype)); + memmove($1ptr, &$1, sizeof($1_type)); + *($&1_ltype*)&$result = $1ptr; +} +#endif + +%typemap(directorin,descriptor="L$packagepath/$&javaclassname;") SWIGTYPE +%{ $input = 0; + *(($&1_ltype*)&$input) = &$1; %} +%typemap(javadirectorin) SWIGTYPE "new $&javaclassname($jniinput, false)" +%typemap(javadirectorout) SWIGTYPE "$&javaclassname.getCPtr($javacall)" + +/* Generic pointers and references */ +%typemap(in) SWIGTYPE * %{ $1 = *($&1_ltype)&$input; %} +%typemap(in, fragment="SWIG_UnPackData") SWIGTYPE (CLASS::*) { + const char *temp = 0; + if ($input) { + temp = JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!temp) return $null; + } + SWIG_UnpackData(temp, (void *)&$1, sizeof($1)); +} +%typemap(in) SWIGTYPE & %{ $1 = *($&1_ltype)&$input; + if (!$1) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "$1_type reference is null"); + return $null; + } %} +%typemap(out) SWIGTYPE * +%{ *($&1_ltype)&$result = $1; %} +%typemap(out, fragment="SWIG_PackData", noblock=1) SWIGTYPE (CLASS::*) { + char buf[128]; + char *data = SWIG_PackData(buf, (void *)&$1, sizeof($1)); + *data = '\0'; + $result = JCALL1(NewStringUTF, jenv, buf); +} +%typemap(out) SWIGTYPE & +%{ *($&1_ltype)&$result = $1; %} + +%typemap(directorout, warning=SWIGWARN_TYPEMAP_DIRECTOROUT_PTR_MSG) SWIGTYPE * +%{ $result = *($&1_ltype)&$input; %} +%typemap(directorout, warning=SWIGWARN_TYPEMAP_DIRECTOROUT_PTR_MSG) SWIGTYPE (CLASS::*) +%{ $result = *($&1_ltype)&$input; %} + +%typemap(directorin,descriptor="L$packagepath/$javaclassname;") SWIGTYPE * +%{ *(($&1_ltype)&$input) = ($1_ltype) $1; %} +%typemap(directorin,descriptor="L$packagepath/$javaclassname;") SWIGTYPE (CLASS::*) +%{ *(($&1_ltype)&$input) = ($1_ltype) $1; %} + +%typemap(directorout, warning=SWIGWARN_TYPEMAP_DIRECTOROUT_PTR_MSG) SWIGTYPE & +%{ if (!$input) { + SWIG_JavaThrowException(jenv, SWIG_JavaNullPointerException, "Unexpected null return for type $1_type"); + return $null; + } + $result = *($&1_ltype)&$input; %} +%typemap(directorin,descriptor="L$packagepath/$javaclassname;") SWIGTYPE & +%{ *($&1_ltype)&$input = ($1_ltype) &$1; %} + +%typemap(javadirectorin) SWIGTYPE *, SWIGTYPE (CLASS::*) "($jniinput == 0) ? null : new $javaclassname($jniinput, false)" +%typemap(javadirectorin) SWIGTYPE & "new $javaclassname($jniinput, false)" +%typemap(javadirectorout) SWIGTYPE *, SWIGTYPE (CLASS::*), SWIGTYPE & "$javaclassname.getCPtr($javacall)" + +/* Default array handling */ +%typemap(in) SWIGTYPE [] %{ $1 = *($&1_ltype)&$input; %} +%typemap(out) SWIGTYPE [] %{ *($&1_ltype)&$result = $1; %} +%typemap(freearg) SWIGTYPE [ANY], SWIGTYPE [] "" + +/* char arrays - treat as String */ +%typemap(in, noblock=1) char[ANY], char[] { + $1 = 0; + if ($input) { + $1 = ($1_ltype)JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!$1) return $null; + } +} + +%typemap(directorout, noblock=1) char[ANY], char[] { + $1 = 0; + if ($input) { + $result = ($1_ltype)JCALL2(GetStringUTFChars, jenv, $input, 0); + if (!$result) return $null; + } +} + +%typemap(directorin, descriptor="Ljava/lang/String;", noblock=1) char[ANY], char[] { + $input = 0; + if ($1) { + $input = JCALL1(NewStringUTF, jenv, (const char *)$1); + if (!$input) return $null; + } +} + +%typemap(argout) char[ANY], char[] "" +%typemap(freearg, noblock=1) char[ANY], char[] { if ($1) JCALL2(ReleaseStringUTFChars, jenv, $input, (const char *)$1); } +%typemap(out, noblock=1) char[ANY], char[] { if ($1) $result = JCALL1(NewStringUTF, jenv, (const char *)$1); } +%typemap(javadirectorin) char[ANY], char[] "$jniinput" +%typemap(javadirectorout) char[ANY], char[] "$javacall" + +/* JNI types */ +%typemap(in) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray +%{ $1 = $input; %} + +%typemap(directorout) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray +%{ $result = $input; %} + +%typemap(out) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray +%{ $result = $1; %} + +%typemap(directorin,descriptor="Z") jboolean "$input = $1;" +%typemap(directorin,descriptor="C") jchar "$input = $1;" +%typemap(directorin,descriptor="B") jbyte "$input = $1;" +%typemap(directorin,descriptor="S") jshort "$input = $1;" +%typemap(directorin,descriptor="I") jint "$input = $1;" +%typemap(directorin,descriptor="J") jlong "$input = $1;" +%typemap(directorin,descriptor="F") jfloat "$input = $1;" +%typemap(directorin,descriptor="D") jdouble "$input = $1;" +%typemap(directorin,descriptor="Ljava/lang/String;") jstring "$input = $1;" +%typemap(directorin,descriptor="Ljava/lang/Object;",nouse="1") jobject "$input = $1;" +%typemap(directorin,descriptor="[Z") jbooleanArray "$input = $1;" +%typemap(directorin,descriptor="[C") jcharArray "$input = $1;" +%typemap(directorin,descriptor="[B") jbyteArray "$input = $1;" +%typemap(directorin,descriptor="[S") jshortArray "$input = $1;" +%typemap(directorin,descriptor="[I") jintArray "$input = $1;" +%typemap(directorin,descriptor="[J") jlongArray "$input = $1;" +%typemap(directorin,descriptor="[F") jfloatArray "$input = $1;" +%typemap(directorin,descriptor="[D") jdoubleArray "$input = $1;" +%typemap(directorin,descriptor="[Ljava/lang/Object;",nouse="1") jobjectArray "$input = $1;" + +%typemap(javadirectorin) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray + "$jniinput" + +%typemap(javadirectorout) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray + "$javacall" + +/* Typecheck typemaps - The purpose of these is merely to issue a warning for overloaded C++ functions + * that cannot be overloaded in Java as more than one C++ type maps to a single Java type */ + +%typecheck(SWIG_TYPECHECK_BOOL) /* Java boolean */ + jboolean, + bool, + const bool & + "" + +%typecheck(SWIG_TYPECHECK_CHAR) /* Java char */ + jchar, + char, + const char & + "" + +%typecheck(SWIG_TYPECHECK_INT8) /* Java byte */ + jbyte, + signed char, + const signed char & + "" + +%typecheck(SWIG_TYPECHECK_INT16) /* Java short */ + jshort, + unsigned char, + short, + const unsigned char &, + const short & + "" + +%typecheck(SWIG_TYPECHECK_INT32) /* Java int */ + jint, + unsigned short, + int, + long, + const unsigned short &, + const int &, + const long & + "" + +%typecheck(SWIG_TYPECHECK_INT64) /* Java long */ + jlong, + unsigned int, + unsigned long, + long long, + const unsigned int &, + const unsigned long &, + const long long & + "" + +%typecheck(SWIG_TYPECHECK_INT128) /* Java BigInteger */ + unsigned long long, + const unsigned long long & + "" + +%typecheck(SWIG_TYPECHECK_FLOAT) /* Java float */ + jfloat, + float, + const float & + "" + +%typecheck(SWIG_TYPECHECK_DOUBLE) /* Java double */ + jdouble, + double, + const double & + "" + +%typecheck(SWIG_TYPECHECK_STRING) /* Java String */ + jstring, + char *, + char *&, + char[ANY], + char [] + "" + +%typecheck(SWIG_TYPECHECK_BOOL_ARRAY) /* Java boolean[] */ + jbooleanArray + "" + +%typecheck(SWIG_TYPECHECK_CHAR_ARRAY) /* Java char[] */ + jcharArray + "" + +%typecheck(SWIG_TYPECHECK_INT8_ARRAY) /* Java byte[] */ + jbyteArray + "" + +%typecheck(SWIG_TYPECHECK_INT16_ARRAY) /* Java short[] */ + jshortArray + "" + +%typecheck(SWIG_TYPECHECK_INT32_ARRAY) /* Java int[] */ + jintArray + "" + +%typecheck(SWIG_TYPECHECK_INT64_ARRAY) /* Java long[] */ + jlongArray + "" + +%typecheck(SWIG_TYPECHECK_FLOAT_ARRAY) /* Java float[] */ + jfloatArray + "" + +%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY) /* Java double[] */ + jdoubleArray + "" + +%typecheck(SWIG_TYPECHECK_OBJECT_ARRAY) /* Java jobject[] */ + jobjectArray + "" + +%typecheck(SWIG_TYPECHECK_POINTER) /* Default */ + SWIGTYPE, + SWIGTYPE *, + SWIGTYPE &, + SWIGTYPE *const&, + SWIGTYPE [], + SWIGTYPE (CLASS::*) + "" + + +/* Exception handling */ + +%typemap(throws) int, + long, + short, + unsigned int, + unsigned long, + unsigned short +%{ char error_msg[256]; + sprintf(error_msg, "C++ $1_type exception thrown, value: %d", $1); + SWIG_JavaThrowException(jenv, SWIG_JavaRuntimeException, error_msg); + return $null; %} + +%typemap(throws) SWIGTYPE, SWIGTYPE &, SWIGTYPE *, SWIGTYPE [], SWIGTYPE [ANY] +%{ (void)$1; + SWIG_JavaThrowException(jenv, SWIG_JavaRuntimeException, "C++ $1_type exception thrown"); + return $null; %} + +%typemap(throws) char * +%{ SWIG_JavaThrowException(jenv, SWIG_JavaRuntimeException, $1); + return $null; %} + + +/* Typemaps for code generation in proxy classes and Java type wrapper classes */ + +/* The javain typemap is used for converting function parameter types from the type + * used in the proxy, module or type wrapper class to the type used in the JNI class. */ +%typemap(javain) bool, const bool &, + char, const char &, + signed char, const signed char &, + unsigned char, const unsigned char &, + short, const short &, + unsigned short, const unsigned short &, + int, const int &, + unsigned int, const unsigned int &, + long, const long &, + unsigned long, const unsigned long &, + long long, const long long &, + unsigned long long, const unsigned long long &, + float, const float &, + double, const double & + "$javainput" +%typemap(javain) char *, char *&, char[ANY], char[] "$javainput" +%typemap(javain) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray + "$javainput" +%typemap(javain) SWIGTYPE "$&javaclassname.getCPtr($javainput)" +%typemap(javain) SWIGTYPE *, SWIGTYPE &, SWIGTYPE [] "$javaclassname.getCPtr($javainput)" +%typemap(javain) SWIGTYPE (CLASS::*) "$javaclassname.getCMemberPtr($javainput)" + +/* The javaout typemap is used for converting function return types from the return type + * used in the JNI class to the type returned by the proxy, module or type wrapper class. */ +%typemap(javaout) bool, const bool &, + char, const char &, + signed char, const signed char &, + unsigned char, const unsigned char &, + short, const short &, + unsigned short, const unsigned short &, + int, const int &, + unsigned int, const unsigned int &, + long, const long &, + unsigned long, const unsigned long &, + long long, const long long &, + unsigned long long, const unsigned long long &, + float, const float &, + double, const double & { + return $jnicall; + } +%typemap(javaout) char *, char *&, char[ANY], char[] { + return $jnicall; + } +%typemap(javaout) jboolean, + jchar, + jbyte, + jshort, + jint, + jlong, + jfloat, + jdouble, + jstring, + jobject, + jbooleanArray, + jcharArray, + jbyteArray, + jshortArray, + jintArray, + jlongArray, + jfloatArray, + jdoubleArray, + jobjectArray { + return $jnicall; + } +%typemap(javaout) void { + $jnicall; + } +%typemap(javaout) SWIGTYPE { + return new $&javaclassname($jnicall, true); + } +%typemap(javaout) SWIGTYPE & { + return new $javaclassname($jnicall, $owner); + } +%typemap(javaout) SWIGTYPE *, SWIGTYPE [] { + long cPtr = $jnicall; + return (cPtr == 0) ? null : new $javaclassname(cPtr, $owner); + } +%typemap(javaout) SWIGTYPE (CLASS::*) { + String cMemberPtr = $jnicall; + return (cMemberPtr == null) ? null : new $javaclassname(cMemberPtr, $owner); + } + +/* Pointer reference typemaps */ +%typemap(jni) SWIGTYPE *const& "jlong" +%typemap(jtype) SWIGTYPE *const& "long" +%typemap(jstype) SWIGTYPE *const& "$*javaclassname" +%typemap(javain) SWIGTYPE *const& "$*javaclassname.getCPtr($javainput)" +%typemap(javaout) SWIGTYPE *const& { + long cPtr = $jnicall; + return (cPtr == 0) ? null : new $*javaclassname(cPtr, $owner); + } +%typemap(in) SWIGTYPE *const& ($*1_ltype temp = 0) +%{ temp = *($1_ltype)&$input; + $1 = ($1_ltype)&temp; %} +%typemap(out) SWIGTYPE *const& +%{ *($1_ltype)&$result = *$1; %} + +/* Typemaps used for the generation of proxy and type wrapper class code */ +%typemap(javabase) SWIGTYPE, SWIGTYPE *, SWIGTYPE &, SWIGTYPE [], SWIGTYPE (CLASS::*) "" +%typemap(javaclassmodifiers) SWIGTYPE, SWIGTYPE *, SWIGTYPE &, SWIGTYPE [], SWIGTYPE (CLASS::*) "public class" +%typemap(javacode) SWIGTYPE, SWIGTYPE *, SWIGTYPE &, SWIGTYPE [], SWIGTYPE (CLASS::*) "" +%typemap(javaimports) SWIGTYPE, SWIGTYPE *, SWIGTYPE &, SWIGTYPE [], SWIGTYPE (CLASS::*) "" +%typemap(javainterfaces) SWIGTYPE, SWIGTYPE *, SWIGTYPE &, SWIGTYPE [], SWIGTYPE (CLASS::*) "" + +/* javabody typemaps */ + +%define SWIG_JAVABODY_METHODS(PTRCTOR_VISIBILITY, CPTR_VISIBILITY, TYPE...) SWIG_JAVABODY_PROXY(PTRCTOR_VISIBILITY, CPTR_VISIBILITY, TYPE) %enddef // legacy name + +%define SWIG_JAVABODY_PROXY(PTRCTOR_VISIBILITY, CPTR_VISIBILITY, TYPE...) +// Base proxy classes +%typemap(javabody) TYPE %{ + private long swigCPtr; + protected boolean swigCMemOwn; + + PTRCTOR_VISIBILITY $javaclassname(long cPtr, boolean cMemoryOwn) { + swigCMemOwn = cMemoryOwn; + swigCPtr = cPtr; + } + + CPTR_VISIBILITY static long getCPtr($javaclassname obj) { + return (obj == null) ? 0 : obj.swigCPtr; + } +%} + +// Derived proxy classes +%typemap(javabody_derived) TYPE %{ + private long swigCPtr; + + PTRCTOR_VISIBILITY $javaclassname(long cPtr, boolean cMemoryOwn) { + super($imclassname.$javaclazznameSWIGUpcast(cPtr), cMemoryOwn); + swigCPtr = cPtr; + } + + CPTR_VISIBILITY static long getCPtr($javaclassname obj) { + return (obj == null) ? 0 : obj.swigCPtr; + } +%} +%enddef + +%define SWIG_JAVABODY_TYPEWRAPPER(PTRCTOR_VISIBILITY, DEFAULTCTOR_VISIBILITY, CPTR_VISIBILITY, TYPE...) +// Typewrapper classes +%typemap(javabody) TYPE *, TYPE &, TYPE [] %{ + private long swigCPtr; + + PTRCTOR_VISIBILITY $javaclassname(long cPtr, boolean futureUse) { + swigCPtr = cPtr; + } + + DEFAULTCTOR_VISIBILITY $javaclassname() { + swigCPtr = 0; + } + + CPTR_VISIBILITY static long getCPtr($javaclassname obj) { + return (obj == null) ? 0 : obj.swigCPtr; + } +%} + +%typemap(javabody) TYPE (CLASS::*) %{ + private String swigCMemberPtr; + + PTRCTOR_VISIBILITY $javaclassname(String cMemberPtr, boolean futureUse) { + swigCMemberPtr = cMemberPtr; + } + + DEFAULTCTOR_VISIBILITY $javaclassname() { + swigCMemberPtr = null; + } + + CPTR_VISIBILITY static String getCMemberPtr($javaclassname obj) { + return obj.swigCMemberPtr; + } +%} +%enddef + +/* Set the default javabody typemaps to use protected visibility. + Use the macros to change to public if using multiple modules. */ +SWIG_JAVABODY_PROXY(protected, protected, SWIGTYPE) +SWIG_JAVABODY_TYPEWRAPPER(protected, protected, protected, SWIGTYPE) + +%typemap(javafinalize) SWIGTYPE %{ + protected void finalize() { + delete(); + } +%} + +/* + * Java constructor typemaps: + * + * The javaconstruct typemap is inserted when a proxy class's constructor is generated. + * This typemap allows control over what code is executed in the constructor as + * well as specifying who owns the underlying C/C++ object. Normally, Java has + * ownership and the underlying C/C++ object is deallocated when the Java object + * is finalized (swigCMemOwn is true.) If swigCMemOwn is false, C/C++ is + * ultimately responsible for deallocating the underlying object's memory. + * + * The SWIG_PROXY_CONSTRUCTOR macro defines the javaconstruct typemap for a proxy + * class for a particular TYPENAME. OWNERSHIP is passed as the value of + * swigCMemOwn to the pointer constructor method. WEAKREF determines which kind + * of Java object reference will be used by the C++ director class (WeakGlobalRef + * vs. GlobalRef.) + * + * The SWIG_DIRECTOR_OWNED macro sets the ownership of director-based proxy + * classes and the weak reference flag to false, meaning that the underlying C++ + * object will be reclaimed by C++. + */ + +%define SWIG_PROXY_CONSTRUCTOR(OWNERSHIP, WEAKREF, TYPENAME...) +%typemap(javaconstruct,directorconnect="\n $imclassname.$javaclazznamedirector_connect(this, swigCPtr, swigCMemOwn, WEAKREF);") TYPENAME { + this($imcall, OWNERSHIP);$directorconnect + } +%enddef + +%define SWIG_DIRECTOR_OWNED(TYPENAME...) +SWIG_PROXY_CONSTRUCTOR(true, false, TYPENAME) +%enddef + +// Set the default for SWIGTYPE: Java owns the C/C++ object. +SWIG_PROXY_CONSTRUCTOR(true, true, SWIGTYPE) + +%typemap(javadestruct, methodname="delete", methodmodifiers="public synchronized") SWIGTYPE { + if (swigCPtr != 0) { + if (swigCMemOwn) { + swigCMemOwn = false; + $jnicall; + } + swigCPtr = 0; + } + } + +%typemap(javadestruct_derived, methodname="delete", methodmodifiers="public synchronized") SWIGTYPE { + if (swigCPtr != 0) { + if (swigCMemOwn) { + swigCMemOwn = false; + $jnicall; + } + swigCPtr = 0; + } + super.delete(); + } + +%typemap(directordisconnect, methodname="swigDirectorDisconnect") SWIGTYPE %{ + protected void $methodname() { + swigCMemOwn = false; + $jnicall; + } +%} + +%typemap(directorowner_release, methodname="swigReleaseOwnership") SWIGTYPE %{ + public void $methodname() { + swigCMemOwn = false; + $jnicall; + } +%} + +%typemap(directorowner_take, methodname="swigTakeOwnership") SWIGTYPE %{ + public void $methodname() { + swigCMemOwn = true; + $jnicall; + } +%} + +/* Java specific directives */ +#define %javaconst(flag) %feature("java:const","flag") +#define %javaconstvalue(value) %feature("java:constvalue",value) +#define %javaenum(wrapapproach) %feature("java:enum","wrapapproach") +#define %javamethodmodifiers %feature("java:methodmodifiers") +#define %javaexception(exceptionclasses) %feature("except",throws=exceptionclasses) +#define %nojavaexception %feature("except","0",throws="") +#define %clearjavaexception %feature("except","",throws="") + +%pragma(java) jniclassclassmodifiers="public class" +%pragma(java) moduleclassmodifiers="public class" + +/* Some ANSI C typemaps */ + +%apply unsigned long { size_t }; +%apply const unsigned long & { const size_t & }; + +/* Array reference typemaps */ +%apply SWIGTYPE & { SWIGTYPE ((&)[ANY]) } + +/* const pointers */ +%apply SWIGTYPE * { SWIGTYPE *const } + +/* String & length */ +%typemap(jni) (char *STRING, size_t LENGTH) "jbyteArray" +%typemap(jtype) (char *STRING, size_t LENGTH) "byte[]" +%typemap(jstype) (char *STRING, size_t LENGTH) "byte[]" +%typemap(javain) (char *STRING, size_t LENGTH) "$javainput" +%typemap(freearg) (char *STRING, size_t LENGTH) "" +%typemap(in) (char *STRING, size_t LENGTH) { + if ($input) { + $1 = (char *) JCALL2(GetByteArrayElements, jenv, $input, 0); + $2 = (size_t) JCALL1(GetArrayLength, jenv, $input); + } else { + $1 = 0; + $2 = 0; + } +} +%typemap(argout) (char *STRING, size_t LENGTH) { + if ($input) JCALL3(ReleaseByteArrayElements, jenv, $input, (jbyte *)$1, 0); +} +%typemap(directorin, descriptor="[B") (char *STRING, size_t LENGTH) { + jbyteArray jb = (jenv)->NewByteArray($2); + (jenv)->SetByteArrayRegion(jb, 0, $2, (jbyte *)$1); + $input = jb; +} +%typemap(directorargout) (char *STRING, size_t LENGTH) +%{(jenv)->GetByteArrayRegion($input, 0, $2, (jbyte *)$1); %} +%apply (char *STRING, size_t LENGTH) { (char *STRING, int LENGTH) } + +/* java keywords */ +%include <javakw.swg> + +// Default enum handling +%include <enumtypesafe.swg> + diff --git a/tests/examplefiles/swig_std_vector.i b/tests/examplefiles/swig_std_vector.i new file mode 100644 index 00000000..baecf850 --- /dev/null +++ b/tests/examplefiles/swig_std_vector.i @@ -0,0 +1,225 @@ +// +// std::vector +// + +%include <std_container.i> + +// Vector + +%define %std_vector_methods(vector...) + %std_sequence_methods(vector) + + void reserve(size_type n); + size_type capacity() const; +%enddef + + +%define %std_vector_methods_val(vector...) + %std_sequence_methods_val(vector) + + void reserve(size_type n); + size_type capacity() const; +%enddef + + +// ------------------------------------------------------------------------ +// std::vector +// +// The aim of all that follows would be to integrate std::vector with +// as much as possible, namely, to allow the user to pass and +// be returned tuples or lists. +// const declarations are used to guess the intent of the function being +// exported; therefore, the following rationale is applied: +// +// -- f(std::vector<T>), f(const std::vector<T>&): +// the parameter being read-only, either a sequence or a +// previously wrapped std::vector<T> can be passed. +// -- f(std::vector<T>&), f(std::vector<T>*): +// the parameter may be modified; therefore, only a wrapped std::vector +// can be passed. +// -- std::vector<T> f(), const std::vector<T>& f(): +// the vector is returned by copy; therefore, a sequence of T:s +// is returned which is most easily used in other functions +// -- std::vector<T>& f(), std::vector<T>* f(): +// the vector is returned by reference; therefore, a wrapped std::vector +// is returned +// -- const std::vector<T>* f(), f(const std::vector<T>*): +// for consistency, they expect and return a plain vector pointer. +// ------------------------------------------------------------------------ + +%{ +#include <vector> +%} + +// exported classes + + +namespace std { + + template<class _Tp, class _Alloc = allocator< _Tp > > + class vector { + public: + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef _Tp value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef _Tp& reference; + typedef const _Tp& const_reference; + typedef _Alloc allocator_type; + + %traits_swigtype(_Tp); + %traits_enum(_Tp); + + %fragment(SWIG_Traits_frag(std::vector<_Tp, _Alloc >), "header", + fragment=SWIG_Traits_frag(_Tp), + fragment="StdVectorTraits") { + namespace swig { + template <> struct traits<std::vector<_Tp, _Alloc > > { + typedef pointer_category category; + static const char* type_name() { + return "std::vector<" #_Tp "," #_Alloc " >"; + } + }; + } + } + + %typemap_traits_ptr(SWIG_TYPECHECK_VECTOR, std::vector<_Tp, _Alloc >); + +#ifdef %swig_vector_methods + // Add swig/language extra methods + %swig_vector_methods(std::vector<_Tp, _Alloc >); +#endif + + %std_vector_methods(vector); + }; + + // *** + // This specialization should disappear or get simplified when + // a 'const SWIGTYPE*&' can be defined + // *** + template<class _Tp, class _Alloc > + class vector<_Tp*, _Alloc > { + public: + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef _Tp* value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef value_type reference; + typedef value_type const_reference; + typedef _Alloc allocator_type; + + %traits_swigtype(_Tp); + + %fragment(SWIG_Traits_frag(std::vector<_Tp*, _Alloc >), "header", + fragment=SWIG_Traits_frag(_Tp), + fragment="StdVectorTraits") { + namespace swig { + template <> struct traits<std::vector<_Tp*, _Alloc > > { + typedef value_category category; + static const char* type_name() { + return "std::vector<" #_Tp " *," #_Alloc " >"; + } + }; + } + } + + %typemap_traits_ptr(SWIG_TYPECHECK_VECTOR, std::vector<_Tp*, _Alloc >); + +#ifdef %swig_vector_methods_val + // Add swig/language extra methods + %swig_vector_methods_val(std::vector<_Tp*, _Alloc >); +#endif + + %std_vector_methods_val(vector); + }; + + // *** + // const pointer specialization + // *** + template<class _Tp, class _Alloc > + class vector<_Tp const *, _Alloc > { + public: + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef _Tp const * value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef value_type reference; + typedef value_type const_reference; + typedef _Alloc allocator_type; + + %traits_swigtype(_Tp); + + %fragment(SWIG_Traits_frag(std::vector<_Tp const*, _Alloc >), "header", + fragment=SWIG_Traits_frag(_Tp), + fragment="StdVectorTraits") { + namespace swig { + template <> struct traits<std::vector<_Tp const*, _Alloc > > { + typedef value_category category; + static const char* type_name() { + return "std::vector<" #_Tp " const*," #_Alloc " >"; + } + }; + } + } + + %typemap_traits_ptr(SWIG_TYPECHECK_VECTOR, std::vector<_Tp const*, _Alloc >); + +#ifdef %swig_vector_methods_val + // Add swig/language extra methods + %swig_vector_methods_val(std::vector<_Tp const*, _Alloc >); +#endif + + %std_vector_methods_val(vector); + }; + + // *** + // bool specialization + // *** + + template<class _Alloc > + class vector<bool,_Alloc > { + public: + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef bool value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef value_type reference; + typedef value_type const_reference; + typedef _Alloc allocator_type; + + %traits_swigtype(bool); + + %fragment(SWIG_Traits_frag(std::vector<bool, _Alloc >), "header", + fragment=SWIG_Traits_frag(bool), + fragment="StdVectorTraits") { + namespace swig { + template <> struct traits<std::vector<bool, _Alloc > > { + typedef value_category category; + static const char* type_name() { + return "std::vector<bool, _Alloc >"; + } + }; + } + } + + %typemap_traits_ptr(SWIG_TYPECHECK_VECTOR, std::vector<bool, _Alloc >); + + +#ifdef %swig_vector_methods_val + // Add swig/language extra methods + %swig_vector_methods_val(std::vector<bool, _Alloc >); +#endif + + %std_vector_methods_val(vector); + +#if defined(SWIG_STD_MODERN_STL) && !defined(SWIG_STD_NOMODERN_STL) + void flip(); +#endif + + }; + +} diff --git a/tests/examplefiles/test.bb b/tests/examplefiles/test.bb new file mode 100644 index 00000000..026ef22a --- /dev/null +++ b/tests/examplefiles/test.bb @@ -0,0 +1,95 @@ +
+;foobar!
+
+;Include "blurg/blurg.bb"
+
+Const ca = $10000000 ; Hex
+Const cb = %10101010 ; Binary
+Global ga$ = "blargh"
+Local a = 124, b$ = "abcdef"
+
+Function name_123#(zorp$, ll = False, blah#, waffles% = 100)
+ Return 235.7804 ; comment
+End Function
+Function TestString$()
+End Function
+
+Function hub(blah$, abc = Pi)
+End Function
+Function Blar%()
+ Local aa %, ab # ,ac #, ad# ,ae$,af% ; Intentional mangling
+ Local ba#, bb.TBlarf , bc%,bd#,be. TFooBar,ff = True
+End Function
+
+abc()
+
+Function abc()
+ Print "abc" ; I cannot find a way to parse these as function calls without messing something up
+ Print ; Anyhow, they're generally not used in this way
+ Goto Eww_Goto
+ .Eww_Goto
+End Function
+
+Type TBlarf
+End Type
+
+Type TFooBar
+End Type
+
+Local myinst.MyClass = New MyClass
+TestMethod(myinst)
+
+Type MyClass
+
+ Field m_foo.MyClass
+ Field m_bar.MyClass
+
+; abc
+; def
+End Type
+
+Function TestMethod(self.MyClass) ; foobar
+ self\m_foo = self
+ self\m_bar = Object.MyClass(Handle self\m_foo)
+ Yell self\m_foo\m_bar\m_foo\m_bar
+End Function
+
+Function Yell(self.MyClass)
+ Print("huzzah!")
+End Function
+
+Function Wakka$(foo$)
+ Return foo + "bar"
+End Function
+
+
+Print("blah " + "blah " + "blah.")
+
+Local i : For i = 0 To 10 Step 1
+ Print("Index: " + i)
+Next
+Local array$[5]
+array[0] = "foo": array[1] = "bar":array[2] = "11":array[3] = "22":array[4] = "33"
+For i = 0 To 4
+ Local value$ = array[i]
+ Print("Value: " + value)
+Next
+
+Local foobar = Not (1 Or (2 And (4 Shl 5 Shr 6)) Sar 7) Mod (8+2)
+Local az = 1234567890
+az = az + 1
+az = az - 2
+az = az* 3
+az = az/ 4
+az = az And 5
+az = az Or 6
+az= ~ 7
+az = az Shl 8
+az= az Shr 9
+az = az Sar 10
+az = az Mod 11
+az = ((10-5+2/4*2)>(((8^2)) < 2)) And 12 Or 2
+
+
+;~IDEal Editor Parameters:
+;~C#Blitz3D
\ No newline at end of file diff --git a/tests/examplefiles/test.ebnf b/tests/examplefiles/test.ebnf new file mode 100644 index 00000000..a96171b0 --- /dev/null +++ b/tests/examplefiles/test.ebnf @@ -0,0 +1,31 @@ +letter = "A" | "B" | "C" | "D" | "E" | "F" | "G" + | "H" | "I" | "J" | "K" | "L" | "M" | "N" + | "O" | "P" | "Q" | "R" | "S" | "T" | "U" + | "V" | "W" | "X" | "Y" | "Z" ; +digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ; +symbol = "[" | "]" | "{" | "}" | "(" | ")" | "<" | ">" + | "'" | '"' | "=" | "|" | "." | "," | ";" ; +character = letter | digit | symbol | " " ; + +identifier = letter , { letter | digit | " " } ; +terminal = "'" , character , { character } , "'" + | '"' , character , { character } , '"' ; + +special = "?" , any , "?" ; + +comment = (* this is a comment "" *) "(*" , any-symbol , "*)" ; +any-symbol = ? any visible character ? ; (* ? ... ? *) + +lhs = identifier ; +rhs = identifier + | terminal + | comment , rhs + | rhs , comment + | "[" , rhs , "]" + | "{" , rhs , "}" + | "(" , rhs , ")" + | rhs , "|" , rhs + | rhs , "," , rhs ; + +rule = lhs , "=" , rhs , ";" | comment ; +grammar = { rule } ; diff --git a/tests/examplefiles/test.p6 b/tests/examplefiles/test.p6 new file mode 100644 index 00000000..3d12b56c --- /dev/null +++ b/tests/examplefiles/test.p6 @@ -0,0 +1,252 @@ +#!/usr/bin/env perl6 + +use v6; + +my $string = 'I look like a # comment!'; + +if $string eq 'foo' { + say 'hello'; +} + +regex http-verb { + 'GET' + | 'POST' + | 'PUT' + | 'DELETE' + | 'TRACE' + | 'OPTIONS' + | 'HEAD' +} + +# a sample comment + +say 'Hello from Perl 6!' + + +#`{ +multi-line comment! +} + +say 'here'; + +#`( +multi-line comment! +) + +say 'here'; + +#`{{{ +I'm a special comment! +}}} + +say 'there'; + +#`{{ +I'm { even } specialer! +}} + +say 'there'; + +#`{{ +does {{nesting}} work? +}} + +#`«< +trying mixed delimiters +» + +my $string = qq<Hooray, arbitrary delimiter!>; +my $string = qq«Hooray, arbitrary delimiter!»; +my $string = q <now with whitespace!>; +my $string = qq<<more strings>>; + +my %hash := Hash.new; + +=begin pod + +Here's some POD! Wooo + +=end pod + +=for Testing + This is POD (see? role isn't highlighted) + +say('this is not!'); + +=table + Of role things + +say('not in your table'); +#= A single line declarator "block" (with a keyword like role) +#| Another single line declarator "block" (with a keyword like role) +#={ + A declarator block (with a keyword like role) + } +#|{ + Another declarator block (with a keyword like role) + } +#= { A single line declarator "block" with a brace (with a keyword like role) +#=« + More declarator blocks! (with a keyword like role) + » +#|« + More declarator blocks! (with a keyword like role) + » + +say 'Moar code!'; + +my $don't = 16; + +sub don't($x) { + !$x +} + +say don't 'foo'; + +my %hash = ( + :foo(1), +); + +say %hash<foo>; +say %hash<<foo>>; +say %hash«foo»; + +say %*hash<foo>; +say %*hash<<foo>>; +say %*hash«foo»; + +say $<todo>; +say $<todo>; + +for (@A Z @B) -> $a, $b { + say $a + $b; +} + +Q:PIR { + .loadlib "somelib" +} + +my $longstring = q/ + lots + of + text +/; + +my $heredoc = q:to/END_SQL/; +SELECT * FROM Users +WHERE first_name = 'Rob' +END_SQL +my $hello; + +# Fun with regexen + +if 'food' ~~ /foo/ { + say 'match!' +} + +my $re = /foo/; +my $re2 = m/ foo /; +my $re3 = m:i/ FOO /; + +call-a-sub(/ foo /); +call-a-sub(/ foo \/ bar /); + +my $re4 = rx/something | something-else/; +my $result = ms/regexy stuff/; +my $sub0 = s/regexy stuff/more stuff/; +my $sub = ss/regexy stuff/more stuff/; +my $trans = tr/regexy stuff/more stuff/; + +my @values = <a b c d>; +call-sub(<a b c d>); +call-sub <a b c d>; + +my $result = $a < $b; + +for <a b c d> -> $letter { + say $letter; +} + +sub test-sub { + say @_; + say $!; + say $/; + say $0; + say $1; + say @*ARGS; + say $*ARGFILES; + say &?BLOCK; + say ::?CLASS; + say $?CLASS; + say @=COMMENT; + say %?CONFIG; + say $*CWD; + say $=data; + say %?DEEPMAGIC; + say $?DISTRO; + say $*DISTRO; + say $*EGID; + say %*ENV; + say $*ERR; + say $*EUID; + say $*EXECUTABLE_NAME; + say $?FILE; + say $?GRAMMAR; + say $*GID; + say $*IN; + say @*INC; + say %?LANG; + say $*LANG; + say $?LINE; + say %*META-ARGS; + say $?MODULE; + say %*OPTS; + say %*OPT; + say $?KERNEL; + say $*KERNEL; + say $*OUT; + say $?PACKAGE; + say $?PERL; + say $*PERL; + say $*PID; + say %=pod; + say $*PROGRAM_NAME; + say %*PROTOCOLS; + say ::?ROLE; + say $?ROLE; + say &?ROUTINE; + say $?SCOPE; + say $*TZ; + say $*UID; + say $?USAGE; + say $?VM; + say $?XVM; +} + +say <a b c>; + +my $perl5_re = m:P5/ fo{2} /; +my $re5 = rx«something | something-else»; + +my $M := %*COMPILING<%?OPTIONS><M>; + +say $M; + +sub regex-name { ... } +my $pair = role-name => 'foo'; +$pair = rolesque => 'foo'; + +my sub something(Str:D $value) { ... } + +my $s = q«< +some +string +stuff +»; + +my $regex = m«< some chars »; +# after + +say $/<foo><bar>; + +roleq; diff --git a/tests/examplefiles/type.lisp b/tests/examplefiles/type.lisp index 9c769379..c02c29df 100644 --- a/tests/examplefiles/type.lisp +++ b/tests/examplefiles/type.lisp @@ -1200,3 +1200,19 @@ Henry Baker: (unless (clos::funcallable-instance-p #'clos::class-name) (fmakunbound 'clos::class-name)) + + +(keywordp :junk) + T + +(keywordp ::junk) + T + +(symbol-name ::junk) + "JUNK" + +(symbol-name :#junk) + "#JUNK" + +(symbol-name :#.junk) + "#.JUNK" diff --git a/tests/test_lexers_other.py b/tests/test_lexers_other.py new file mode 100644 index 00000000..d3cfa246 --- /dev/null +++ b/tests/test_lexers_other.py @@ -0,0 +1,68 @@ +# -*- coding: utf-8 -*- +""" + Tests for other lexers + ~~~~~~~~~~~~~~~~~~~~~~ + + :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import glob +import os +import unittest + +from pygments.lexers import guess_lexer +from pygments.lexers.other import RexxLexer + + +def _exampleFilePath(filename): + return os.path.join(os.path.dirname(__file__), 'examplefiles', filename) + + +class AnalyseTextTest(unittest.TestCase): + def _testCanRecognizeAndGuessExampleFiles(self, lexer): + assert lexer is not None + + for pattern in lexer.filenames: + exampleFilesPattern = _exampleFilePath(pattern) + for exampleFilePath in glob.glob(exampleFilesPattern): + exampleFile = open(exampleFilePath, 'rb') + try: + text = exampleFile.read() + probability = lexer.analyse_text(text) + self.assertTrue(probability > 0, + '%s must recognize %r' % ( + lexer.name, exampleFilePath)) + guessedLexer = guess_lexer(text) + self.assertEqual(guessedLexer.name, lexer.name) + finally: + exampleFile.close() + + def testCanRecognizeAndGuessExampleFiles(self): + self._testCanRecognizeAndGuessExampleFiles(RexxLexer) + + +class RexxLexerTest(unittest.TestCase): + def testCanGuessFromText(self): + self.assertAlmostEqual(0.01, + RexxLexer.analyse_text('/* */')) + self.assertAlmostEqual(1.0, + RexxLexer.analyse_text('''/* Rexx */ + say "hello world"''')) + self.assertLess(0.5, + RexxLexer.analyse_text('/* */\n' + 'hello:pRoceduRe\n' + ' say "hello world"')) + self.assertLess(0.2, + RexxLexer.analyse_text('''/* */ + if 1 > 0 then do + say "ok" + end + else do + say "huh?" + end''')) + self.assertLess(0.2, + RexxLexer.analyse_text('''/* */ + greeting = "hello world!" + parse value greeting "hello" name "!" + say name''')) |