diff options
Diffstat (limited to 'doc/docs/lexerdevelopment.rst')
-rw-r--r-- | doc/docs/lexerdevelopment.rst | 45 |
1 files changed, 42 insertions, 3 deletions
diff --git a/doc/docs/lexerdevelopment.rst b/doc/docs/lexerdevelopment.rst index fd6e76b9..63bd01a3 100644 --- a/doc/docs/lexerdevelopment.rst +++ b/doc/docs/lexerdevelopment.rst @@ -88,8 +88,47 @@ one. Adding and testing a new lexer ============================== -Using a lexer that is not part of Pygments can be done via the Python API. You -can import and instantiate the lexer, and pass it to :func:`pygments.highlight`. +The easiest way to use a new lexer is to use Pygments' support for loading +the lexer from a file relative to your current directory. + +First, change the name of your lexer class to CustomLexer: + +.. code-block:: python + + from pygments.lexer import RegexLexer + from pygments.token import * + + class CustomLexer(RegexLexer): + """All your lexer code goes here!""" + +Then you can load the lexer from the command line with the additional +flag ``-x``: + +.. code-block:: console + + $ pygmentize -l your_lexer_file.py -x + +To specify a class name other than CustomLexer, append it with a colon: + +.. code-block:: console + + $ pygmentize -l your_lexer.py:SomeLexer -x + +Or, using the Python API: + +.. code-block:: python + + # For a lexer named CustomLexer + your_lexer = load_lexer_from_file(filename, **options) + + # For a lexer named MyNewLexer + your_named_lexer = load_lexer_from_file(filename, "MyNewLexer", **options) + +When loading custom lexers and formatters, be extremely careful to use only +trusted files; Pygments will perform the equivalent of ``eval`` on them. + +If you only want to use your lexer with the Pygments API, you can import and +instantiate the lexer yourself, then pass it to :func:`pygments.highlight`. To prepare your new lexer for inclusion in the Pygments distribution, so that it will be found when passing filenames or lexer aliases from the command line, you @@ -361,7 +400,7 @@ There are a few more things you can do with states: tokens = {...} def get_tokens_unprocessed(self, text, stack=('root', 'otherstate')): - for item in RegexLexer.get_tokens_unprocessed(text, stack): + for item in RegexLexer.get_tokens_unprocessed(self, text, stack): yield item Some lexers like the `PhpLexer` use this to make the leading ``<?php`` |