summaryrefslogtreecommitdiff
path: root/doc/docs/lexerdevelopment.rst
diff options
context:
space:
mode:
Diffstat (limited to 'doc/docs/lexerdevelopment.rst')
-rw-r--r--doc/docs/lexerdevelopment.rst45
1 files changed, 42 insertions, 3 deletions
diff --git a/doc/docs/lexerdevelopment.rst b/doc/docs/lexerdevelopment.rst
index fd6e76b9..63bd01a3 100644
--- a/doc/docs/lexerdevelopment.rst
+++ b/doc/docs/lexerdevelopment.rst
@@ -88,8 +88,47 @@ one.
Adding and testing a new lexer
==============================
-Using a lexer that is not part of Pygments can be done via the Python API. You
-can import and instantiate the lexer, and pass it to :func:`pygments.highlight`.
+The easiest way to use a new lexer is to use Pygments' support for loading
+the lexer from a file relative to your current directory.
+
+First, change the name of your lexer class to CustomLexer:
+
+.. code-block:: python
+
+ from pygments.lexer import RegexLexer
+ from pygments.token import *
+
+ class CustomLexer(RegexLexer):
+ """All your lexer code goes here!"""
+
+Then you can load the lexer from the command line with the additional
+flag ``-x``:
+
+.. code-block:: console
+
+ $ pygmentize -l your_lexer_file.py -x
+
+To specify a class name other than CustomLexer, append it with a colon:
+
+.. code-block:: console
+
+ $ pygmentize -l your_lexer.py:SomeLexer -x
+
+Or, using the Python API:
+
+.. code-block:: python
+
+ # For a lexer named CustomLexer
+ your_lexer = load_lexer_from_file(filename, **options)
+
+ # For a lexer named MyNewLexer
+ your_named_lexer = load_lexer_from_file(filename, "MyNewLexer", **options)
+
+When loading custom lexers and formatters, be extremely careful to use only
+trusted files; Pygments will perform the equivalent of ``eval`` on them.
+
+If you only want to use your lexer with the Pygments API, you can import and
+instantiate the lexer yourself, then pass it to :func:`pygments.highlight`.
To prepare your new lexer for inclusion in the Pygments distribution, so that it
will be found when passing filenames or lexer aliases from the command line, you
@@ -361,7 +400,7 @@ There are a few more things you can do with states:
tokens = {...}
def get_tokens_unprocessed(self, text, stack=('root', 'otherstate')):
- for item in RegexLexer.get_tokens_unprocessed(text, stack):
+ for item in RegexLexer.get_tokens_unprocessed(self, text, stack):
yield item
Some lexers like the `PhpLexer` use this to make the leading ``<?php``