Date: Fri, 29 Jun 2007 03:29:36 +0000 Subject: renamed .html to .htm to avoid collisions with the docutils-update website updater script; it was sending me email because the code-block directive caused error messages; this is kindof a hack, but it should work for now; Guenther, if the changed extensions are a problem for you, please feel free to revert (or drop me a line) git-svn-id: http://svn.code.sf.net/p/docutils/code/trunk@5305 929543f6-e4f2-0310-98a6-ba3bd3dd1d04 --- .../code-block-directive/docs/for-else-test.py.htm | 185 ++++++++++++ .../docs/for-else-test.py.html | 185 ------------ .../code-block-directive/docs/myfunction.py.htm | 29 ++ .../code-block-directive/docs/myfunction.py.html | 29 -- .../docs/pygments_code_block_directive-bunt.py.htm | 314 +++++++++++++++++++++ .../pygments_code_block_directive-bunt.py.html | 314 --------------------- .../docs/pygments_code_block_directive.py.htm | 263 +++++++++++++++++ .../docs/pygments_code_block_directive.py.html | 263 ----------------- 8 files changed, 791 insertions(+), 791 deletions(-) create mode 100644 sandbox/code-block-directive/docs/for-else-test.py.htm delete mode 100644 sandbox/code-block-directive/docs/for-else-test.py.html create mode 100644 sandbox/code-block-directive/docs/myfunction.py.htm delete mode 100644 sandbox/code-block-directive/docs/myfunction.py.html create mode 100644 sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.htm delete mode 100644 sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.html create mode 100644 sandbox/code-block-directive/docs/pygments_code_block_directive.py.htm delete mode 100644 sandbox/code-block-directive/docs/pygments_code_block_directive.py.html (limited to 'sandbox/code-block-directive') diff --git a/sandbox/code-block-directive/docs/for-else-test.py.htm b/sandbox/code-block-directive/docs/for-else-test.py.htm new file mode 100644 index 000000000..1e6057be1 --- /dev/null +++ b/sandbox/code-block-directive/docs/for-else-test.py.htm @@ -0,0 +1,185 @@ + + + + + + +Example for syntax highlight with Pygments + + + +

Example for syntax highlight with Pygments

Translate this document to HTML with a pygments enhanced frontend:

+rst2html-pygments --stylesheet=pygments-default.css
+

or to LaTeX with:

+rst2latex-pygments --stylesheet=pygments-default.sty
+

to gain syntax highlight in the output.

Convert between text <-> code source formats with:

+pylit --code-block-marker='.. code-block:: python'
+

Run the doctests with:

+pylit --doctest for-else-test.py
+

for-else-test

Test the flow in a for loop with else statement.

First define a simple for loop.

def loop1(iterable):
+    """simple for loop with `else` statement"""
+    for i in iterable:
+        print i
+    else:
+        print "iterable empty"
+    print "Ende"
+

Now test it:

The first test runs as I expect: iterator empty -> else clause applies:

>>> execfile('for-else-test.py')
+>>> loop1(range(0))
+iterable empty
+Ende
+

However, the else clause even runs if the iterator is not empty in the first +place but after it is "spent":

>>> loop1(range(3))
+0
+1
+2
+iterable empty
+Ende
+

It seems like the else clause can only be prevented, if we break out of +the loop. Let's try

def loop2(iterable):
+    """for loop with `break` and `else` statement"""
+    for i in iterable:
+        print i
+        break
+    else:
+        print "iterable empty"
+    print "Ende"
+

And indeed, the else clause is skipped after breaking out of the loop:

>>> loop2(range(3))
+0
+Ende
+

The empty iterator runs as expected:

>>> loop2(range(0))
+iterable empty
+Ende
+

+ + + diff --git a/sandbox/code-block-directive/docs/for-else-test.py.html b/sandbox/code-block-directive/docs/for-else-test.py.html deleted file mode 100644 index 1e6057be1..000000000 --- a/sandbox/code-block-directive/docs/for-else-test.py.html +++ /dev/null @@ -1,185 +0,0 @@ - - - - - - -Example for syntax highlight with Pygments - - - -

Example for syntax highlight with Pygments

Translate this document to HTML with a pygments enhanced frontend:

-rst2html-pygments --stylesheet=pygments-default.css
-

or to LaTeX with:

-rst2latex-pygments --stylesheet=pygments-default.sty
-

to gain syntax highlight in the output.

Convert between text <-> code source formats with:

-pylit --code-block-marker='.. code-block:: python'
-

Run the doctests with:

-pylit --doctest for-else-test.py
-

for-else-test

Test the flow in a for loop with else statement.

First define a simple for loop.

def loop1(iterable):
-    """simple for loop with `else` statement"""
-    for i in iterable:
-        print i
-    else:
-        print "iterable empty"
-    print "Ende"
-

Now test it:

The first test runs as I expect: iterator empty -> else clause applies:

>>> execfile('for-else-test.py')
->>> loop1(range(0))
-iterable empty
-Ende
-

However, the else clause even runs if the iterator is not empty in the first -place but after it is "spent":

>>> loop1(range(3))
-0
-1
-2
-iterable empty
-Ende
-

It seems like the else clause can only be prevented, if we break out of -the loop. Let's try

def loop2(iterable):
-    """for loop with `break` and `else` statement"""
-    for i in iterable:
-        print i
-        break
-    else:
-        print "iterable empty"
-    print "Ende"
-

And indeed, the else clause is skipped after breaking out of the loop:

>>> loop2(range(3))
-0
-Ende
-

The empty iterator runs as expected:

>>> loop2(range(0))
-iterable empty
-Ende
-

- - - diff --git a/sandbox/code-block-directive/docs/myfunction.py.htm b/sandbox/code-block-directive/docs/myfunction.py.htm new file mode 100644 index 000000000..edc700246 --- /dev/null +++ b/sandbox/code-block-directive/docs/myfunction.py.htm @@ -0,0 +1,29 @@ + + + + + + + + + + +

This is a test of the new code-block directive:

+ +

+def my_function():
+    "just a test"
+    print 8/2
+
+

+ + + diff --git a/sandbox/code-block-directive/docs/myfunction.py.html b/sandbox/code-block-directive/docs/myfunction.py.html deleted file mode 100644 index edc700246..000000000 --- a/sandbox/code-block-directive/docs/myfunction.py.html +++ /dev/null @@ -1,29 +0,0 @@ - - - - - - - - - - -

This is a test of the new code-block directive:

- -

-def my_function():
-    "just a test"
-    print 8/2
-
-

- - - diff --git a/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.htm b/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.htm new file mode 100644 index 000000000..78bb809e9 --- /dev/null +++ b/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.htm @@ -0,0 +1,314 @@ + + + + + + + + + + + + + +

+ +++ + + + + + + + +

Author:	a Pygments author\|contributor; Felix Wiemann; Guenter Milde
Date:	2007-06-08
Copyright:	This module has been placed in the public domain.

+ +

This is a merge of Using Pygments in ReST documents from the pygments +documentation, and a proof of concept by Felix Wiemann.

+ ++++ + + + + + + + + + + + + + + + + + +

2007-06-01	Removed redundancy from class values.
2007-06-04	Merge of successive tokens of same type +(code taken from pygments.formatters.others).
2007-06-05	Separate docutils formatter script +Use pygments' CSS class names (like the html formatter) +allowing the use of pygments-produced style sheets.
2007-06-07	Merge in the formatting of the parsed tokens +(misnamed as docutils_formatter) as class DocutilsInterface
2007-06-08	Failsave implementation (fallback to a standard literal block +if pygments not found)

+ +

+"""Define and register a code-block directive using pygments
+"""
+
+

Requirements

+from docutils import nodes
+from docutils.parsers.rst import directives
+try:
+    import pygments
+    from pygments.lexers import get_lexer_by_name
+    from pygments.formatters.html import _get_ttype_class
+except ImportError:
+    pass
+
+

Customisation

Do not insert inline nodes for the following tokens. +(You could add e.g. Token.Punctuation like ['', 'p'].)

+unstyled_tokens = ['']
+
+

DocutilsInterface

This interface class combines code from +pygments.formatters.html and pygments.formatters.others.

It does not require anything of docutils and could also become a part of +pygments

+class DocutilsInterface(object):
+    """Parse `code` string and yield "classified" tokens.
+
+    Arguments
+
+      code     -- string of source code to parse
+      language -- formal language the code is written in.
+
+    Merge subsequent tokens of the same token-type.
+
+    Yields the tokens as ``(ttype_class, value)`` tuples,
+    where ttype_class is taken from pygments.token.STANDARD_TYPES and
+    corresponds to the class argument used in pygments html output.
+
+    """
+
+    def __init__(self, code, language):
+        self.code = code
+        self.language = language
+
+    def lex(self):
+        # Get lexer for language (use text as fallback)
+        try:
+            lexer = get_lexer_by_name(self.language)
+        except ValueError:
+            # info: "no pygments lexer for %s, using 'text'"%self.language
+            lexer = get_lexer_by_name('text')
+        return pygments.lex(self.code, lexer)
+
+
+    def join(self, tokens):
+        """join subsequent tokens of same token-type
+        """
+        tokens = iter(tokens)
+        (lasttype, lastval) = tokens.next()
+        for ttype, value in tokens:
+            if ttype is lasttype:
+                lastval += value
+            else:
+                yield(lasttype, lastval)
+                (lasttype, lastval) = (ttype, value)
+        yield(lasttype, lastval)
+
+    def __iter__(self):
+        """parse code string and yield "clasified" tokens
+        """
+        try:
+            tokens = self.lex()
+        except IOError:
+            print "INFO: Pygments lexer not found, using fallback"
+            # TODO: write message to INFO
+            yield ('', self.code)
+            return
+
+        for ttype, value in self.join(tokens):
+            yield (_get_ttype_class(ttype), value)
+
+

code_block_directive

+def code_block_directive(name, arguments, options, content, lineno,
+                       content_offset, block_text, state, state_machine):
+    """parse and classify content of a code_block
+    """
+    language = arguments[0]
+    # create a literal block element and set class argument
+    code_block = nodes.literal_block(classes=["code-block", language])
+
+    # parse content with pygments and add to code_block element
+    for cls, value in DocutilsInterface(u'\n'.join(content), language):
+        if cls in unstyled_tokens:
+            # insert as Text to decrease the verbosity of the output.
+            code_block += nodes.Text(value, value)
+        else:
+            code_block += nodes.inline(value, value, classes=[cls])
+
+    return [code_block]
+
+

Register Directive

+code_block_directive.arguments = (1, 0, 1)
+code_block_directive.content = 1
+directives.register_directive('code-block', code_block_directive)
+
+

Test output

If called from the command line, call the docutils publisher to render the +input

+if __name__ == '__main__':
+    from docutils.core import publish_cmdline, default_description
+    description = "code-block directive test output" + default_description
+    try:
+        import locale
+        locale.setlocale(locale.LC_ALL, '')
+    except:
+        pass
+    # Uncomment the desired output format:
+    publish_cmdline(writer_name='pseudoxml', description=description)
+    # publish_cmdline(writer_name='xml', description=description)
+    # publish_cmdline(writer_name='html', description=description)
+    # publish_cmdline(writer_name='latex', description=description)
+    # publish_cmdline(writer_name='newlatex2e', description=description)
+
+

+ + + diff --git a/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.html b/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.html deleted file mode 100644 index 78bb809e9..000000000 --- a/sandbox/code-block-directive/docs/pygments_code_block_directive-bunt.py.html +++ /dev/null @@ -1,314 +0,0 @@ - - - - - - - - - - - - - -

- --- - - - - - - - -

Author:	a Pygments author\|contributor; Felix Wiemann; Guenter Milde
Date:	2007-06-08
Copyright:	This module has been placed in the public domain.

- -

This is a merge of Using Pygments in ReST documents from the pygments -documentation, and a proof of concept by Felix Wiemann.

- ---- - - - - - - - - - - - - - - - - - -

2007-06-01	Removed redundancy from class values.
2007-06-04	Merge of successive tokens of same type -(code taken from pygments.formatters.others).
2007-06-05	Separate docutils formatter script -Use pygments' CSS class names (like the html formatter) -allowing the use of pygments-produced style sheets.
2007-06-07	Merge in the formatting of the parsed tokens -(misnamed as docutils_formatter) as class DocutilsInterface
2007-06-08	Failsave implementation (fallback to a standard literal block -if pygments not found)

- -

-"""Define and register a code-block directive using pygments
-"""
-
-

Requirements

-from docutils import nodes
-from docutils.parsers.rst import directives
-try:
-    import pygments
-    from pygments.lexers import get_lexer_by_name
-    from pygments.formatters.html import _get_ttype_class
-except ImportError:
-    pass
-
-

Customisation

Do not insert inline nodes for the following tokens. -(You could add e.g. Token.Punctuation like ['', 'p'].)

-unstyled_tokens = ['']
-
-

DocutilsInterface

This interface class combines code from -pygments.formatters.html and pygments.formatters.others.

It does not require anything of docutils and could also become a part of -pygments

-class DocutilsInterface(object):
-    """Parse `code` string and yield "classified" tokens.
-
-    Arguments
-
-      code     -- string of source code to parse
-      language -- formal language the code is written in.
-
-    Merge subsequent tokens of the same token-type.
-
-    Yields the tokens as ``(ttype_class, value)`` tuples,
-    where ttype_class is taken from pygments.token.STANDARD_TYPES and
-    corresponds to the class argument used in pygments html output.
-
-    """
-
-    def __init__(self, code, language):
-        self.code = code
-        self.language = language
-
-    def lex(self):
-        # Get lexer for language (use text as fallback)
-        try:
-            lexer = get_lexer_by_name(self.language)
-        except ValueError:
-            # info: "no pygments lexer for %s, using 'text'"%self.language
-            lexer = get_lexer_by_name('text')
-        return pygments.lex(self.code, lexer)
-
-
-    def join(self, tokens):
-        """join subsequent tokens of same token-type
-        """
-        tokens = iter(tokens)
-        (lasttype, lastval) = tokens.next()
-        for ttype, value in tokens:
-            if ttype is lasttype:
-                lastval += value
-            else:
-                yield(lasttype, lastval)
-                (lasttype, lastval) = (ttype, value)
-        yield(lasttype, lastval)
-
-    def __iter__(self):
-        """parse code string and yield "clasified" tokens
-        """
-        try:
-            tokens = self.lex()
-        except IOError:
-            print "INFO: Pygments lexer not found, using fallback"
-            # TODO: write message to INFO
-            yield ('', self.code)
-            return
-
-        for ttype, value in self.join(tokens):
-            yield (_get_ttype_class(ttype), value)
-
-

code_block_directive

-def code_block_directive(name, arguments, options, content, lineno,
-                       content_offset, block_text, state, state_machine):
-    """parse and classify content of a code_block
-    """
-    language = arguments[0]
-    # create a literal block element and set class argument
-    code_block = nodes.literal_block(classes=["code-block", language])
-
-    # parse content with pygments and add to code_block element
-    for cls, value in DocutilsInterface(u'\n'.join(content), language):
-        if cls in unstyled_tokens:
-            # insert as Text to decrease the verbosity of the output.
-            code_block += nodes.Text(value, value)
-        else:
-            code_block += nodes.inline(value, value, classes=[cls])
-
-    return [code_block]
-
-

Register Directive

-code_block_directive.arguments = (1, 0, 1)
-code_block_directive.content = 1
-directives.register_directive('code-block', code_block_directive)
-
-

Test output

If called from the command line, call the docutils publisher to render the -input

-if __name__ == '__main__':
-    from docutils.core import publish_cmdline, default_description
-    description = "code-block directive test output" + default_description
-    try:
-        import locale
-        locale.setlocale(locale.LC_ALL, '')
-    except:
-        pass
-    # Uncomment the desired output format:
-    publish_cmdline(writer_name='pseudoxml', description=description)
-    # publish_cmdline(writer_name='xml', description=description)
-    # publish_cmdline(writer_name='html', description=description)
-    # publish_cmdline(writer_name='latex', description=description)
-    # publish_cmdline(writer_name='newlatex2e', description=description)
-
-

- - - diff --git a/sandbox/code-block-directive/docs/pygments_code_block_directive.py.htm b/sandbox/code-block-directive/docs/pygments_code_block_directive.py.htm new file mode 100644 index 000000000..ba7364dbe --- /dev/null +++ b/sandbox/code-block-directive/docs/pygments_code_block_directive.py.htm @@ -0,0 +1,263 @@ + + + + + + + + + + + + + +

+ +++ + + + + + + + +

Author:	a Pygments author\|contributor; Felix Wiemann; Guenter Milde
Date:	2007-06-08
Copyright:	This module has been placed in the public domain.

+ +

This is a merge of Using Pygments in ReST documents from the pygments +documentation, and a proof of concept by Felix Wiemann.

+ ++++ + + + + + + + + + + + + + + + + + +

2007-06-01	Removed redundancy from class values.
2007-06-04	Merge of successive tokens of same type +(code taken from pygments.formatters.others).
2007-06-05	Separate docutils formatter script +Use pygments' CSS class names (like the html formatter) +allowing the use of pygments-produced style sheets.
2007-06-07	Merge in the formatting of the parsed tokens +(misnamed as docutils_formatter) as class DocutilsInterface
2007-06-08	Failsave implementation (fallback to a standard literal block +if pygments not found)

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 23)

Unknown directive type "code-block".

+.. code-block::
+
+  """Define and register a code-block directive using pygments
+  """
+
+

Requirements

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 31)

Unknown directive type "code-block".

+.. code-block::
+
+  from docutils import nodes
+  from docutils.parsers.rst import directives
+  try:
+      import pygments
+      from pygments.lexers import get_lexer_by_name
+      from pygments.formatters.html import _get_ttype_class
+  except ImportError:
+      pass
+
+
+
+

Customisation

Do not insert inline nodes for the following tokens. +(You could add e.g. Token.Punctuation like ['', 'p'].)

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 50)

Unknown directive type "code-block".

+.. code-block::
+
+  unstyled_tokens = ['']
+
+

DocutilsInterface

This interface class combines code from +pygments.formatters.html and pygments.formatters.others.

It does not require anything of docutils and could also become a part of +pygments

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 63)

Unknown directive type "code-block".

+.. code-block::
+
+  class DocutilsInterface(object):
+      """Parse `code` string and yield "classified" tokens.
+
+      Arguments
+
+        code     -- string of source code to parse
+        language -- formal language the code is written in.
+
+      Merge subsequent tokens of the same token-type.
+
+      Yields the tokens as ``(ttype_class, value)`` tuples,
+      where ttype_class is taken from pygments.token.STANDARD_TYPES and
+      corresponds to the class argument used in pygments html output.
+
+      """
+
+      def __init__(self, code, language):
+          self.code = code
+          self.language = language
+
+      def lex(self):
+          # Get lexer for language (use text as fallback)
+          try:
+              lexer = get_lexer_by_name(self.language)
+          except ValueError:
+              # info: "no pygments lexer for %s, using 'text'"%self.language
+              lexer = get_lexer_by_name('text')
+          return pygments.lex(self.code, lexer)
+
+
+      def join(self, tokens):
+          """join subsequent tokens of same token-type
+          """
+          tokens = iter(tokens)
+          (lasttype, lastval) = tokens.next()
+          for ttype, value in tokens:
+              if ttype is lasttype:
+                  lastval += value
+              else:
+                  yield(lasttype, lastval)
+                  (lasttype, lastval) = (ttype, value)
+          yield(lasttype, lastval)
+
+      def __iter__(self):
+          """parse code string and yield "clasified" tokens
+          """
+          try:
+              tokens = self.lex()
+          except IOError:
+              print "INFO: Pygments lexer not found, using fallback"
+              # TODO: write message to INFO
+              yield ('', self.code)
+              return
+
+          for ttype, value in self.join(tokens):
+              yield (_get_ttype_class(ttype), value)
+
+
+
+

code_block_directive

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 127)

Unknown directive type "code-block".

+.. code-block::
+
+  def code_block_directive(name, arguments, options, content, lineno,
+                         content_offset, block_text, state, state_machine):
+      """parse and classify content of a code_block
+      """
+      language = arguments[0]
+      # create a literal block element and set class argument
+      code_block = nodes.literal_block(classes=["code-block", language])
+
+      # parse content with pygments and add to code_block element
+      for cls, value in DocutilsInterface(u'\n'.join(content), language):
+          if cls in unstyled_tokens:
+              # insert as Text to decrease the verbosity of the output.
+              code_block += nodes.Text(value, value)
+          else:
+              code_block += nodes.inline(value, value, classes=[cls])
+
+      return [code_block]
+
+
+

Register Directive

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 151)

Unknown directive type "code-block".

+.. code-block::
+
+  code_block_directive.arguments = (1, 0, 1)
+  code_block_directive.content = 1
+  directives.register_directive('code-block', code_block_directive)
+
+

Test output

If called from the command line, call the docutils publisher to render the +input

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 169)

Unknown directive type "code-block".

+.. code-block::
+
+  if __name__ == '__main__':
+      from docutils.core import publish_cmdline, default_description
+      description = "code-block directive test output" + default_description
+      try:
+          import locale
+          locale.setlocale(locale.LC_ALL, '')
+      except:
+          pass
+      # Uncomment the desired output format:
+      publish_cmdline(writer_name='pseudoxml', description=description)
+      # publish_cmdline(writer_name='xml', description=description)
+      # publish_cmdline(writer_name='html', description=description)
+      # publish_cmdline(writer_name='latex', description=description)
+      # publish_cmdline(writer_name='newlatex2e', description=description)
+
+
+
+

+ + + diff --git a/sandbox/code-block-directive/docs/pygments_code_block_directive.py.html b/sandbox/code-block-directive/docs/pygments_code_block_directive.py.html deleted file mode 100644 index ba7364dbe..000000000 --- a/sandbox/code-block-directive/docs/pygments_code_block_directive.py.html +++ /dev/null @@ -1,263 +0,0 @@ - - - - - - - - - - - - - -

- --- - - - - - - - -

Author:	a Pygments author\|contributor; Felix Wiemann; Guenter Milde
Date:	2007-06-08
Copyright:	This module has been placed in the public domain.

- -

This is a merge of Using Pygments in ReST documents from the pygments -documentation, and a proof of concept by Felix Wiemann.

- ---- - - - - - - - - - - - - - - - - - -

2007-06-01	Removed redundancy from class values.
2007-06-04	Merge of successive tokens of same type -(code taken from pygments.formatters.others).
2007-06-05	Separate docutils formatter script -Use pygments' CSS class names (like the html formatter) -allowing the use of pygments-produced style sheets.
2007-06-07	Merge in the formatting of the parsed tokens -(misnamed as docutils_formatter) as class DocutilsInterface
2007-06-08	Failsave implementation (fallback to a standard literal block -if pygments not found)

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 23)

Unknown directive type "code-block".

-.. code-block::
-
-  """Define and register a code-block directive using pygments
-  """
-
-

Requirements

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 31)

Unknown directive type "code-block".

-.. code-block::
-
-  from docutils import nodes
-  from docutils.parsers.rst import directives
-  try:
-      import pygments
-      from pygments.lexers import get_lexer_by_name
-      from pygments.formatters.html import _get_ttype_class
-  except ImportError:
-      pass
-
-
-
-

Customisation

Do not insert inline nodes for the following tokens. -(You could add e.g. Token.Punctuation like ['', 'p'].)

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 50)

Unknown directive type "code-block".

-.. code-block::
-
-  unstyled_tokens = ['']
-
-

DocutilsInterface

This interface class combines code from -pygments.formatters.html and pygments.formatters.others.

It does not require anything of docutils and could also become a part of -pygments

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 63)

Unknown directive type "code-block".

-.. code-block::
-
-  class DocutilsInterface(object):
-      """Parse `code` string and yield "classified" tokens.
-
-      Arguments
-
-        code     -- string of source code to parse
-        language -- formal language the code is written in.
-
-      Merge subsequent tokens of the same token-type.
-
-      Yields the tokens as ``(ttype_class, value)`` tuples,
-      where ttype_class is taken from pygments.token.STANDARD_TYPES and
-      corresponds to the class argument used in pygments html output.
-
-      """
-
-      def __init__(self, code, language):
-          self.code = code
-          self.language = language
-
-      def lex(self):
-          # Get lexer for language (use text as fallback)
-          try:
-              lexer = get_lexer_by_name(self.language)
-          except ValueError:
-              # info: "no pygments lexer for %s, using 'text'"%self.language
-              lexer = get_lexer_by_name('text')
-          return pygments.lex(self.code, lexer)
-
-
-      def join(self, tokens):
-          """join subsequent tokens of same token-type
-          """
-          tokens = iter(tokens)
-          (lasttype, lastval) = tokens.next()
-          for ttype, value in tokens:
-              if ttype is lasttype:
-                  lastval += value
-              else:
-                  yield(lasttype, lastval)
-                  (lasttype, lastval) = (ttype, value)
-          yield(lasttype, lastval)
-
-      def __iter__(self):
-          """parse code string and yield "clasified" tokens
-          """
-          try:
-              tokens = self.lex()
-          except IOError:
-              print "INFO: Pygments lexer not found, using fallback"
-              # TODO: write message to INFO
-              yield ('', self.code)
-              return
-
-          for ttype, value in self.join(tokens):
-              yield (_get_ttype_class(ttype), value)
-
-
-
-

code_block_directive

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 127)

Unknown directive type "code-block".

-.. code-block::
-
-  def code_block_directive(name, arguments, options, content, lineno,
-                         content_offset, block_text, state, state_machine):
-      """parse and classify content of a code_block
-      """
-      language = arguments[0]
-      # create a literal block element and set class argument
-      code_block = nodes.literal_block(classes=["code-block", language])
-
-      # parse content with pygments and add to code_block element
-      for cls, value in DocutilsInterface(u'\n'.join(content), language):
-          if cls in unstyled_tokens:
-              # insert as Text to decrease the verbosity of the output.
-              code_block += nodes.Text(value, value)
-          else:
-              code_block += nodes.inline(value, value, classes=[cls])
-
-      return [code_block]
-
-
-

Register Directive

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 151)

Unknown directive type "code-block".

-.. code-block::
-
-  code_block_directive.arguments = (1, 0, 1)
-  code_block_directive.content = 1
-  directives.register_directive('code-block', code_block_directive)
-
-

Test output

If called from the command line, call the docutils publisher to render the -input

System Message: ERROR/3 (/usr/src/docutils-svn/sandbox/code-block-directive/pygments_code_block_directive.py.txt, line 169)

Unknown directive type "code-block".

-.. code-block::
-
-  if __name__ == '__main__':
-      from docutils.core import publish_cmdline, default_description
-      description = "code-block directive test output" + default_description
-      try:
-          import locale
-          locale.setlocale(locale.LC_ALL, '')
-      except:
-          pass
-      # Uncomment the desired output format:
-      publish_cmdline(writer_name='pseudoxml', description=description)
-      # publish_cmdline(writer_name='xml', description=description)
-      # publish_cmdline(writer_name='html', description=description)
-      # publish_cmdline(writer_name='latex', description=description)
-      # publish_cmdline(writer_name='newlatex2e', description=description)
-
-
-
-

- - - -- cgit v1.2.1