A regular paragraph of text.
First paragraph of wrapped text.
Second Paragraph of **wrapped** text.
Another regular paragraph of text.
#### Usages Some block processors in the Markdown source tree include: | Class | Kind | Description | | ----------------------------|-----------|---------------------------------------------| | [`HashHeaderProcessor`][b1] | built-in | Title hashes (`#`), which may split blocks | | [`HRProcessor`][b2] | built-in | Horizontal lines, e.g., `---` | | [`OListProcessor`][b3] | built-in | Ordered lists; complex and using `state` | | [`Admonition`][b4] | extension | Render each [Admonition][] in a new `div` | [b1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py [b2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py [b3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py [Admonition]: https://python-markdown.github.io/extensions/admonition/ [b4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/admonition.py ### Tree processors {: #treeprocessors } Tree processors manipulate the tree created by block processors. They can even create an entirely new ElementTree object. This is an excellent place for creating summaries, adding collected references, or last minute adjustments. A tree processor must inherit from `markdown.treeprocessors.Treeprocessor` (note the capitalization). A tree processor must implement a `run` method which takes a single argument `root`. In most cases `root` would be an `xml.etree.ElementTree.Element` instance; however, in rare cases it could be some other type of ElementTree object. The `run` method may return `None`, in which case the (possibly modified) original `root` object is used, or it may return an entirely new `Element` object, which will replace the existing `root` object and all of its children. It is generally preferred to modify `root` in place and return `None`, which avoids creating multiple copies of the entire document tree in memory. For specifics on manipulating the ElementTree, see [Working with the ElementTree][workingwithetree] below. #### Example A pseudo example: ```python from markdown.treeprocessors import Treeprocessor class MyTreeprocessor(Treeprocessor): def run(self, root): root.text = 'modified content' # No return statement is same as `return None` ``` #### Usages The core `InlineProcessor` class is a tree processor. It walks the tree, matches patterns, and splits and creates nodes on matches. Additional tree processors in the Markdown source tree include: | Class | Kind | Description | | ----------------------------------|-----------|---------------------------------------------------------------| | [`PrettifyTreeprocessor`][e1] | built-in | Add line breaks to the html document | | [`TocTreeprocessor`][e2] | extension | Builds a [table of contents][] from the finished tree | | [`FootnoteTreeprocessor`][e3] | extension | Create [footnote][] div at end of document | | [`FootnotePostTreeprocessor`][e4] | extension | Amend div created by `FootnoteTreeprocessor` with duplicates | [e1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/treeprocessors.py [e2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/toc.py [e3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py [e4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py [table of contents]: https://python-markdown.github.io/extensions/toc/ [footnote]: https://python-markdown.github.io/extensions/footnotes/ ### Inline Processors {: #inlineprocessors } Inline processors, previously called inline patterns, are used to add formatting, such as `**emphasis**`, by replacing a matched pattern with a new element tree node. It is an excellent for adding new syntax for inline tags. Inline processor code is often quite short. Inline processors inherit from `InlineProcessor`, are initialized, and implement `handleMatch`: * `__init__(self, pattern, md=None)` is the inherited constructor. You do not need to implement your own. * `pattern` is the regular expression string that must match the code block in order for the `handleMatch` method to be called. * `md`, an optional parameter, is a pointer to the instance of `markdown.Markdown` and is available as `self.md` on the `InlineProcessor` instance. * `handleMatch(self, m, data)` must be implemented in all `InlineProcessor` subclasses. * `m` is the regular expression [match object][] found by the `pattern` passed to `__init__`. * `data` is a single, multi-line, Unicode string containing the entire block of text around the pattern. A block is text set apart by blank lines. * Returns either `(None, None, None)`, indicating the provided match was rejected or `(el, start, end)`, if the match was successfully processed. On success, `el` is the element being added the tree, `start` and `end` are indexes in `data` that were "consumed" by the pattern. The "consumed" span will be replaced by a placeholder. The same inline processor may be called several times on the same block. Inline Processors can define the property `ANCESTOR_EXCLUDES` which is either a list or tuple of undesirable ancestors. The processor will be skipped if it would cause the content to be a descendant of one of the listed tag names. ##### Convenience Classes Convenience subclasses of `InlineProcessor` are provide for common operations: * [`SimpleTextInlineProcessor`][i1] returns the text of `group(1)` of the match. * [`SubstituteTagInlineProcessor`][i4] is initialized as `SubstituteTagInlineProcessor(pattern, tag)`. It returns a new element `tag` whenever `pattern` is matched. * [`SimpleTagInlineProcessor`][i3] is initialized as `SimpleTagInlineProcessor(pattern, tag)`. It returns an element `tag` with a text field of `group(2)` of the match. ##### Example This example changes `--strike--` to `First line of the block.
This is strike one.
This is strike two.
End of the block.
tags as a diagnostic. """
def run(self, text):
return '\n' + re.sub('<', '<', text) + '\n'
```
#### Usages
Some postprocessors in the Markdown source tree include:
| Class | Kind | Description |
| ------------------------------|-----------|----------------------------------------------------|
| [`raw_html`][p1] | built-in | Restore raw html from `htmlStash`, stored by `HTMLBlockPreprocessor`, and code highlighters |
| [`amp_substitute`][p2] | built-in | Convert ampersand substitutes to `&`; used in links |
| [`unescape`][p3] | built-in | Convert some escaped characters back from integers; used in links |
| [`FootnotePostProcessor`][p4] | extension | Replace footnote placeholders with html entities; as set by other stages |
[p1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py
[p2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py
[p3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py
[p4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py
## Working with the ElementTree {: #working_with_et }
As mentioned, the Markdown parser converts a source document to an [ElementTree][ElementTree] object before
serializing that back to Unicode text. Markdown has provided some helpers to ease that manipulation within the context
of the Markdown module.
First, import the ElementTree module:
```python
import xml.etree.ElementTree as etree
```
Sometimes you may want text inserted into an element to be parsed by [Inline Patterns][]. In such a situation, simply
insert the text as you normally would and the text will be automatically run through the Inline Patterns. However, if
you do *not* want some text to be parsed by Inline Patterns, then insert the text as an `AtomicString`.
```python
from markdown.util import AtomicString
some_element.text = AtomicString(some_text)
```
Here's a basic example which creates an HTML table (note that the contents of the second cell (`td2`) will be run
through Inline Patterns latter):
```python
table = etree.Element("table")
table.set("cellpadding", "2") # Set cellpadding to 2
tr = etree.SubElement(table, "tr") # Add child tr to table
td1 = etree.SubElement(tr, "td") # Add child td1 to tr
td1.text = markdown.util.AtomicString("Cell content") # Add plain text content
td2 = etree.SubElement(tr, "td") # Add second td to tr
td2.text = "*text* with **inline** formatting." # Add markup text
table.tail = "Text after table" # Add text after table
```
You can also manipulate an existing tree. Consider the following example which adds a `class` attribute to ``
elements:
```python
def set_link_class(self, element):
for child in element:
if child.tag == "a":
child.set("class", "myclass") #set the class attribute
set_link_class(child) # run recursively on children
```
For more information about working with ElementTree see the [ElementTree
Documentation][ElementTree].
## Working with Raw HTML {: #working_with_raw_html }
Occasionally an extension may need to call out to a third party library which returns a pre-made string
of raw HTML that needs to be inserted into the document unmodified. Raw strings can be stashed for later
retrieval using an `htmlStash` instance, rather than converting them into `ElementTree` objects. A raw string
(which may or may not be raw HTML) passed to `self.md.htmlStash.store()` will be saved to the stash and a
placeholder string will be returned which should be inserted into the tree instead. After the tree is
serialized, a postprocessor will replace the placeholder with the raw string. This prevents subsequent
processing steps from modifying the HTML data. For example,
```python
html = "This is some raw HTML data
"
el = etree.Element("div")
el.text = self.md.htmlStash.store(html)
```
For the global `htmlStash` instance to be available from a processor, the `markdown.Markdown` instance must
be passed to the processor from [extendMarkdown](#extendmarkdown) and will be available as `self.md.htmlStash`.
## Integrating Your Code Into Markdown {: #integrating_into_markdown }
Once you have the various pieces of your extension built, you need to tell Markdown about them and ensure that they
are run in the proper sequence. Markdown accepts an `Extension` instance for each extension. Therefore, you will need
to define a class that extends `markdown.extensions.Extension` and over-rides the `extendMarkdown` method. Within this
class you will manage configuration options for your extension and attach the various processors and patterns to the
Markdown instance.
It is important to note that the order of the various processors and patterns matters. For example, if we replace
`http://...` links with `` elements, and *then* try to deal with inline HTML, we will end up with a mess.
Therefore, the various types of processors and patterns are stored within an instance of the `markdown.Markdown` class
in a [Registry][]. Your `Extension` class will need to manipulate those registries appropriately. You may `register`
instances of your processors and patterns with an appropriate priority, `deregister` built-in instances, or replace a
built-in instance with your own.
### `extendMarkdown` {: #extendmarkdown }
The `extendMarkdown` method of a `markdown.extensions.Extension` class accepts one argument:
* **`md`**:
A pointer to the instance of the `markdown.Markdown` class. You should use this to access the
[Registries][Registry] of processors and patterns. They are found under the following attributes:
* `md.preprocessors`
* `md.inlinePatterns`
* `md.parser.blockprocessors`
* `md.treeprocessors`
* `md.postprocessors`
Some other things you may want to access on the `markdown.Markdown` instance are:
* `md.htmlStash`
* `md.output_formats`
* `md.set_output_format()`
* `md.output_format`
* `md.serializer`
* `md.registerExtension()`
* `md.tab_length`
* `md.block_level_elements`
* `md.isBlockLevel()`
!!! Warning
With access to the above items, theoretically you have the option to change anything through various
[monkey_patching][] techniques. However, you should be aware that the various undocumented parts of Markdown may
change without notice and your monkey_patches may break with a new release. Therefore, what you really should be
doing is inserting processors and patterns into the Markdown pipeline. Consider yourself warned!
[monkey_patching]: https://en.wikipedia.org/wiki/Monkey_patch
A simple example:
```python
from markdown.extensions import Extension
class MyExtension(Extension):
def extendMarkdown(self, md):
# Register instance of 'mypattern' with a priority of 175
md.inlinePatterns.register(MyPattern(md), 'mypattern', 175)
```
### registerExtension {: #registerextension }
Some extensions may need to have their state reset between multiple runs of the `markdown.Markdown` class. For
example, consider the following use of the [Footnotes][] extension:
```python
md = markdown.Markdown(extensions=['footnotes'])
html1 = md.convert(text_with_footnote)
md.reset()
html2 = md.convert(text_without_footnote)
```
Without calling `reset`, the footnote definitions from the first document will be inserted into the second document as
they are still stored within the class instance. Therefore the `Extension` class needs to define a `reset` method that
will reset the state of the extension (i.e.: `self.footnotes = {}`). However, as many extensions do not have a need
for `reset`, `reset` is only called on extensions that are registered.
To register an extension, call `md.registerExtension` from within your `extendMarkdown` method:
```python
def extendMarkdown(self, md):
md.registerExtension(self)
# insert processors and patterns here
```
Then, each time `reset` is called on the `markdown.Markdown` instance, the `reset` method of each registered extension
will be called as well. You should also note that `reset` will be called on each registered extension after it is
initialized the first time. Keep that in mind when over-riding the extension's `reset` method.
### Configuration Settings {: #configsettings }
If an extension uses any parameters that the user may want to change, those parameters should be stored in
`self.config` of your `markdown.extensions.Extension` class in the following format:
```python
class MyExtension(markdown.extensions.Extension):
def __init__(self, **kwargs):
self.config = {
'option1' : ['value1', 'description1'],
'option2' : ['value2', 'description2']
}
super(MyExtension, self).__init__(**kwargs)
```
When implemented this way the configuration parameters can be over-ridden at run time (thus the call to `super`). For
example:
```python
markdown.Markdown(extensions=[MyExtension(option1='other value')])
```
Note that if a keyword is passed in that is not already defined in `self.config`, then a `KeyError` is raised.
The `markdown.extensions.Extension` class and its subclasses have the following methods available to assist in working
with configuration settings:
* **`getConfig(key [, default])`**:
Returns the stored value for the given `key` or `default` if the `key` does not exist. If not set, `default`
returns an empty string.
* **`getConfigs()`**:
Returns a dict of all key/value pairs.
* **`getConfigInfo()`**:
Returns all configuration descriptions as a list of tuples.
* **`setConfig(key, value)`**:
Sets a configuration setting for `key` with the given `value`. If `key` is unknown, a `KeyError` is raised. If the
previous value of `key` was a Boolean value, then `value` is converted to a Boolean value. If the previous value
of `key` is `None`, then `value` is converted to a Boolean value except when it is `None`. No conversion takes
place when the previous value of `key` is a string.
* **`setConfigs(items)`**:
Sets multiple configuration settings given a dict of key/value pairs.
### Naming an Extension { #naming_an_extension }
As noted in the [library reference] an instance of an extension can be passed directly to `markdown.Markdown`. In
fact, this is the preferred way to use third-party extensions.
For example:
```python
import markdown
from path.to.module import MyExtension
md = markdown.Markdown(extensions=[MyExtension(option='value')])
```
However, Markdown also accepts "named" third party extensions for those occasions when it is impractical to import an
extension directly (from the command line or from within templates). A "name" can either be a registered [entry
point](#entry_point) or a string using Python's [dot notation](#dot_notation).
#### Entry Point { #entry_point }
[Entry points] are defined in a Python package's `setup.py` script. The script must use [setuptools] to support entry
points. Python-Markdown extensions must be assigned to the `markdown.extensions` group. An entry point definition
might look like this:
```python
from setuptools import setup
setup(
# ...
entry_points={
'markdown.extensions': ['myextension = path.to.module:MyExtension']
}
)
```
After a user installs your extension using the above script, they could then call the extension using the
`myextension` string name like this:
```python
markdown.markdown(text, extensions=['myextension'])
```
Note that if two or more entry points within the same group are assigned the same name, Python-Markdown will only ever
use the first one found and ignore all others. Therefore, be sure to give your extension a unique name.
For more information on writing `setup.py` scripts, see the Python documentation on [Packaging and Distributing
Projects].
#### Dot Notation { #dot_notation }
If an extension does not have a registered entry point, Python's dot notation may be used instead. The extension must
be installed as a Python module on your PYTHONPATH. Generally, a class should be specified in the name. The class must
be at the end of the name and be separated by a colon from the module.
Therefore, if you were to import the class like this:
```python
from path.to.module import MyExtension
```
Then the extension can be loaded as follows:
```python
markdown.markdown(text, extensions=['path.to.module:MyExtension'])
```
You do not need to do anything special to support this feature. As long as your extension class is able to be
imported, a user can include it with the above syntax.
The above two methods are especially useful if you need to implement a large number of extensions with more than one
residing in a module. However, if you do not want to require that your users include the class name in their string,
you must define only one extension per module and that module must contain a module-level function called
`makeExtension` that accepts `**kwargs` and returns an extension instance.
For example:
```python
class MyExtension(markdown.extensions.Extension)
# Define extension here...
def makeExtension(**kwargs):
return MyExtension(**kwargs)
```
When `markdown.Markdown` is passed the "name" of your extension as a dot notation string that does not include a class
(for example `path.to.module`), it will import the module and call the `makeExtension` function to initiate your
extension.
## Registries
The `markdown.util.Registry` class is a priority sorted registry which Markdown uses internally to determine the
processing order of its various processors and patterns.
A `Registry` instance provides two public methods to alter the data of the registry: `register` and `deregister`. Use
`register` to add items and `deregister` to remove items. See each method for specifics.
When registering an item, a "name" and a "priority" must be provided. All items are automatically sorted by the value
of the "priority" parameter such that the item with the highest value will be processed first. The "name" is used to
remove (`deregister`) and get items.
A `Registry` instance is like a list (which maintains order) when reading data. You may iterate over the items, get an
item and get a count (length) of all items. You may also check that the registry contains an item.
When getting an item you may use either the index of the item or the string-based "name". For example:
```python
registry = Registry()
registry.register(SomeItem(), 'itemname', 20)
# Get the item by index
item = registry[0]
# Get the item by name
item = registry['itemname']
```
When checking that the registry contains an item, you may use either the string-based "name", or a reference to the
actual item. For example:
```python
someitem = SomeItem()
registry.register(someitem, 'itemname', 20)
# Contains the name
assert 'itemname' in registry
# Contains the item instance
assert someitem in registry
```
`markdown.util.Registry` has the following methods:
### `Registry.register(self, item, name, priority)` {: #registry.register data-toc-label='Registry.register'}
: Add an item to the registry with the given name and priority.
Parameters:
* `item`: The item being registered.
* `name`: A string used to reference the item.
* `priority`: An integer or float used to sort against all items.
If an item is registered with a "name" which already exists, the existing item is replaced with the new item.
Tread carefully as the old item is lost with no way to recover it. The new item will be sorted according to its
priority and will **not** retain the position of the old item.
### `Registry.deregister(self, name, strict=True)` {: #registry.deregister data-toc-label='Registry.deregister'}
: Remove an item from the registry.
Set `strict=False` to fail silently.
### `Registry.get_index_for_name(self, name)` {: #registry.get_index_for_name data-toc-label='Registry.get_index_for_name'}
: Return the index of the given `name`.
[match object]: https://docs.python.org/3/library/re.html#match-objects
[bug tracker]: https://github.com/Python-Markdown/markdown/issues
[extension source]: https://github.com/Python-Markdown/markdown/tree/master/markdown/extensions
[tutorial]: https://github.com/Python-Markdown/markdown/wiki/Tutorial-1---Writing-Extensions-for-Python-Markdown
[workingwithetree]: #working_with_et
[Integrating your code into Markdown]: #integrating_into_markdown
[extendMarkdown]: #extendmarkdown
[Registry]: #registry
[registerExtension]: #registerextension
[Config Settings]: #configsettings
[makeExtension]: #makeextension
[ElementTree]: https://docs.python.org/3/library/xml.etree.elementtree.html
[Available Extensions]: index.md
[Footnotes]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py
[Definition Lists]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/definition_lists
[library reference]: ../reference.md
[setuptools]: https://packaging.python.org/key_projects/#setuptools
[Entry points]: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
[Packaging and Distributing Projects]: https://packaging.python.org/tutorials/distributing-packages/