diff options
Diffstat (limited to 'Doc/library/sgmllib.rst')
| -rw-r--r-- | Doc/library/sgmllib.rst | 270 |
1 files changed, 270 insertions, 0 deletions
diff --git a/Doc/library/sgmllib.rst b/Doc/library/sgmllib.rst new file mode 100644 index 0000000000..c0ef1a2953 --- /dev/null +++ b/Doc/library/sgmllib.rst @@ -0,0 +1,270 @@ + +:mod:`sgmllib` --- Simple SGML parser +===================================== + +.. module:: sgmllib + :synopsis: Only as much of an SGML parser as needed to parse HTML. + + +.. index:: single: SGML + +This module defines a class :class:`SGMLParser` which serves as the basis for +parsing text files formatted in SGML (Standard Generalized Mark-up Language). +In fact, it does not provide a full SGML parser --- it only parses SGML insofar +as it is used by HTML, and the module only exists as a base for the +:mod:`htmllib` module. Another HTML parser which supports XHTML and offers a +somewhat different interface is available in the :mod:`HTMLParser` module. + + +.. class:: SGMLParser() + + The :class:`SGMLParser` class is instantiated without arguments. The parser is + hardcoded to recognize the following constructs: + + * Opening and closing tags of the form ``<tag attr="value" ...>`` and + ``</tag>``, respectively. + + * Numeric character references of the form ``&#name;``. + + * Entity references of the form ``&name;``. + + * SGML comments of the form ``<!--text-->``. Note that spaces, tabs, and + newlines are allowed between the trailing ``>`` and the immediately preceding + ``--``. + +A single exception is defined as well: + + +.. exception:: SGMLParseError + + Exception raised by the :class:`SGMLParser` class when it encounters an error + while parsing. + + .. versionadded:: 2.1 + +:class:`SGMLParser` instances have the following methods: + + +.. method:: SGMLParser.reset() + + Reset the instance. Loses all unprocessed data. This is called implicitly at + instantiation time. + + +.. method:: SGMLParser.setnomoretags() + + Stop processing tags. Treat all following input as literal input (CDATA). + (This is only provided so the HTML tag ``<PLAINTEXT>`` can be implemented.) + + +.. method:: SGMLParser.setliteral() + + Enter literal mode (CDATA mode). + + +.. method:: SGMLParser.feed(data) + + Feed some text to the parser. It is processed insofar as it consists of + complete elements; incomplete data is buffered until more data is fed or + :meth:`close` is called. + + +.. method:: SGMLParser.close() + + Force processing of all buffered data as if it were followed by an end-of-file + mark. This method may be redefined by a derived class to define additional + processing at the end of the input, but the redefined version should always call + :meth:`close`. + + +.. method:: SGMLParser.get_starttag_text() + + Return the text of the most recently opened start tag. This should not normally + be needed for structured processing, but may be useful in dealing with HTML "as + deployed" or for re-generating input with minimal changes (whitespace between + attributes can be preserved, etc.). + + +.. method:: SGMLParser.handle_starttag(tag, method, attributes) + + This method is called to handle start tags for which either a :meth:`start_tag` + or :meth:`do_tag` method has been defined. The *tag* argument is the name of + the tag converted to lower case, and the *method* argument is the bound method + which should be used to support semantic interpretation of the start tag. The + *attributes* argument is a list of ``(name, value)`` pairs containing the + attributes found inside the tag's ``<>`` brackets. + + The *name* has been translated to lower case. Double quotes and backslashes in + the *value* have been interpreted, as well as known character references and + known entity references terminated by a semicolon (normally, entity references + can be terminated by any non-alphanumerical character, but this would break the + very common case of ``<A HREF="url?spam=1&eggs=2">`` when ``eggs`` is a valid + entity name). + + For instance, for the tag ``<A HREF="http://www.cwi.nl/">``, this method would + be called as ``unknown_starttag('a', [('href', 'http://www.cwi.nl/')])``. The + base implementation simply calls *method* with *attributes* as the only + argument. + + .. versionadded:: 2.5 + Handling of entity and character references within attribute values. + + +.. method:: SGMLParser.handle_endtag(tag, method) + + This method is called to handle endtags for which an :meth:`end_tag` method has + been defined. The *tag* argument is the name of the tag converted to lower + case, and the *method* argument is the bound method which should be used to + support semantic interpretation of the end tag. If no :meth:`end_tag` method is + defined for the closing element, this handler is not called. The base + implementation simply calls *method*. + + +.. method:: SGMLParser.handle_data(data) + + This method is called to process arbitrary data. It is intended to be + overridden by a derived class; the base class implementation does nothing. + + +.. method:: SGMLParser.handle_charref(ref) + + This method is called to process a character reference of the form ``&#ref;``. + The base implementation uses :meth:`convert_charref` to convert the reference to + a string. If that method returns a string, it is passed to :meth:`handle_data`, + otherwise ``unknown_charref(ref)`` is called to handle the error. + + .. versionchanged:: 2.5 + Use :meth:`convert_charref` instead of hard-coding the conversion. + + +.. method:: SGMLParser.convert_charref(ref) + + Convert a character reference to a string, or ``None``. *ref* is the reference + passed in as a string. In the base implementation, *ref* must be a decimal + number in the range 0-255. It converts the code point found using the + :meth:`convert_codepoint` method. If *ref* is invalid or out of range, this + method returns ``None``. This method is called by the default + :meth:`handle_charref` implementation and by the attribute value parser. + + .. versionadded:: 2.5 + + +.. method:: SGMLParser.convert_codepoint(codepoint) + + Convert a codepoint to a :class:`str` value. Encodings can be handled here if + appropriate, though the rest of :mod:`sgmllib` is oblivious on this matter. + + .. versionadded:: 2.5 + + +.. method:: SGMLParser.handle_entityref(ref) + + This method is called to process a general entity reference of the form + ``&ref;`` where *ref* is an general entity reference. It converts *ref* by + passing it to :meth:`convert_entityref`. If a translation is returned, it calls + the method :meth:`handle_data` with the translation; otherwise, it calls the + method ``unknown_entityref(ref)``. The default :attr:`entitydefs` defines + translations for ``&``, ``&apos``, ``>``, ``<``, and ``"``. + + .. versionchanged:: 2.5 + Use :meth:`convert_entityref` instead of hard-coding the conversion. + + +.. method:: SGMLParser.convert_entityref(ref) + + Convert a named entity reference to a :class:`str` value, or ``None``. The + resulting value will not be parsed. *ref* will be only the name of the entity. + The default implementation looks for *ref* in the instance (or class) variable + :attr:`entitydefs` which should be a mapping from entity names to corresponding + translations. If no translation is available for *ref*, this method returns + ``None``. This method is called by the default :meth:`handle_entityref` + implementation and by the attribute value parser. + + .. versionadded:: 2.5 + + +.. method:: SGMLParser.handle_comment(comment) + + This method is called when a comment is encountered. The *comment* argument is + a string containing the text between the ``<!--`` and ``-->`` delimiters, but + not the delimiters themselves. For example, the comment ``<!--text-->`` will + cause this method to be called with the argument ``'text'``. The default method + does nothing. + + +.. method:: SGMLParser.handle_decl(data) + + Method called when an SGML declaration is read by the parser. In practice, the + ``DOCTYPE`` declaration is the only thing observed in HTML, but the parser does + not discriminate among different (or broken) declarations. Internal subsets in + a ``DOCTYPE`` declaration are not supported. The *data* parameter will be the + entire contents of the declaration inside the ``<!``...\ ``>`` markup. The + default implementation does nothing. + + +.. method:: SGMLParser.report_unbalanced(tag) + + This method is called when an end tag is found which does not correspond to any + open element. + + +.. method:: SGMLParser.unknown_starttag(tag, attributes) + + This method is called to process an unknown start tag. It is intended to be + overridden by a derived class; the base class implementation does nothing. + + +.. method:: SGMLParser.unknown_endtag(tag) + + This method is called to process an unknown end tag. It is intended to be + overridden by a derived class; the base class implementation does nothing. + + +.. method:: SGMLParser.unknown_charref(ref) + + This method is called to process unresolvable numeric character references. + Refer to :meth:`handle_charref` to determine what is handled by default. It is + intended to be overridden by a derived class; the base class implementation does + nothing. + + +.. method:: SGMLParser.unknown_entityref(ref) + + This method is called to process an unknown entity reference. It is intended to + be overridden by a derived class; the base class implementation does nothing. + +Apart from overriding or extending the methods listed above, derived classes may +also define methods of the following form to define processing of specific tags. +Tag names in the input stream are case independent; the *tag* occurring in +method names must be in lower case: + + +.. method:: SGMLParser.start_tag(attributes) + :noindex: + + This method is called to process an opening tag *tag*. It has preference over + :meth:`do_tag`. The *attributes* argument has the same meaning as described for + :meth:`handle_starttag` above. + + +.. method:: SGMLParser.do_tag(attributes) + :noindex: + + This method is called to process an opening tag *tag* for which no + :meth:`start_tag` method is defined. The *attributes* argument has the same + meaning as described for :meth:`handle_starttag` above. + + +.. method:: SGMLParser.end_tag() + :noindex: + + This method is called to process a closing tag *tag*. + +Note that the parser maintains a stack of open elements for which no end tag has +been found yet. Only tags processed by :meth:`start_tag` are pushed on this +stack. Definition of an :meth:`end_tag` method is optional for these tags. For +tags processed by :meth:`do_tag` or by :meth:`unknown_tag`, no :meth:`end_tag` +method must be defined; if defined, it will not be used. If both +:meth:`start_tag` and :meth:`do_tag` methods exist for a tag, the +:meth:`start_tag` method takes precedence. + |