diff options
Diffstat (limited to 'testasciidoc.txt')
-rw-r--r-- | testasciidoc.txt | 256 |
1 files changed, 0 insertions, 256 deletions
diff --git a/testasciidoc.txt b/testasciidoc.txt deleted file mode 100644 index 632f592..0000000 --- a/testasciidoc.txt +++ /dev/null @@ -1,256 +0,0 @@ -AsciiDoc Tests -============== - -AsciiDoc includes 'testasciidoc', a Python script runs a set of -AsciiDoc conformance tests. 'testasciidoc' runs through a list of -AsciiDoc source files, generates backend outputs and then compares -them to expected result files. The whole process is driven by a -configuration file containing a list of user configurable test -specifications. - - -Rationale ---------- -When modifying AsciiDoc configuration files or source code it's very -easy to introduce regression errors. 'testasciidoc' is a tool for -writing many and varied test cases that can be run after changes have -been made in order to verify that existing behavior has not been -broken. - -The 'testasciidoc' 'update' command automates the (re)generation of -expected test result data. Result data regeneration has to be -performed after changes to test case source files or after legitimate -changes to the AsciiDoc output formats -- doing this manually is -prohibitively tedious. - - -Running testasciidoc --------------------- -The `testasciidoc.py` script and the default `testasciidoc.conf` -configuration file are located in the AsciiDoc distribution `tests` -directory. - -To view the command usage run: - ---------------------------------------------------------------------- -$ python3 tests/testasciidoc.py -usage: testasciidoc.py [-h] [-v] [-f CONF_FILE] command ... - -Run AsciiDoc conformance tests specified in configuration FILE. - -positional arguments: - command - list List tests - run Execute tests - update Regenerate and update test data - -optional arguments: - -h, --help show this help message and exit - -v, --version show program's version number and exit - -f CONF_FILE, --conf-file CONF_FILE - Use configuration file CONF_FILE (default - configuration file is testasciidoc.conf in - testasciidoc.py directory) ---------------------------------------------------------------------- - -To view the list of tests in the default `testasciidoc.conf` -configuration file run the 'list' command: - ---------------------------------------------------------------------- -$ python tests/testasciidoc.py list -1: Test cases -2: Tables -3: Source highlighter -4: Example article -5: Example book -6: Example multi-part book -7: !User Guide ---------------------------------------------------------------------- - -The '!' prefix signals that a test is currently disabled. - -Before running the tests you will need to regenerate the expected -outputs by running the 'update' command: - ---------------------------------------------------------------------- -$ python tests/testasciidoc.py update -WRITING: tests/data/testcases-html4.html -WRITING: tests/data/testcases-xhtml11.html -WRITING: tests/data/testcases-docbook.xml - : -WRITING: tests/data/book-multi-docbook.xml ---------------------------------------------------------------------- - -Now you can run the tests: - ---------------------------------------------------------------------- -$ python tests/testasciidoc.py run -1: Test cases -SOURCE: asciidoc: tests/data/testcases.txt -PASSED: html4: tests/data/testcases-html4.html -PASSED: xhtml11: tests/data/testcases-xhtml11.html -PASSED: docbook: tests/data/testcases-docbook.xml - : -6: Example multi-part book -SOURCE: asciidoc: doc/book-multi.txt -PASSED: html4: tests/data/book-multi-html4.html -PASSED: xhtml11: tests/data/book-multi-xhtml11.html -PASSED: docbook: tests/data/book-multi-docbook.xml - -TOTAL PASSED: 18 ---------------------------------------------------------------------- - -[NOTE] -===================================================================== -- 'testasciidoc' requires AsciiDoc is located in your environment - search path, if not you'll need to set the `ASCIIDOC_PY` environment - variable to point to `asciidoc.py`. -- If you don't have GNU source-highlight installed you should disable - the 'Tables' and 'Source highlighter' tests in the - `tests/testasciidoc.conf` configuration file. -- The `run` and `update` commands both take optional `--number` and - `--backend` options to specify a test case number and/or backend to - test against. Use `python3 tests/testasciidoc.py run --help` for more - information. -===================================================================== - - -testasciidoc commands ---------------------- -'list':: - List test numbers and titles. A '!' title prefix signals that a - test is currently disabled. - -'run':: - Read and execute tests from the test configuration file. A test - specifies AsciiDoc test case source file and command options. The - test compares generated outputs to expected outputs and any - differences displayed as a diff. You can run selected tests by - specifying the test number and/or backend after the `run` command. - Examples: - - python tests/testasciidoc.py run - python tests/testasciidoc.py run --number 3 - python tests/testasciidoc.py run --backend html4 - python tests/testasciidoc.py run --number 3 --backend html4 - -'update':: - Generates and updates missing and out of date test output data - files, this eliminates one of the most time consuming aspect of test - management. Use the `--force` option to force updates. - Examples: - - python tests/testasciidoc.py update - python tests/testasciidoc.py update --number 4 --force - -NOTE: You can run or update disabled tests by explicitly specifying -the test number. - - -Test configuration file ------------------------ -The tests configuration file format consists of one or more test specs -separated by a line of one or more percent characters. Each test spec -consists of an optional test title and description followed by one or -more optional directives (a directive starts with a percent -character). A 'directive' consists begins with a line containing a '%' -character followed by a directive name followed by zero or more lines -of directive data. - -Test spec format -~~~~~~~~~~~~~~~~ ---------------------------------------------------------------------- -Optional test title -Optional test description... - -% name -Optional base output file name. Defaults to base source file name. - -% requires -Optional external requirements necessary to run the test, e.g. dot. - -% source -AsciiDoc source file name. - -% backends -Optional list of backends to be tested(default is all backends). - -% options -Optional list of command-line option tuples. - -% attributes -Optional dictionary of attribute values. - -% artifacts -Optional list of artifacts that get generated from test, and which -will get cleaned up. - ---------------------------------------------------------------------- - -Example test spec: - ---------------------------------------------------------------------- -Example book - -% options -['--section-numbers',('--doctype','book')] - -% requires -['source-highlight', 'dot'] - -% attributes -# Exclude date from document footer. -{'docdate':None} - -% source -../doc/book.txt ---------------------------------------------------------------------- - -TIP: Take a look at the default `tests/testasciidoc.conf` -configuration file that comes with AsciiDoc. - -- Tests can be disabled by prefixing the test title with an - exclamation '!' character. -- All relative file names are relative to the configuration file - directory. -- Multiple tests must by separated by a `%` separator line (one or - more percent characters). -- Lines starting with a percent character specify a test 'directive' - and may be followed by zero or more lines of directive data. -- Directive data lines cannot start with a percent character. -- Lines starting with a `#` hash character are ignored. -- The 'source' directive data is a single line containing the - AsciiDoc source file name. -- The 'options' directive data is a Python list of `(name,value)` - tuples specifying AsciiDoc command-line options. A string item is - equivalent to a `(name,None)` tuple. -- The 'attributes' directive data specifies a Python dictionary - containing AsciiDoc attributes to be passed to AsciiDoc. -- The `requires` directive data specifies a Python list of strings - which correspond to the command line program specified by filters - in that test. -- The `artifacts` directive data specifies a Python list of strings - which corresponds to the list of files (usually image or other such - files) that get generated when asciidoc is run over the input file. - The files are deleted after the test is for a given source file is - run for all backends. - -globals directive -~~~~~~~~~~~~~~~~~ -An optional 'globals' directive can precede all test specs, the -globals directive data is a Python dictionary containing global -values. Currently the only global is 'datadir', the directory -containing expected output files (defaults to configuration file -directory). For example: - ---------------------------------------------------------------------- -% globals -{'datadir': 'data'} ---------------------------------------------------------------------- - -Expected output test data files are stored in the 'datadir' and are -named after the corresponding AsciiDoc input source file. For example -if the AsciiDoc file name is `article.txt` then the corresponding -backend output files will be `article-html4.html`, -`article-xhtml11.html`, `article-docbook.xml` (stored in the 'datadir' -directory). |