summaryrefslogtreecommitdiff
path: root/Help
diff options
context:
space:
mode:
authorscivision <scivision@users.noreply.github.com>2023-02-15 16:09:09 -0500
committerscivision <scivision@users.noreply.github.com>2023-02-15 16:23:35 -0500
commit6e71c31afda09d6df03c058911773ac6cd67b3a4 (patch)
tree38c448b89141897982a5c933e62eba409c02ab21 /Help
parent37f068083ba53e4140e5660a82bda9708e53c356 (diff)
downloadcmake-6e71c31afda09d6df03c058911773ac6cd67b3a4.tar.gz
Help:add_test: emphasize important points
* enable_testing must be called to enable testing * can only set properties on tests created in same directory * WILL_FAIL still fails on segfault and similar * compact wording
Diffstat (limited to 'Help')
-rw-r--r--Help/command/add_test.rst65
-rw-r--r--Help/command/set_property.rst4
-rw-r--r--Help/prop_test/WILL_FAIL.rst5
3 files changed, 35 insertions, 39 deletions
diff --git a/Help/command/add_test.rst b/Help/command/add_test.rst
index 53555a4b04..02dd3986aa 100644
--- a/Help/command/add_test.rst
+++ b/Help/command/add_test.rst
@@ -12,13 +12,24 @@ Add a test to the project to be run by :manual:`ctest(1)`.
Adds a test called ``<name>``. The test name may contain arbitrary
characters, expressed as a :ref:`Quoted Argument` or :ref:`Bracket Argument`
-if necessary. See policy :policy:`CMP0110`. The options are:
+if necessary. See policy :policy:`CMP0110`.
+
+CMake only generates tests if the :command:`enable_testing` command has been
+invoked. The :module:`CTest` module invokes ``enable_testing`` automatically
+unless ``BUILD_TESTING`` is set to ``OFF``.
+
+Tests added with the ``add_test(NAME)`` signature support using
+:manual:`generator expressions <cmake-generator-expressions(7)>`
+in test properties set by :command:`set_property(TEST)` or
+:command:`set_tests_properties`. Test properties may only be set in the
+directory the test is created in.
+
+``add_test`` options are:
``COMMAND``
- Specify the test command-line. If ``<command>`` specifies an
- executable target (created by :command:`add_executable`) it will
- automatically be replaced by the location of the executable created
- at build time.
+ Specify the test command-line. If ``<command>`` specifies an executable
+ target created by :command:`add_executable`, it will automatically be
+ replaced by the location of the executable created at build time.
The command may be specified using
:manual:`generator expressions <cmake-generator-expressions(7)>`.
@@ -27,38 +38,29 @@ if necessary. See policy :policy:`CMP0110`. The options are:
Restrict execution of the test only to the named configurations.
``WORKING_DIRECTORY``
- Set the :prop_test:`WORKING_DIRECTORY` test property to
- specify the working directory in which to execute the test.
- If not specified the test will be run with the current working
- directory set to the build directory corresponding to the
- current source directory.
-
- The working directory may be specified using
- :manual:`generator expressions <cmake-generator-expressions(7)>`.
+ Set the test property :prop_test:`WORKING_DIRECTORY` in which to execute the
+ test. If not specified, the test will be run in
+ :variable:`CMAKE_CURRENT_BINARY_DIR`. The working directory may be specified
+ using :manual:`generator expressions <cmake-generator-expressions(7)>`.
``COMMAND_EXPAND_LISTS``
.. versionadded:: 3.16
- Lists in ``COMMAND`` arguments will be expanded, including those
- created with
+ Lists in ``COMMAND`` arguments will be expanded, including those created with
:manual:`generator expressions <cmake-generator-expressions(7)>`.
-The given test command is expected to exit with code ``0`` to pass and
-non-zero to fail, or vice-versa if the :prop_test:`WILL_FAIL` test
-property is set. Any output written to stdout or stderr will be
-captured by :manual:`ctest(1)` but does not affect the pass/fail status
-unless the :prop_test:`PASS_REGULAR_EXPRESSION`,
-:prop_test:`FAIL_REGULAR_EXPRESSION` or
-:prop_test:`SKIP_REGULAR_EXPRESSION` test property is used.
+If the test command exits with code ``0`` the test passes. Non-zero exit code
+is a "failed" test. The test property :prop_test:`WILL_FAIL` inverts this
+logic. Note that system-level test failures such as segmentation faults or
+heap errors will still fail the test even if ``WILL_FALL`` is true. Output
+written to stdout or stderr is captured by :manual:`ctest(1)` and only
+affects the pass/fail status via the :prop_test:`PASS_REGULAR_EXPRESSION`,
+:prop_test:`FAIL_REGULAR_EXPRESSION`, or :prop_test:`SKIP_REGULAR_EXPRESSION`
+test properties.
.. versionadded:: 3.16
Added :prop_test:`SKIP_REGULAR_EXPRESSION` property.
-Tests added with the ``add_test(NAME)`` signature support using
-:manual:`generator expressions <cmake-generator-expressions(7)>`
-in test properties set by :command:`set_property(TEST)` or
-:command:`set_tests_properties`.
-
Example usage:
.. code-block:: cmake
@@ -71,16 +73,9 @@ This creates a test ``mytest`` whose command runs a ``testDriver`` tool
passing the configuration name and the full path to the executable
file produced by target ``myexe``.
-.. note::
-
- CMake will generate tests only if the :command:`enable_testing`
- command has been invoked. The :module:`CTest` module invokes the
- command automatically unless the ``BUILD_TESTING`` option is turned
- ``OFF``.
-
---------------------------------------------------------------------
-This command also supports a simpler, but less flexible, signature:
+The command syntax above is recommended over the older, less flexible form:
.. code-block:: cmake
diff --git a/Help/command/set_property.rst b/Help/command/set_property.rst
index ca19e0cc14..8dd4b94b9f 100644
--- a/Help/command/set_property.rst
+++ b/Help/command/set_property.rst
@@ -82,8 +82,8 @@ It must be one of the following:
to the installation prefix.
``TEST``
- Scope may name zero or more existing tests.
- See also the :command:`set_tests_properties` command.
+ Scope is limited to the directory the command is called in. It may name zero
+ or more existing tests. See also command :command:`set_tests_properties`.
Test property values may be specified using
:manual:`generator expressions <cmake-generator-expressions(7)>`
diff --git a/Help/prop_test/WILL_FAIL.rst b/Help/prop_test/WILL_FAIL.rst
index f1f94a43a7..4926f40779 100644
--- a/Help/prop_test/WILL_FAIL.rst
+++ b/Help/prop_test/WILL_FAIL.rst
@@ -3,5 +3,6 @@ WILL_FAIL
If set to true, this will invert the pass/fail flag of the test.
-This property can be used for tests that are expected to fail and
-return a non zero return code.
+This property can be used for tests that are expected to fail and return a
+non-zero return code. Note that system-level test failures such as segmentation
+faults or heap errors will still fail the test even if ``WILL_FALL`` is true.