CTest: Report tests not run due to unknown configuration

When add_test(NAME) is called without the CONFIGURATIONS argument then
the test is intended to run in any configuration.  In multi-config
generators like the VS IDE and Xcode tests created by add_test(NAME) can
only be run when testing a known configuration (otherwise there is no
way to generate the test command line).  If no test command line is
known for a particular configuration, or if no configuration is given to
ctest, report the test as not run instead of silently skipping it.

Also fix CMake's own TestsWorkingDirectory test invocation to correct a
previously silent failure exposed by this change.

2 files changed, 28 insertions, 0 deletions

diff --git a/Source/CTest/cmCTestRunTest.cxx b/Source/CTest/cmCTestRunTest.cxx
index b5b46f60b0..60695daa11 100644
--- a/Source/CTest/cmCTestRunTest.cxx
+++ b/Source/CTest/cmCTestRunTest.cxx
@@ -412,6 +412,30 @@ bool cmCTestRunTest::StartTest(size_t total)
   this->TestResult.TestCount = this->TestProperties->Index;  
   this->TestResult.Name = this->TestProperties->Name;
   this->TestResult.Path = this->TestProperties->Directory.c_str();
+
+  if(args.size() >= 2 && args[1] == "NOT_AVAILABLE")
+    {
+    this->TestProcess = new cmProcess;
+    std::string msg;
+    if(this->CTest->GetConfigType().empty())
+      {
+      msg = "Test not available without configuration.";
+      msg += "  (Missing \"-C <config>\"?)";
+      }
+    else
+      {
+      msg = "Test not available in configuration \"";
+      msg += this->CTest->GetConfigType();
+      msg += "\".";
+      }
+    *this->TestHandler->LogFile << msg << std::endl;
+    cmCTestLog(this->CTest, ERROR_MESSAGE, msg << std::endl);
+    this->TestResult.Output = msg;
+    this->TestResult.FullCommandLine = "";
+    this->TestResult.CompletionStatus = "Not Run";
+    this->TestResult.Status = cmCTestTestHandler::NOT_RUN;
+    return false;
+    }
   
   // Check if all required files exist
   for(std::vector<std::string>::iterator i =
diff --git a/Source/CTest/cmCTestTestHandler.cxx b/Source/CTest/cmCTestTestHandler.cxx
index 56f6630090..e3b81df882 100644
--- a/Source/CTest/cmCTestTestHandler.cxx
+++ b/Source/CTest/cmCTestTestHandler.cxx
@@ -1320,6 +1320,10 @@ std::string cmCTestTestHandler::FindTheExecutable(const char *exe)
   std::string resConfig;
   std::vector<std::string> extraPaths;
   std::vector<std::string> failedPaths;
+  if(strcmp(exe, "NOT_AVAILABLE") == 0)
+    {
+    return exe;
+    }
   return cmCTestTestHandler::FindExecutable(this->CTest,
                                             exe, resConfig,
                                             extraPaths,