[IMPORTANT NOTICE] ------------------ Do _not_ ask to developers why some or all tests are failed under your environment! Let us know if you find why it fails. Thank you. [Testing Basics] ---------------- To execute test scripts, you must build PHP with some SAPI, then you type "make test" to execute all or some test scripts saved under "tests" directory under source root directory. Usage: make test "make test" basically executes "run-tests.php" script under source root. Therefore you can execute the script as follows ./sapi/cli/php -c /path/to/php.ini/ run-tests.php [ext/some_extension_name] [Which "php" executable "make test" look for] --------------------------------------------- "make test" executes "run-tests.php" script with "php" binary. Some test scripts such as session must be executed by CGI SAPI. Therefore, you must build PHP with CGI SAPI to perform all tests. If PHP is not build with CGI SAPI, "run-tests.php" script uses CLI SAPI. Tests that may not executed by CLI SAPI will be skipped. NOTE: PHP binary executing "run-tests.php" and php binary used for executing test scripts may differ. If you use different PHP binary for executing "run-tests.php" script, you may get errors. [Which php.ini is used] ----------------------- "make test" force to use php.ini-dist as default config file. If you would like to test with other configuration file, user "run-tests.php" script. Example: ./sapi/cli/php -c /path/to/php.ini/ ext/standard If you use php.ini other than php.ini-dist, you may see more failed tests. [Which test scripts are executed] --------------------------------- "run-tests.php" ("make test") executes all test scripts by default by looking all directory named "tests". If there are files have "phpt" extension, "run-tests.php" takes test php code from the file and executes it. Tester can easily executes tests selectively with as follows. Example: ./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/mbstring [Test results] -------------- Test results are printed to standard output. If there is a failed test, "run-tests.php" script saves the result, expected result and code executed to the test script directory. For example, if ext/myext/tests/myext.phpt is failed to pass, following files are created: ext/myext/tests/myext.out - output from test script ext/myext/tests/myext.exp - expected output ext/myext/tests/myext.php - test script executed Tester can verify these files, if failed test is actually a bug or not. [Creating new test files] ------------------------- Writing test file is very easy if you are used to PHP. Test file has following format. Here is a actual test file from iconv module. ===== ext/iconv/002.phpt ======= --TEST-- UCS4BE to ASCII --SKIPIF-- --POST-- --GET-- --FILE-- --EXPECT-- abcd abcd ===== end of ext/iconv/002.phpt ======= "--TEST--" is title of the test. "--SKIPIF--" is condition when to skip this test. "--POST--" is POST variable passed to test script. "--GET--" is GET variable passed to test script. "--FILE--" is the test script. "--EXPECT--" is the expected output from the test script. ext/iconv/002.phpt uses 2 files. "skipif.inc" is used to skip test when test cannot be performed such as iconv module is not compiled or loaded. ==== ext/iconv/skipif.inc ==== ==== end of ext/iconv/skipif.inc ==== ext/inconv/002.inc is the test script. "run-tests.php" script executes test script with CGI executable. ==== ext/iconv/002.inc === === end of ext/iconv/002.inc === Test script and skipif code may be directly write into *.phpt. However, it is recommended to use other files for ease of writing test script. For instance, you can execute test script under ext/iconv as follows: ./sapi/cli/php -c /path/to/php.ini/ ext/iconv [How to help us] ---------------- If you find bug in PHP, you can submit bug report AND test script for us. You don't have to write complete script, just give us test script with following format. Please test the script and make sure you write the correct ACTUAL OUTPUT and EXPECTED OUTPUT before you submit.