02: Running the test suite
Now that we've compiled PHP from source, we need to learn how to run the tests using PHP's black-box testing tool called run-tests.php.
Where the test files live
The test files live in various places throughout the PHP source code.
-
ext/{extension-name}/tests/Extension tests -
sapi/{sapi-name}/tests/SAPI tests -
Zend/tests/Zend engine tests -
tests/More Zend engine tests
Running a test as plain-old PHP
We looked at the basic test at tests/basic/001.phpt and saw it was a normal PHP file so we ran it with our compiled binary.
$ sapi/cli/php tests/basic/001.phpt
Running the test suite
The two primary ways of running the test suites are 1) directly through run-tests.php and 2) the make target test.
Running run-tests directly
We first invoked the help screen for run-tests.
$ sapi/cli/php run-tests.php --help
The run-tests tool won't run unless we specify the PHP binary that will run the tests. There are three primary ways of specifying a PHP binary.
- You can set the environment variable
TEST_PHP_EXECUTABLE.
$ export TEST_PHP_EXECUTABLE=`pwd`/sapi/cli/php
$ sapi/cli/php run-tests.php
- You can set the
-pflag.
$ sapi/cli/php run-tests.php -p `pwd`/sapi/cli/php
- You can set the
-Pflag.
$ sapi/cli/php run-tests.php -P
By default run-tests will run the entire test suite so we specified a single test by passing it as an argument.
$ sapi/cli/php run-tests.php -P tests/basic/001.phpt
You can also specify a directory or list other tests or directories.
$ sapi/cli/php run-tests.php -P tests/basic/ ext/standard/tests/random/
Running run-tests with make
The easiest way to run run-tests is with the make target test.
$ make test
We specify the tests we want to run with TESTS.
$ make test TESTS=ext/standard/tests/random/
$ make test TESTS="ext/standard/tests/random/ ext/standard/tests/password"
Test statuses
After running the tests, run-tests will mark the test as one of the following statuses:
- Passed The test passed
- Failed The test failed
- Borked Invalid test file (this shouldn't happen out of the box)
-
Warned A warning was raised from the
--SKIPIF--section (more on this later) -
Leaked A memory leak was detected (Valgrind must be enabled with the
-mflag) - XFailed The test failed, but it was expected to (weird)
-
Slow Test took longer than timeout (defaults to 60 seconds or 300 seconds if testing for memory leaks; timeout can be customized with
--set-timeout)
Resources
Truncated by Planet PHP, read more at the original (another 1344 bytes)