<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/php-git.git/run-tests.php, branch master</title>
<subtitle>git.php.net: repository/php-src.git
</subtitle>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/'/>
<entry>
<title>run-tests: use the EXTENSIONS section for skipping</title>
<updated>2021-03-22T08:53:38+00:00</updated>
<author>
<name>Max Semenik</name>
<email>maxsem.wiki@gmail.com</email>
</author>
<published>2021-03-18T11:53:14+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=6c9a05667b72c6420eb21269ffaffbd93178923c'/>
<id>6c9a05667b72c6420eb21269ffaffbd93178923c</id>
<content type='text'>
Currently, most skip checks are just for making sure an extension is
available. Even with recent addition of skip caching, this makes tests
needlessly slow:
* Checks for the same extension in its tests can have small differences
  impacting cacheability.
* Even identical skip checks in two tests can still be executed twice if
  they're run by different workers.

To remedy this, I'm repurposing the existing --EXTENSIONS-- section of
.phpt files to specify wjich extensions are required for current test to
run. Current behavior:
1) If the extension is already visible to PHP, all is good
2) If it isn't, assume it's present as a shared module and attempt to add
   it via a command line parameter
3) If that works, all is good
4) If it doesn't, PHP fails with a cryptic error message trying to
   execute the test itself

After this commit:
1) and 2) are kept unchanged
3) Check if shared extension file from 2) is actually present
4) Skip the test if it isn't

Other benefits include clear skip reasons (vs. sometimes none in many
current skip checks) and moving test information from code to metadata,
opening more opportunities for search and analysis.

Since --EXTENSIONS-- is barely ever used, this change poses no risk of
hidden failures.

As a demonstration of the new approach, this commit migrates one
extension to it. If merged, I will migrate other extensions in
subsequent PRs.

Closes GH-6787.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Currently, most skip checks are just for making sure an extension is
available. Even with recent addition of skip caching, this makes tests
needlessly slow:
* Checks for the same extension in its tests can have small differences
  impacting cacheability.
* Even identical skip checks in two tests can still be executed twice if
  they're run by different workers.

To remedy this, I'm repurposing the existing --EXTENSIONS-- section of
.phpt files to specify wjich extensions are required for current test to
run. Current behavior:
1) If the extension is already visible to PHP, all is good
2) If it isn't, assume it's present as a shared module and attempt to add
   it via a command line parameter
3) If that works, all is good
4) If it doesn't, PHP fails with a cryptic error message trying to
   execute the test itself

After this commit:
1) and 2) are kept unchanged
3) Check if shared extension file from 2) is actually present
4) Skip the test if it isn't

Other benefits include clear skip reasons (vs. sometimes none in many
current skip checks) and moving test information from code to metadata,
opening more opportunities for search and analysis.

Since --EXTENSIONS-- is barely ever used, this change poses no risk of
hidden failures.

As a demonstration of the new approach, this commit migrates one
extension to it. If merged, I will migrate other extensions in
subsequent PRs.

Closes GH-6787.
</pre>
</div>
</content>
</entry>
<entry>
<title>run-tests.php: class for test file loading</title>
<updated>2021-03-16T10:51:55+00:00</updated>
<author>
<name>Max Semenik</name>
<email>maxsem.wiki@gmail.com</email>
</author>
<published>2021-02-10T12:17:03+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=9140c9038a83ff55a78f357f8485de086d83d94e'/>
<id>9140c9038a83ff55a78f357f8485de086d83d94e</id>
<content type='text'>
This moves a bunch of code outside of run_tests(), making it a bit
more manageable. Additionally, accessors provide better readability
than isset() and friends.

This is a minimal patch that moves the code but does not refactor
much. For the sake of reviewing experience, it does not involve
further refactoring which could include:
* Removing setSection()
* Fixing up the mess with hasSection() vs. sectionNotEmpty(), only
  one of which is really needed.
* Moving more repetitive code into the new class.
All of this will be done with later commits.

Closes GH-6678.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This moves a bunch of code outside of run_tests(), making it a bit
more manageable. Additionally, accessors provide better readability
than isset() and friends.

This is a minimal patch that moves the code but does not refactor
much. For the sake of reviewing experience, it does not involve
further refactoring which could include:
* Removing setSection()
* Fixing up the mess with hasSection() vs. sectionNotEmpty(), only
  one of which is really needed.
* Moving more repetitive code into the new class.
All of this will be done with later commits.

Closes GH-6678.
</pre>
</div>
</content>
</entry>
<entry>
<title>Make the nocache tag actually work</title>
<updated>2021-03-04T15:34:10+00:00</updated>
<author>
<name>Nikita Popov</name>
<email>nikita.ppv@gmail.com</email>
</author>
<published>2021-03-04T15:32:58+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=0074a1d4e3a85d0d63118e7a30f4b7ed6da64695'/>
<id>0074a1d4e3a85d0d63118e7a30f4b7ed6da64695</id>
<content type='text'>
If nocache is used, then we should return an empty string result,
so the test doesn't get marked as borked.

Also limit to an occurrence at the start, just like other skip
tags.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If nocache is used, then we should return an empty string result,
so the test doesn't get marked as borked.

Also limit to an occurrence at the start, just like other skip
tags.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add mechanism to skip the skipif cache</title>
<updated>2021-03-04T15:28:11+00:00</updated>
<author>
<name>Nikita Popov</name>
<email>nikita.ppv@gmail.com</email>
</author>
<published>2021-03-04T15:26:47+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=4d43cbe333690171753e9b8663df93d3762e02a8'/>
<id>4d43cbe333690171753e9b8663df93d3762e02a8</id>
<content type='text'>
If the SKIPIF section return nocache, don't cache the result. This
is used by tests that perform complex test set in conjunction with
skip checks, and need to be evaluated every time.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If the SKIPIF section return nocache, don't cache the result. This
is used by tests that perform complex test set in conjunction with
skip checks, and need to be evaluated every time.
</pre>
</div>
</content>
</entry>
<entry>
<title>run-tests.php: Preserve .php on leak in one more place</title>
<updated>2021-03-04T15:09:29+00:00</updated>
<author>
<name>Nikita Popov</name>
<email>nikita.ppv@gmail.com</email>
</author>
<published>2021-03-04T15:08:53+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=b853201a6d201da692b83b7d986170a2bcc989e8'/>
<id>b853201a6d201da692b83b7d986170a2bcc989e8</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>run-tests: drop support for ancient Valgrind versions</title>
<updated>2021-03-01T10:21:30+00:00</updated>
<author>
<name>Max Semenik</name>
<email>maxsem.wiki@gmail.com</email>
</author>
<published>2021-02-25T10:15:03+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=bdd7c487646bbaaf4ab5616fa73661f4e9337fa4'/>
<id>bdd7c487646bbaaf4ab5616fa73661f4e9337fa4</id>
<content type='text'>
Valgrind 3.3.0 was released in 2007, not even RHEL has crap older than
this. It could be argued that 3.8.0, released in 2012, could be a safe
cutoff too.

Closes GH-6728.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Valgrind 3.3.0 was released in 2007, not even RHEL has crap older than
this. It could be argued that 3.8.0, released in 2012, could be a safe
cutoff too.

Closes GH-6728.
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'PHP-8.0'</title>
<updated>2021-02-25T11:05:27+00:00</updated>
<author>
<name>Christoph M. Becker</name>
<email>cmbecker69@gmx.de</email>
</author>
<published>2021-02-25T11:05:27+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=51914610ab8bf75d87e5cdec15bd1b38de7907c5'/>
<id>51914610ab8bf75d87e5cdec15bd1b38de7907c5</id>
<content type='text'>
* PHP-8.0:
  Capture Microsoft-defined HRESULT exit codes exit codes
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* PHP-8.0:
  Capture Microsoft-defined HRESULT exit codes exit codes
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'PHP-7.4' into PHP-8.0</title>
<updated>2021-02-25T11:03:43+00:00</updated>
<author>
<name>Christoph M. Becker</name>
<email>cmbecker69@gmx.de</email>
</author>
<published>2021-02-25T11:03:43+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=fb683030bf770a946c53c4d622a9d830c2fa2ad6'/>
<id>fb683030bf770a946c53c4d622a9d830c2fa2ad6</id>
<content type='text'>
* PHP-7.4:
  Capture Microsoft-defined HRESULT exit codes exit codes
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* PHP-7.4:
  Capture Microsoft-defined HRESULT exit codes exit codes
</pre>
</div>
</content>
</entry>
<entry>
<title>Capture Microsoft-defined HRESULT exit codes exit codes</title>
<updated>2021-02-25T11:01:04+00:00</updated>
<author>
<name>Dylan K. Taylor</name>
<email>odigiman@gmail.com</email>
</author>
<published>2021-02-23T21:34:51+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=a480bf80933e4d6a8c91a5992b4e5d73da625e17'/>
<id>a480bf80933e4d6a8c91a5992b4e5d73da625e17</id>
<content type='text'>
The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue. I
discovered this a while ago due to bad tests in an extension I maintain.

Closes GH-6722.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue. I
discovered this a while ago due to bad tests in an extension I maintain.

Closes GH-6722.
</pre>
</div>
</content>
</entry>
<entry>
<title>run-tests: add skip cache</title>
<updated>2021-02-23T14:05:49+00:00</updated>
<author>
<name>Max Semenik</name>
<email>maxsem.wiki@gmail.com</email>
</author>
<published>2021-02-11T17:33:03+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/php-git.git/commit/?id=dda0cea5cd7875a27dc052df4c8c1ac40a5b17b4'/>
<id>dda0cea5cd7875a27dc052df4c8c1ac40a5b17b4</id>
<content type='text'>
Currently every --SKIPIF-- section in every test file results in 1
extra execution of PHP, every --EXTENSIONS-- section - in 2 executions.
This is quite wasteful, as skip checking code is extremely repetitive
and extensions are fixed for every binary/ini/command parameters
combination.

This patch adds caching to all such checks.

On my machine, the gains are quite noticeable: 36s instead of 43s
with -j16, 292s instead of 337s without concurrency. Cache stats are
3780 hits, 1247 misses in the latter case. In the future, tests could
be adjusted to have more uniform skip checks to improve performance even
more.

Closes GH-6681.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Currently every --SKIPIF-- section in every test file results in 1
extra execution of PHP, every --EXTENSIONS-- section - in 2 executions.
This is quite wasteful, as skip checking code is extremely repetitive
and extensions are fixed for every binary/ini/command parameters
combination.

This patch adds caching to all such checks.

On my machine, the gains are quite noticeable: 36s instead of 43s
with -j16, 292s instead of 337s without concurrency. Cache stats are
3780 hits, 1247 misses in the latter case. In the future, tests could
be adjusted to have more uniform skip checks to improve performance even
more.

Closes GH-6681.
</pre>
</div>
</content>
</entry>
</feed>
