summaryrefslogtreecommitdiff
path: root/lib/Benchmark.t
Commit message (Collapse)AuthorAgeFilesLines
* Benchmark.t - sanity check for a broken times()Yves Orton2023-02-231-0/+83
| | | | | | | | If time() or times() is broken then Benchmark can infinite loop. This adds a sanity check that will die early if it appears that either are broken. This fixes the infinite loop part of GH Issue #20839
* use is_deeply instead of eq_ Test::More functionsMichiel Beijen2021-12-241-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the eq_set, eq_hash, and eq_array functions in Test::More are discouraged to use, and is_deeply is recommended. Ref: https://metacpan.org/pod/Test::More#Discouraged-comparison-functions The reason for this is that, if the tests fail, is_deeply has much better diagnostics. The other thing is that is_deeply is a test function directly, where eq_hash and such need to be wrapped inside ok(). This is easy to forget -- proof of this is in Benchmark.t, where we had this code, that did not test anything: eq_set([keys %$got], [qw(Foo Bar Baz)], 'should be exactly three objects'); It is now replaced by: is_deeply([sort keys %$got], [sort qw(Foo Bar Baz)], 'should be exactly three objects'); this commit replaces all usage of eq_set, eq_hash, and eq_array in lib/ and ext/ for tests that use Test::More. One small exception is where a negated test is used; Test::More does not have is_not_deeply() or such. Test2 has `isnt()` for this, but that is not in core. In those cases, we still keep using the eq_ operators.
* do not count tests, instead use done_testing()Michiel Beijen2021-12-041-1/+2
| | | | | | | | | | | | | | | Having to count tests is annoying for maintainers. Also, manually updating count tests is problematic when multiple people are working on the same code; it causes merge conflicts and recounts. done_testing() is available since Test::More 0.88 which was released in 2009. This commit changes all tests under lib/ that use Test::More and were planning the number of tests. Michiel Beijen is now a Perl author
* Replace multiple 'use vars' by 'our' in libNicolas R2017-11-111-1/+1
| | | | | | | | Using vars pragma is discouraged and has been superseded by 'our' declarations available in Perl v5.6.0 or later. This commit is about replacing the usage of 'vars' pragma by 'our' in 'lib' directory.
* Benchmark.t: better diagnosticsDavid Mitchell2015-11-281-22/+22
| | | | | | | | use cmp_ok() instesd of ok() where apppriate and similar things to get better diagnostucs when things fail (e.g. display what the bad value was, rather than just letting you know it was bad). Also, use diag() rather than 'print STDERR "# .....\n";'
* Benchmark.t/.pm: deal with short timesDavid Mitchell2015-11-261-5/+8
| | | | | | | | | | | | | | | | | | | The number of iterations to do on various tests was set at 3 or 10 in the first incarnation of the test file 13 years ago, and hasn't changed since. In the meantime, CPUs have gotten faster. So bump the iteration count to 100. This makes no appreciable difference to total wall time for the test file on my newish x86_64 platform, but will hopefully make tests less likely to be running within the noise of a single 100Hz clock tick. In particular, the NetBSD smokes were quite frequently failing tests 127 and 128, due to the CPU taken to do an empty loop being greater than that for a full loop, thus leading to negative apparent execution time. This was likely to be due to the first taking "1" clock tick and the second taking "0" ticks. Although this is less likely to happen now that the iterations has been increased, this commit also adds a check to Benchmark.pm for a negative apparent execution time, and if detected, prints a "too few iterations" warning and resets it to zero.
* Benchmark.t: add more diag outputDavid Mitchell2015-11-251-1/+24
| | | | | | | | | 2 tests in a particular block of code are consistently failing on a newbsd smoker. Make a copy of that block of code but with the result object and chart object generated separately, then diag dump both if any of the tests in that new block fail, so we can see better what's going wrong.
* Benchmark.t: fix marginal testsDavid Mitchell2015-11-251-2/+2
| | | | | | | | Two tests fail a lot of smokes; those tests assert that two code variants when one is faster than the other should show a percentage difference > 0. Then the differences are slight, the percentage may round down to zero and the test will fail. Reduce the likelihood of this my making one code variant a lot slower than the other
* Benchmark.t: avoid ultra-lightweight codeDavid Mitchell2013-10-211-2/+5
| | | | | | | | Some tests benchmarked the code '$i++', which is so lightweight that it could trigger the die "Timing is consistently zero in estimation loop" in Benchmark.pm. So make the code slightly more heavyweight.
* Benchmark.t: remove CPU-speed-sensitive testDavid Mitchell2013-07-191-80/+1
| | | | | | | | | | | | | | Benchmark.t has been randomly failing test 15 in smokes for ages. This is the one that checks that a loop run 3*N times burns approximately 3 times more CPU than when run just N times. For the last month the test has included a calibration loop and test, which does much the same thing, but without using any code from Benchmark.pm. This has failed just as much, which confirms that its an issue with the smoke host (such as a variable speed CPU or whatever), rather than any flaw in the Benchmark.pm library logic. So just remove the calibration loop and the dodgy test.
* Benchmark.t: fix skip() argument orderBrian Gottreu2013-06-201-1/+1
|
* Benchmark.t: fix skip() callDavid Mitchell2013-06-131-1/+1
|
* Benchmark.t: make calibration a testDavid Mitchell2013-06-131-1/+2
| | | | | | As a temporary measure, make a calibration failure not only a skip but a failed test too, so I can see whether the real test fails more often in smokes than the calibration.
* Benchmark.t: consistent delta comparingDavid Mitchell2013-06-121-10/+15
| | | | | | | | | when we compare the number of iterations done in 3 seconds with 3 * (the number of iterations done in 1 second), the comparison's effective delta of toleration depended on which value was larger: if a > b, it tested for a/b <= 1.666; if b > a, it tested for b/a < 1.4. Make it consistently 1.4 (or 1+$DELTA to be precise).
* Benchmark.t: disdplay calibration diag outputDavid Mitchell2013-06-121-0/+3
| | | | | When the notorious test 15 fails, show the ratio of our earlier 3sec and 1sec calibration in the diag output.
* Benchmark.t: test for inconsistent clockDavid Mitchell2013-06-101-2/+63
| | | | | | | | | | | | | | | test 15 has been failing intermittently in smokes for ages now. This does countit(3, ...) and countit(1, ...) and checks that the first takes approx three times longer than the second. This commit adds in near the beginning a crude timing loop that directly uses times() rather than anything in the Benchmark module, and that does approx 1s and 3s of loops, and if the results aren't consistent, sets a global flag, $INCONSISTENT_CLOCK, that causes timing-sensitive tests to be skipped. For now it only skips test 15. If this is successful, I'll look to expand it to other failing tests like 128/129.
* Benchmark.t: ignore sys CPU timeDavid Mitchell2011-06-151-4/+8
| | | | | | | | | | Currently we work out the CPU burned by a loop by summing user and sys CPU. On the grounds that the burning we're interested in should have all been carried out with user CPU and that therefore any sys CPU is a red herring, ignore sys CPU for the infamous test 15. Yes, this is clutching at straws. Still, diagnostics output may show in future whether I was right!
* Benchmark.t: fix count estimateDavid Mitchell2011-06-151-1/+1
| | | | | Commit bb6c6e4b8d10f2e460a7fe48e677d3d998a7f77d, which added improved diagnostics, also broke the count estimate.
* Benchmart.t: improved diagnosticsDavid Mitchell2011-06-101-12/+22
| | | | | use diag() to show absolutely all variables on failing the notorious test 15. Maybe this will help nail it one day...
* RT72688 - Benchmark.t test 24 bug when the testing system is under load.Todd Rinaldo2011-05-181-1/+1
|
* Benchmark.t: note line# of check_graph_consistencyDavid Mitchell2011-03-261-0/+1
| | | | | | This function is called 6 times, each each call puts out about 15 tests, with the same set of descriptions, so output a note at the start of each call showing where we're called from.
* Improve Benchmark.t countit() testsDavid Mitchell2011-03-261-3/+11
| | | | | | | | | | | | | | | | The test currently does a 3 sec run, then a 1 sec run, and checks that the count from the first run is approximately three times greater than that from the second run. However, the countit() function doesn't guarantee that it will run for exactly n seconds, so as well as returning how many iterations it did, it also returns how much actual CPU time was used. Make the test use that returned time value to scale the iteration counts accordingly, before trying to compare them. Hopefully this will reduce the number of spurious failed test 13's in smokes (although after this commit it's now test 15).
* Fix typos (spelling errors) in lib/*Peter J. Acklam) (via RT2011-01-071-3/+3
| | | | | | | | | # New Ticket Created by (Peter J. Acklam) # Please include the string: [perl #81890] # in the subject line of all future correspondence about this issue. # <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=81890 > Signed-off-by: Abigail <abigail@abigail.be>
* Diagnostic improvements to better understand why lib/Benchark.t test 13 has ↵Todd Rinaldo2010-02-021-3/+2
| | | | | | | | been failing smoke randomly. Fix 1: Original code tests for less than but not =. I think that if these values are the same, the test should pass. I don't know the code well enough to be 100% sure. D Fix 2: convert ok() to cmp_ok() for better diagnostic messages from smoke tests when they happen. Fix 3: convert print to diag() so it will properly print through harness.
* [perl #32327] Benchmark calls coderef with spurious arguments Anno Siegel2004-11-051-1/+6
| | | | | | From: Anno Siegel (via RT) <perlbug-followup@perl.org> Message-ID: <rt-3.0.11-32327-99325.8.9408996026507@perl.org> p4raw-id: //depot/perl@23473
* The #20546 was too ambitious, broke the test.Jarkko Hietaniemi2003-08-071-1/+1
| | | p4raw-id: //depot/perl@20556
* [PATCH 5.8.1] Benchmark problemJarkko Hietaniemi2003-08-071-1/+1
| | | | | | | | | | | | From: Radu Greab <rgreab@fx.ro> Date: Thu, 07 Aug 2003 16:18:25 +0300 (EEST) Message-Id: <20030807.161825.106541372.radu@yx.primIT.ro> Subject: Re: [PATCH 5.8.1] Benchmark problem From: Rafael Garcia-Suarez <rgarciasuarez@free.fr> Date: Thu, 7 Aug 2003 15:48:38 +0200 Message-Id: <20030807154838.5d240dbb.rgarciasuarez@free.fr> p4raw-id: //depot/perl@20546
* Re: [PATCH] Re: [perl #23206] Benchmark::cmpthese usage message wrongMichael G. Schwern2003-08-041-21/+95
| | | | | Message-ID: <20030803231235.GJ24350@windhund.schwern.org> p4raw-id: //depot/perl@20463
* OpenBSD sometimes gets -0, probably due to fp fuzziness..Jarkko Hietaniemi2003-07-051-2/+4
| | | p4raw-id: //depot/perl@20016
* Benchmark nit from Nicholas Clark (in OpenBSD the result couldJarkko Hietaniemi2003-04-121-2/+2
| | | | | be a negative zero, -0). p4raw-id: //depot/perl@19191
* Make cmpthese work as documented.Abigail2002-08-251-2/+24
| | | | | Message-ID: <20020822041039.A2089@ucan.foad.org> p4raw-id: //depot/perl@17774
* Re: FreeBSD mostly OK. [nwc10@colon.colondot.net: Report ↵Nicholas Clark2002-05-131-4/+8
| | | | | | | /export/home/nwc10/Even-Smoke/Smoke] Message-ID: <20020513204738.GD310@Bagpuss.unfortu.net> p4raw-id: //depot/perl@16583
* Benchmark.t 75,90,109Nicholas Clark2002-04-131-22/+45
| | | | | Message-ID: <20020413204303.GB12835@Bagpuss.unfortu.net> p4raw-id: //depot/perl@15897
* Better patch for Benchmark.t and arith.tPaul Green2002-03-141-1/+1
| | | | | Message-Id: <200203130025.TAA20113@mailhub1.stratus.com> p4raw-id: //depot/perl@15226
* More diagnostics from Nicholas Clark.Jarkko Hietaniemi2002-02-081-1/+2
| | | p4raw-id: //depot/perl@14592
* Re: [FAIL] Benchmark.t intermittent failureNicholas Clark2002-01-131-2/+2
| | | | | Message-ID: <20020113155833.C314@Bagpuss.unfortu.net> p4raw-id: //depot/perl@14237
* Re: Benchmark.t failure on LinuxNicholas Clark2001-12-181-1/+1
| | | | | Message-ID: <20011218225124.N21702@plum.flirble.org> p4raw-id: //depot/perl@13767
* Show value of $fastslow on failureMichael G. Schwern2001-12-181-1/+2
| | | | | Message-ID: <20011218055818.GC4362@blackrider> p4raw-id: //depot/perl@13754
* Benchmark test from Nicholas Clark.Jarkko Hietaniemi2001-12-141-0/+484
| | | p4raw-id: //depot/perl@13693
* Misplaced test, noticed by Barrie Slaymaker.Jarkko Hietaniemi2001-11-121-88/+0
| | | | | (We are missing Benchmark tests, then.) p4raw-id: //depot/perl@12955
* The Grand Trek: move the *.t files from t/ to lib/ and ext/.Jarkko Hietaniemi2001-06-181-0/+88
No doubt I made some mistakes like missed some files or misnamed some files. The naming rules were more or less: (1) if the module is from CPAN, follows its ways, be it t/*.t or test.pl. (2) otherwise if there are multiple tests for a module put them in a t/ (3) otherwise if there's only one test put it in Module.t (4) helper files go to module/ (locale, strict, warnings) (5) use longer filenames now that we can (but e.g. the compat-0.6.t and the Text::Balanced test files still were renamed to be more civil against the 8.3 people) installperl was updated appropriately not to install the *.t files or the help files from under lib. TODO: some helper files still remain under t/ that could follow their 'masters'. UPDATE: On second thoughts, why should they. They can continue to live under t/lib, and in fact the locale/strict/warnings helpers that were moved could be moved back. This way the amount of non-installable stuff under lib/ stays smaller. p4raw-id: //depot/perl@10676
View Lorry Controller Status