summaryrefslogtreecommitdiff
path: root/tests/regression
Commit message (Collapse)AuthorAgeFilesLines
* arch: Add 32-bit Motorola 68000 supportJohn Paul Adrian Glaubitz2023-04-251-1/+3
| | | | | | Signed-off-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Acked-by: Paul Moore <paul@paul-moore.com> Signed-off-by: Tom Hromatka <tom.hromatka@oracle.com>
* tests: Add 64-bit LoongArch supportXiaotian Wu2023-02-181-1/+3
| | | | | | | | | | | [xen0n: LoongArch (and a few upcoming architectures / ABIs) does not have fstat, so the fstat in 06-sim-actions is also being changed to fstatfs for uniformity across the board.] Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn> Signed-off-by: WANG Xuerui <git@xen0n.name> Acked-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* doc,pyx,tests: Fix some typosManabu Sugimoto2022-01-051-2/+2
| | | | | | Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com> Reviewed-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add support for the LIBSECCOMP_TSTCFG_BATCHES env variablePaul Moore2021-08-121-0/+9
| | | | | | | | This allows us to specify the test batches via environment variables like we do other parts of the test configuration. Acked-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: allow multiple test types using comma separated valuesPaul Moore2021-08-121-1/+18
| | | | | | | | | | | You can now run multiple test types using the '-T' argument and the LIBSECCOMP_TSTCFG_TYPE environment variable, for example: % cd tests % ./regression -T bpf-valgrind,live Acked-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* arch: Add SuperH 32-bit supportJohn Paul Adrian Glaubitz2021-03-081-4/+7
| | | | | | | | | | Initial support for seccomp for SuperH in Linux was added in 2.6.27-rc2, support for SECCOMP_FILTER was added for Linux 5.9. This adds support for SuperH in libseccomp, both for little-endian and big-endian mode. Signed-off-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Acked-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add support for the LIBSECCOMP_TSTCFG_JOBS env variablemasterPaul Moore2021-01-141-5/+8
| | | | | | | | | | | | | The LIBSECCOMP_TSTCFG_JOBS environment variable allows the caller to specify the number of test jobs through the environment as opposed to the command line. This is primarily useful for running the tests through scripts or via the 'make check' command. The LIBSECCOMP_TSTCFG_JOBS environment variable value behaves the same as the '-j <JOBS>' command line argument. Reviewed-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add basic support for running tests in parallelPaul Moore2021-01-141-48/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the beginning it didn't matter that we were running the regression tests serially, but as we are now running +16k tests the run time is getting rather long. As there is no good reason why we can't run these tests in parallel, let's add some basic support to do so. This patch adds support for running multiple tests jobs at once using the '-j <NUM>' flag, similar to the "make" command. If the number of jobs specified is invalid/zero then it is set to the number of CPUs present on the system. If the '-j <NUM>' flag is not specified then the tests are executed in serial fashion as they are done now. If the '-l <LOG>' option is specified the test run reverts to serial execution regardless of the command line in order to preserve the log output. While the normal console output is preserved regardless of the number of jobs, the logfile output is handled differently and this restriction was the easiest solution. We should consider removing the '-l <LOG>' option at some point since output capture and redirection is likely best handled by the shell anyway. As far as the performance improvements are concerned, the results speak for themselves. On my eight core laptop the runtime drops from ~14 minutes to ~4 minutes (!). * Existing code, single threaded (14 minutes, 14 seconds) % time -- ./regression -m c -m python =============== Tue Jan 5 06:11:52 PM EST 2021 =============== Regression Test Report ("regression -m c -m python") batch name: 01-sim-allow test mode: c test type: bpf-sim Test 01-sim-allow%%001-00001 result: SUCCESS ... Test 58-live-tsync_notify%%001-00001 result: SKIPPED (must specify live tests) Regression Test Summary tests run: 16412 tests skipped: 130 tests passed: 16412 tests failed: 0 tests errored: 0 ============================================================ real 854.37 user 693.87 sys 269.25 * Patched code, 8 jobs (4 minutes, 7 seconds) % time -- ./regression -j 8 -m c -m python =============== Tue Jan 5 06:27:56 PM EST 2021 =============== Regression Test Report ("regression -j 8 -m c -m python") batch name: 01-sim-allow test mode: c test type: bpf-sim Test 01-sim-allow%%001-00001 result: SUCCESS ... Test 58-live-tsync_notify%%001-00001 result: SKIPPED (must specify live tests) Regression Test Summary tests run: 16412 tests skipped: 130 tests passed: 16412 tests failed: 0 tests errored: 0 ============================================================ real 246.96 user 966.08 sys 251.27 Reviewed-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: don't use the "tr" toolPaul Moore2020-03-111-1/+2
| | | | | | | | This appears to cause problems in the Travis CI with the aarch64 architecture. Acked-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* arch: Add RISC-V 64-bit supportAndreas Schwab2020-02-231-2/+4
| | | | | | Signed-off-by: Andreas Schwab <schwab@suse.de> [PM: minor macro shuffling in seccomp.h.in] Signed-off-by: Paul Moore <paul@paul-moore.com>
* travis: avoid running live python tests in TravisCIChris Waldon2020-01-201-0/+4
| | | | | Signed-off-by: Chris Waldon <chris.waldon@ibm.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add support for 32-bit and 64-bit test filteringPaul Moore2019-03-041-0/+36
| | | | | | | Add "[+]all_32" and "[+]all_64" which work the same as the little and big endian test filters. Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add an API level check to the live testsPaul Moore2018-09-221-2/+14
| | | | Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: Add tests for SECCOMP_RET_KILL_PROCESSTom Hromatka2018-09-191-1/+6
| | | | | | | | This addresses GitHub Issue #96 - RFE: add support for SECCOMP_RET_KILL_PROCESS Signed-off-by: Tom Hromatka <tom.hromatka@oracle.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* build: enable distcheck'ing for the python codePaul Moore2018-05-101-1/+6
| | | | | | | | I'm not particularly proud of the seccomp.pyx hack, but it works, and enabling the python bindings during the distcheck is definitely the "Greater Good". Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: test suite infrastructure changes for SCMP_ACT_LOGTyler Hicks2017-11-011-0/+5
| | | | | | | | The basics needed to handle tests that use the new SCMP_ACT_LOG action. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: fix conditional that was skipping all basic python testsTyler Hicks2017-09-121-1/+1
| | | | | | | | | | | A conditional added in ec6f45ab was incorrectly comparing the (empty) stdout of grep -q against 0, which always evaluated to be true and skipped the basic python tests. Fix it by using bash's pattern matching. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: fix some problems with 'make distcheck'Paul Moore2017-02-271-1/+9
| | | | | | | This commit also renames some of the variables in the tests directory to better reflect their meaning. Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add a dedicated PFC testPaul Moore2017-02-241-0/+7
| | | | | | | This commit also adds special shell-script handling for the "basic" tests. Signed-off-by: Paul Moore <paul@paul-moore.com>
* all: add tests to ensure that syscall -1 is handled correctlyPaul Moore2017-02-231-2/+2
| | | | Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add support for $LIBSECCOMP_TSTCFG_STRESSCNTPaul Moore2017-02-161-0/+4
| | | | | | | Allow the fuzz tests to be configured via the LIBSECCOMP_TSTCFG_STRESSCNT environment variable. Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: add test configuration via environment variablesPaul Moore2017-02-071-0/+4
| | | | | | | Allow the test type to be specified by the LIBSECCOMP_TSTCFG_TYPE environment variable. Signed-off-by: Paul Moore <paul@paul-moore.com>
* arch: Add parisc architecture supportHelge Deller2016-05-251-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch (v4) adds support for the parisc and parisc64 architectures to libseccomp. I didn't split up the patch, because it's pretty trivial. Those parisc-specific files gets added: src/arch-parisc-syscalls.c src/arch-parisc.c src/arch-parisc.h src/arch-parisc64.c All other changes are trivial because they simply add parisc-specific case statements in variouse switch statements. I did ran a "make check" on x86-64 and parisc and all testcases succeeded. All live testcases succeed as well when running "./regression -T live". The patch applies cleanly to current libseccomp git head. Changes between v4 and v3 of this patch: - Added parisc arch to arch-syscall-check.c and fixup syscall table as needed - Fixed copyright notices in parisc files as suggested by Mike Frysinger Changes between v3 and v2 of this patch: - Stripped out patch which reports if a check was skipped because valgrind isn't installed. - Added tuxcall pseudo syscall for 19-sim-missing_syscalls testcase - Added sysmips pseudo syscall for 29-sim-pseudo_syscall testcase Changes between v2 and v1 of this patch: - Enabled seccomp mode 2 regression tests on parisc. Kernel support for hppa was added in kernel 4.6-rc1 and backported into the kernel v4.5.2 stable series. Signed-off-by: Helge Deller <deller@gmx.de> [PM: corrected a number or errors from 'make check-syntax'] Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: always display skipped testsPaul Moore2016-04-181-3/+7
| | | | Signed-off-by: Paul Moore <paul@paul-moore.com>
* tests: Report if test was skipped because of missing valgrindHelge Deller2016-04-181-0/+1
| | | | | | | | | | | | Print a message if a regression test was skipped just because valgrind isn't installed. This additional info is useful for architectures (e.g. parisc) which aren't as fast as x86_64. On parisc it seemed that the whole testcase was hanging, now people know what's actually happening. Signed-off-by: Helge Deller <deller@gmx.de> Signed-off-by: Paul Moore <paul@paul-moore.com>
* Merge branch 'working-s390' into masterPaul Moore2015-08-281-2/+3
|\ | | | | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
| * arch: add support for s390Jan Willeke2015-07-131-2/+2
| | | | | | | | | | | | | | | | This patch adds support for S390 (32-bit) architecture. Signed-off-by: Jan Willeke <willeke@linux.vnet.ibm.com> [PM: rewrote the subject line, style fixes, s390x/s390 typo bugfixes] Signed-off-by: Paul Moore <pmoore@redhat.com>
| * arch: add support for s390xJan Willeke2015-07-131-2/+2
| | | | | | | | | | | | | | | | This patch adds support for S390x (64-bit) architecture. Signed-off-by: Jan Willeke <willeke@linux.vnet.ibm.com> [PM: rewrote the subject line, style fixes] Signed-off-by: Paul Moore <pmoore@redhat.com>
* | tests: fix miniseq and "make distcheck"Paul Moore2015-08-271-1/+14
| | | | | | | | | | | | This is a bit of a hack, but it works; fix it later. Signed-off-by: Paul Moore <pmoore@redhat.com>
* | Merge branch 'working-ppc64'Paul Moore2015-08-051-3/+9
|\ \ | | | | | | | | | | | | | | | Mike Strosaker <strosake@linux.vnet.ibm.com> reports that the working-ppc64 branch is working for both 'make check' and './regression -T live'.
| * | tests: add ppc support to the regression testsBogdan Purcareata2015-02-171-2/+2
| | | | | | | | | | | | | | | Signed-off-by: Bogdan Purcareata <bogdan.purcareata@freescale.com> Signed-off-by: Paul Moore <pmoore@redhat.com>
| * | tests: add ppc64 support to the regression live testsBogdan Purcareata2015-02-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise The live tests will fail with "ERROR arch ppc64 not supported" Send against the working-ppc64 branch. Signed-off-by: Bogdan Purcareata <bogdan.purcareata@freescale.com> [PM: added ppc64le] Signed-off-by: Paul Moore <pmoore@redhat.com>
| * | tests: add ppc64 support to the regression testsPaul Moore2015-02-061-2/+8
| |/ | | | | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
* | tests: fix sequence number generationMathias Krause2015-07-161-9/+25
|/ | | | | | | | | | | | | | Because of incorrect number sequence generation by seq(1) on at least 32 bit ARM systems using coreutils v8.23/v8.24 we provide a minimal seq(1) implementation that fits our needs. This fixes the bug mentioned in the following mailing thread: https://groups.google.com/forum/#!topic/libseccomp/VtrClkXxLGA Signed-off-by: Mathias Krause <minipli@googlemail.com> [PM: subject line, build locations, and vertical whitespace tweaks] Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: allow the regression test to run properly from 'make distcheck'Paul Moore2014-10-211-5/+7
| | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
* arch: Add AArch64 supportMarcin Juszkiewicz2014-08-271-2/+2
| | | | | | | | This patch adds support for AArch64 (64-bit ARM) architecture. Signed-off-by: Marcin Juszkiewicz <mjuszkiewicz@redhat.com> (Additional fixes/corrections/etc.) Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: better architecture selection support in the automated testsPaul Moore2014-08-271-13/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support for a number of new enhancements to the automated test suite, all of which are focused on the architecture selection of the bpf-sim test type. With this patch, the architecture field can now contain a comma delimited list of architecture names with the following values: * all Add the current native arch to the list. * all_le Add the current native arch to the list only if it is little endian. * +all_le Add all of the supported little endian architectures to the list. * all_be Add the current native arch to the list only if it is big endian. * +all_be Add all of the supported big endian architectures to the list. * <arch> Add the architecture specified by "<arch>" if it is the native architecture. * +<arch> Add the architecture specified by "<arch>" to the list. * -<arch> Remove the architecture specified by "<arch>" to the list if present. Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: Add missing mips{el}64n32 case for live testsMarkos Chandras2014-08-211-1/+1
| | | | | Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Paul Moore <pmoore@redhat.com>
* all: add support for mips64n32 to the tools and testsPaul Moore2014-08-201-2/+2
| | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
* all: add support for mips64 to the tools and testsPaul Moore2014-08-201-3/+3
| | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: detect the python bindings automaticallyPaul Moore2014-05-011-1/+11
| | | | Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: Add MIPS BE return codes for live testsMarkos Chandras2014-04-251-1/+1
| | | | | Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: support both big and little endian test architecturesPaul Moore2014-04-161-3/+6
| | | | | | | | Replace the "+all" architecture tokens in the test files with the "+all_le" and "+all_be" tokens to represent all the little endian and big endian architectures respectively. Signed-off-by: Paul Moore <pmoore@redhat.com>
* arch: Add support for MIPS Little EndianMarkos Chandras2014-04-151-1/+8
| | | | | | | Add support for MIPS 32-bit (O32 ABI) Little Endian Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: add support for valgrind suppressionsPaul Moore2014-04-151-1/+2
| | | | | | | Add support for valgrind suppressions files to the test tools. As part of this patch include a fix for valgrind 3.9.0 and glibc 2.19. Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: add basic support for arch specific return values for the live testsPaul Moore2014-02-131-11/+22
| | | | | | | Evidently the return value for a killed process can differ between architectures/ABIs due to differences in the SIGSYS value. Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: fix a problem with the python bindings and the bpf-valgrind testsPaul Moore2013-10-211-0/+6
| | | | | Reported-by: Andy Lutomirski <luto@amacapital.net> Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: make the regression test valgrind mode more usefulPaul Moore2013-10-181-41/+88
| | | | | | | We don't really need to run valgrind multiple times for each test case, just once should be enough. Signed-off-by: Paul Moore <pmoore@redhat.com>
* tests: return a useful error code from the test harnessPaul Moore2013-10-141-1/+5
| | | | | | | With this patch the test harness returns 0 if there were no errors or failures detected, positive values if a error or failure was detected. Signed-off-by: Paul Moore <pmoore@redhat.com>
* tools: rename some of the tools in preparation for installationPaul Moore2013-05-231-4/+7
| | | | | | | | Some of the tools are quite handy and would likely be useful for developers incorporating libseccomp into their applications so let's go ahead and namespace the tools in preparation for installing them. Signed-off-by: Paul Moore <pmoore@redhat.com>
View Lorry Controller Status