summaryrefslogtreecommitdiff
path: root/test
Commit message (Collapse)AuthorAgeFilesLines
* Add a few coverage for legacy crc codeKota Tsuyuzaki2020-10-021-11/+24
| | | | | | | | | it actually doesn't assert the value becase we now use same if statement for the assertion but hope it to be even a little better than no covarage. Change-Id: I8860a2a8227e43e02afddcbad1e108157c0872f6
* Be willing to write fragments with legacy crcTim Burke2020-09-301-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ...if users *really* want to. They opt-in at run time by setting LIBERASURECODE_WRITE_LEGACY_CRC=1 in the environment; leaving it unset, set to an empty string, or set to the string "0" continues to write zlib crcs. UpgradeImpact ============= This option is intended to allow a smooth upgrade from liberasurecode 1.5.0 and earlier in a system with multiple readers and writers: * Before upgrade, ensure the environment variable is set on all nodes. This will be ignored by earlier versions. * Upgrade liberasurecode on each node in the system, restarting any services that use it. Every node continues writing CRCs that are still usable by nodes that have not yet upgraded. * Now that every node is capable of reading zlib CRCs, remove the environment variable from each node to start writing new CRCs. If you are already using 1.6.0 or later, just upgrade normally. Closes-Bug: #1886088 Closes-Bug: #1867937 Related-Bug: #1666320 Needed-By: https://review.opendev.org/#/c/739164/ Change-Id: I9adfbe631a2dddc592fd08f8a325f3e8331b92f1
* Do not discard a technical return value in testsPete Zaitcev2020-02-051-4/+12
| | | | | | | | | | | | | Compilers are getting smarter, and we started getting this: libec_slap.c: In function 'test_hd_code': libec_slap.c:285:14: error: 'frags.array' may be used uninitialized in this function [-Werror=maybe-uninitialized] The fix is to consume the error code in such a way that the test proceeds further only when frags are indeed initialized. Change-Id: I54db0172a36419206d00b22608523a08818f41f6
* Fix create_fake_frags_no_meta to use memset to fill fragsCorey Bryant2020-01-281-8/+5
| | | | | | | These aren't C strings; we shouldn't be using strncpy and friends. Change-Id: I50cd7922dfa377ea27f3c9558a8a7268120ec733 Closes-Bug: #1859710
* Update test to build on FreeBSD 12donnydavis2019-07-021-0/+1
| | | | | Change-Id: I1ab792c3c509539a73a1808b8c28c6748185ae00 Closes-Bug: 1835071
* Allow reading of little-endian frags on big-endianTim Burke2018-09-111-7/+56
| | | | | | | | ... and vice-versa. We'll fix up frag header values for our output parameter from liberasurecode_get_fragment_metadata but otherwise avoid manipulating the in-memory fragment much. Change-Id: Idd6833bdea60e27c9a0148ee28b4a2c1070be148
* Print test that's going to run *before* running testTim Burke2017-09-131-30/+30
| | | | | | Should make it easier to diagnose failures. Change-Id: I32ed3019c556706634aae91844bf57b0954af86a
* Merge "flat_xor_hd: better validate args"Jenkins2017-09-131-0/+23
|\
| * flat_xor_hd: better validate argsTim Burke2017-09-131-0/+23
| | | | | | | | | | | | Previously, we'd segfault with args like k=1, m=3, hd=3. Change-Id: I8a1d92f388c54fe4f5b357eaf85a27e6b334df11
* | Merge "Negative data or parity args are invalid"Jenkins2017-09-131-19/+17
|\ \ | |/ |/|
| * Negative data or parity args are invalidTim Burke2017-09-131-19/+17
| | | | | | | | | | | | | | While we're at it, tighten up some test_create_backend_invalid_args assertions. Change-Id: Id6c70cdb2d86580280ededc3ec6ec648c6cb7d57
* | Merge "Use preprocessor to build test suites"Jenkins2017-09-131-637/+55
|\ \
| * | Use preprocessor to build test suitesTim Burke2017-07-131-637/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise, we're left with error-prone copy & pasting, which means we'll constantly be cleaning it up. Change-Id: I57e2cbef2c9221cffccf2e73b6af9bd003c04968 Related-Change: Ibd72ba4ae609ad77e8808aa1594b0adb62e34ef0 Related-Change: I9ee9ec3d8f86a10c4c7b5d6425a530b9c44d1156
* | | Merge "Use more backends in existing tests"Jenkins2017-09-131-0/+44
|\ \ \ | |/ /
| * | Use more backends in existing testsTim Burke2017-07-131-0/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to start defining test suites for common capabilities / allowed failure modes. Null Backend: test_decode_with_missing_parity test_decode_with_missing_multi_parity Jerasure RS Vand: test_decode_with_missing_parity Jerasure RS Cauchy: test_decode_with_missing_parity ISA-L RS Vand: test_decode_with_missing_parity test_get_fragment_metadata_crc32 ISA-L RS Cauchy: test_decode_with_missing_parity test_get_fragment_metadata_crc32 SHSS: test_decode_with_missing_parity test_get_fragment_metadata_crc32 liberasurecode RS Vand: test_decode_with_missing_parity Change-Id: I9ee9ec3d8f86a10c4c7b5d6425a530b9c44d1156
* | | Merge "Make test format strings consistent with test function names"Jenkins2017-09-131-71/+71
|\ \ \ | |/ /
| * | Make test format strings consistent with test function namesTim Burke2017-07-131-71/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * When we specify a particular backend, we don't need to include it in the display string; we'll already include it in the output. * test_destroy_backend_invalid_args shouldn't be reported as test_create_backend_invalid_args. * Consistently include the leading "test_" in the display string. Change-Id: Ibd72ba4ae609ad77e8808aa1594b0adb62e34ef0
* | | Un-inline get/set_metatdata_chksumTim Burke2017-07-061-4/+0
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each was only really used in one place, they had some strange return types, and recent versions of clang on OS X would refuse to compile with erasurecode_helpers.c:531:26: error: taking address of packed member 'metadata_chksum' of class or structure 'fragment_header_s' may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member] return (uint32_t *) &header->metadata_chksum; ^~~~~~~~~~~~~~~~~~~~~~~ We don't really *care* about the pointer; we just want the value! Change-Id: I8a5e42312948a75f5dd8b23b6f5ccfa7bd22eb1d
* | Use zlib for CRC-32Tim Burke2017-07-062-2/+36
|/ | | | | | | | | | | | | | | | Previously, we had our own CRC that was almost but not quite like zlib's implementation. However, * it hasn't been subjected to the same rigor with regard to error-detection properties and * it may not even get used, depending upon whether zlib happens to get loaded before or after liberasurecode. Now, we'll use zlib's CRC-32 when writing new frags, while still tolerating frags that were created with the old implementation. Change-Id: Ib5ea2a830c7c23d66bf2ca404a3eb84ad00c5bc5 Closes-Bug: 1666320
* Merge "Stop running targeted tests repeatedly"Jenkins2017-05-101-0/+3
|\
| * Stop running targeted tests repeatedlyTim Burke2017-02-251-0/+3
| | | | | | | | | | | | | | Previously, we would run them four times, despite always using the same backends and arguments. Change-Id: Iede8e9e9689cbaf85c66adce72cb78983afb084b
* | Merge "Stop running test_simple_encode_decode_over32 repeatedly"Jenkins2017-05-101-5/+5
|\ \ | |/
| * Stop running test_simple_encode_decode_over32 repeatedlyTim Burke2017-02-251-5/+5
| | | | | | | | | | | | | | | | | | It uses hard-coded backend and arguments, so it should use EC_BACKEND_MAX and have a name that reflect how it's only targeted to Jerasure. This is similar to how we handled the targeted test case for ISA-L. Change-Id: I0a397930b2f49c0775290970c820bc70310a475e
* | Merge "Jerasure: Handle initialization errors correctly"Jenkins2017-05-101-5/+27
|\ \
| * | Jerasure: Handle initialization errors correctlyTim Burke2017-03-161-5/+27
| |/ | | | | | | | | | | | | Otherwise we can get backtraces where we try to free something that was never initialized. Change-Id: Iaea427b977fd20819e2da5678cc4889d3a42dd65
* | Merge "Add Phazr.IO libphazr backend to liberasurecode"Jenkins2017-03-011-5/+86
|\ \ | |/ |/|
| * Add Phazr.IO libphazr backend to liberasurecodeJim Cheung2017-02-281-5/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, there are several implementations of erasure codes that are available within OpenStack Swift. Most, if not all, of which are based on the Reed Solomon coding algorithm. Phazr.IO’s Erasure Coding technology uses a patented algorithm which are significantly more efficient and improves the speed of coding, decoding and reconstruction. In addition, Phazr.IO Erasure Code use a non-systematic algorithm which provides data protection at rest and in transport without the need to use encryption. Please contact support@phazr.io for more info on our technology. Change-Id: I4e40d02a8951e38409ad3c604c5dd6f050fa7ea0
* | Fix valgrind-check and memory leakKota Tsuyuzaki2017-02-201-22/+88
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Can you believe that we ware testing the memory leak with valgrind to just *bash scripts* instead of actual binaries on liberasurecode_test and libec_slap? That is why we cannot find such an easy memory leak[1] at the gate. Now this patch enable to run the valgrind against to the binaries. With this fix, we found various memory leak at liberasurecode_test as follows and this patch also fixes them: - If we create fake fragments, we're responsible for freeing all of the frags as well as the array holding the pointers to the frags. - If we allocate any space, we're responsible for freeing it. - If we create an EC descriptor, we're responsible for destroying it. - If we create a fragment or skip array, we're responsible for freeing it. - If that happens inside a loop, we're responsible for doing it *inside that same loop*. In addition to the test fix, this patch fixes following memory leaks at the code which is affected to other users (pyeclib, OpenStack Swift) * Refuse to decode fragments that aren't even long enough to include fragment headers. * Fix a small memory leak in the builtin rs_vand implementation. Closes-Bug: #1665242 Co-Authored-By: Tim Burke <tim@swiftstack.com> 1: https://review.openstack.org/#/c/431812 Change-Id: I96f124e4e536bbd7544208acc084de1cda5c19b2
* Merge "Add get_version functionality to liberasurecode lib"Jenkins2016-12-081-0/+9
|\
| * Add get_version functionality to liberasurecode libKota Tsuyuzaki2016-12-041-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently, we have liberasurecode version info in the header and pyeclib is using the info to detect the version. However it's a bit painful because it requires to rebuild pyeclib c code for you to see the actual installed version. This addition for liberasurecode_get_version enables caller to get the version integer from compiled shared library file (.so) and it will rescure to re-compiled operation from pyeclib. Change-Id: I8161ea7da3b069e83c93e11cb41ce12fa60c6f32
* | Merge "ISA-L Cauchy support"Jenkins2016-12-071-1/+68
|\ \
| * | ISA-L Cauchy supportKota Tsuyuzaki2016-12-061-1/+68
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is for supporting ISA-L cauchy based matrix. The difference from isa_l_rs_vand is only the matrix to use the encode/decode calculation. As a known issue, isa_l_rs_vand backend has constraint for the combinations of the available fragment to be able to decode/reconstuct. (See related change in detail) To avoid the constraint, this patch adds another isa-l backend to use cauchy matrix and keep the backward compatibility, this is in another isa_l_rs_cauchy namespace. For implementation consieration, the code is almost same except the matrix generation fucntion so that this patch makes isa_l_common.c file for gathering common fucntions like init/encode/decode/reconstruct. And then the common init funciton takes an extra args "gen_matrix_func_name" for entry point to load the fucntion by dlsym from isa-l .so file. Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Related-Change: Icee788a0931fe692fe0de31fabc4ba450e338a87 Change-Id: I6eb150d9d0c3febf233570fa7729f9f72df2e9be
* | Merge "Beautify fill_buffer()"Jenkins2016-11-101-3/+2
|\ \ | |/ |/|
| * Beautify fill_buffer()Pete Zaitcev2016-10-141-3/+2
| | | | | | | | | | | | | | | | This is completely pointless and speeds up the tests by something like 4% (less than a second), but my eyes BLEED when I see the existing code. Change-Id: Ia1d163e99bd2a2c2196b2669e4faa8707dbfc8d3
* | Fix error handling on gf_ivnert_matrix in isa-l backendKota Tsuyuzaki2016-11-091-0/+100
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current isa-l has possibility to return corrupted decoded data or corrupted reconstructed data on decode/reconstruct without error code. That is from the specification of isa-l rs vandermond matrix discussed at [1]. With many # of parities cases, we may hit the case above due to failing to get the inverse matrix from the encode matrix. The isa-l maintener gbtucker suggests a good way to detect the failing inverse matrix, that we should handle the return value gf_invert_matrix. If gf_invert_matrix returns not 0, we should stop to decode/reconstruct and return failure return code to the caller immediately. Otherwise, the caller regards the garbage data/fragment as correct one. And this patch adds the specific test case we can hit the issue (it happens not so general). 1: https://github.com/01org/isa-l/issues/10 Related-Change: I6eb150d9d0c3febf233570fa7729f9f72df2e9be Change-Id: Icee788a0931fe692fe0de31fabc4ba450e338a87
* | Fix liberasurecode skipping a bunch of invalid_args testsKota Tsuyuzaki2016-11-041-23/+36
|/ | | | | | | | | | | | | | | Since the commit a01b1818c874a65d1d1fb8f11ea441e9d3e18771, we have been to able to test some parameters for each tests. However, instead, the NULL which means the end of the test parameters causes skipping a bunch of unit tests which doesn't take args (e.g. test_create_backend_invalid_args). That is the worse because we have not tested anymore for the tests since the commit. This patch fixes to make it tested and more, fix a minor bug for a case that corrupted header incomming to get_metadata. Closes-Bug: #1634403 Change-Id: Ib99a8aa6032f02d0c7d1ab94b8da1ebfd9047d74
* Fragment metadata checksumming supportfrag_metadata_checksumTushar Gohad2016-03-101-4/+16
|
* Add a method to check if a backend is present.Timur Alperovich2016-03-081-0/+18
| | | | | | | Uses dlopen to check if a backend is present. This may be used by consumers who need to check which backends are present on a system. Issue #23
* Fix prototype declaration / definition mismatch (issue #25)Dirk Mueller2016-03-041-1/+1
|
* Check frag idx validity when verifying frag metadataTushar Gohad2016-02-141-2/+48
| | | | ... also add related test code
* Make libtool archive references relative to builddirTushar Gohad2015-12-101-6/+6
|
* Split helpers.h include for backward compatibilityTushar Gohad2015-09-222-0/+2
| | | | | | | Users of liberasurecode <= 1.0.7 used alloc/free helpers (which they shouldn't have). This change is to make sure we are still able to those older revs of programs and they work with newer liberasurecode.
* Enforce the (k + m < 32) limit, add unit testsKota Tsuyuzaki2015-08-181-2/+33
|
* Rename remaining INTERNAL_RS_VAND instancesTushar Gohad2015-07-201-17/+17
| | | | ... to LIBERASURECODE_RS_VAND
* Rename liberasurecode_rsvand -> liberasurecode_rs_vandliberasurecode_rs_vand-renameTushar Gohad2015-06-221-2/+2
|
* Rename rs_vand_internal to liberasurecode_rs_vandTushar Gohad2015-06-223-41/+43
|
* Fix open/write() and uninitialize var warnings in rs_vand internalTushar Gohad2015-06-221-3/+6
|
* Plugging new internal RS backend into liberasurecode.Kevin Greenan2015-06-181-1/+108
|
* Adding internal tests for the new internal RS codec.Kevin Greenan2015-06-183-1/+409
|
* Fix a uint < 0 warning reported by ClangTushar Gohad2015-04-301-3/+3
| | | | Also add -Wall for future builds
View Lorry Controller Status