| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
| |
Uses dlopen to check if a backend is present. This may be used by
consumers who need to check which backends are present on a system.
Issue #23
|
|
|
|
|
|
|
| |
There are systems, for example Hurd, which doesn't define this constant
because there are no such limit. See [1] link for explanation.
[1] http://www.gnu.org/software/hurd/community/gsoc/project_ideas/maxpath.html
|
| |
|
|
|
|
|
|
|
| |
Users of liberasurecode <= 1.0.7 used alloc/free helpers
(which they shouldn't have). This change is to make sure
we are still able to those older revs of programs and they
work with newer liberasurecode.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
... to LIBERASURECODE_RS_VAND
|
| |
|
| |
|
|
|
|
| |
This is meant to be used in cases where ISA-L and Jerasure cannot be used.
|
| |
|
| |
|
|
|
|
| |
https://bitbucket.org/tsg-/liberasurecode/issue/12/make-valgrind-test-fails
|
|
|
|
|
|
| |
Also added additional test to test_xor_code to do an exhaustive decode test
(all possible 1 and 2 disk failures) and changed teh default liberasurecode
test to test (3, 3, 3).
|
| |
|
| |
|
|
|
|
| |
This patch renames the "metadata_adder" variable to "backend_metadata_size"
|
|
|
|
|
|
|
|
| |
This patch renames following variables and functions:
- frag_adder_size -> frag_backend_metadata_size
- set_fragment_adder_size() -> set_fragment_backend_metadata_size()
- get_fragment_adder_size() -> get_fragment_backend_metadata_size()
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For get_segment_info function of PyECLib, liberasurecode should
support get_fragment_size function because if pyeclib and liberasurecode
have the calculation of fragment size each other, it might cause
the size mismatch (i.e. it'll be a bug) in the future development work.
This patch introduces liberasurecode_get_fragment_size function to return
the fragment_size calculated at liberasurecode accoring to specified
backend descriptor.
It really usefull to help caller knows how large size it have to expect
and all pyeclib has to do for retrieving fragment_size will be just calling
the liberasurecode_get_fragment_size function on get_segment_info.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch allows to get correct fragment size includes metadata_adder.
Current implementaion automatically allocates extra bytes for the metadata_adder
in alloc_buffer, and then, no information about the extra bytes will be returned
to the api caller side. It's too confusable because callers couldn't know how size they
assumes as the fragment size.
To be easy to find out the size infomation, this patch adds "frag_adder_size"
variable into fragment metadata and also some functions to get fragment size.
The definitions of these size infomation are here,
fragment_meta:
- size-> raw data size used to encode/fragment_to_string
- frag_adder_size-> metadata_adder of backend specification
And the definitions of functions are here,
- get_fragment_size:
-> return sizeof(fragument_header) + size + frag_adder_size
- get_fragment_buffer_size:
-> return size + frag_adder_size
- get_fragment_payload_size:
-> return size
By using these function above, users could get the size information
directly from fragments. It results in enabling to return fragment_len
to the caller side easily.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On the first consideration[1], metadata_adder is defined as a extra byte
size for "each" fragment. However, current implementation is an element
affects to data_len. (i.e. aligned_data_size for original segment data)
We should make metadata_adder to be a fixed value against to each fragment,
otherwise the extra bytes for the fragments will have flexible length depends
on "K". It will be quite complex for backend implementation to know "How large
bytes the raw data size is", "How large bytes the backend could use as extra
bytes for each fragment".
1: https://bitbucket.org/tsg-/liberasurecode/commits/032b57d9b1c7aadc547fccbacf88af786c9067e7?at=master
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch achieves a couple of things as follows:
- Undoing the liberasurecode_encode_cleanup specification to
expect "fragment" pointers as its arguments.
- Ensuring liberasurecode_encode to pass "fratment" pointers to
liberasurecode_encode_cleanup.
liberasurecode_encode_cleanup is used also in pyeclib so that
it is expected that the argument pointers (i.e. encoded_data and
encoded_parity) should be the collection of the heads of "fragment"
pointers.
However, when the backend encode fails, liberasurecode keeps "data"
pointers behind of fragment_header, and then, goes to "out:" statement
to cleanup its memories. It causes invalid pointer failure.
This patch adds a translation function from "data" pointers to "fragment"
pointers and ensure liberasurecode_encode to pass correct pointers to
libersurecode_encode_cleanup.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a new plug-able backend called "shss" made by
Nippon Telegraph and Telephone corporation (NTT).
Note that this produces a just plug-in to shss erasure coding binary
so that users have to install a shss binary (i.e. libshss.so) aside from
liberasurecode when using shss.
Please contact us if you are insterested in the NTT backend (welcome!):
Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Co-Author: Ryuta Kon <kon.ryuta@po.ntts.co.jp>
|
|
|
|
| |
Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
|
|
|
|
| |
Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
|
|
|
|
|
| |
tests to a 'fragment verify' test, since it is using is_valid_fragment and
and validating fragment headers, not stripe-level.
|
| |
|
| |
|
|
|
|
| |
with using cmake to build the project and cmake does not generate the config_liberasurecode.h file.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
|
|
|
|
| |
little clean-up, but the tests are passing...
|
|
|
|
| |
elements of the ec_backend_id_t enum.
|
|
|
|
| |
not being placed into the fragment header).
|
|
|
|
| |
existing warnings.
|
|
|
|
|
|
|
|
| |
liberasure_get_fragment_metadata and
liberasurecode_verify_stripe_metadata. In the process removed the
liberasure descriptor from the liberasure_get_fragment_metadata
function's signature as it was not being used in the function's
implementation.
|
|
|
|
| |
Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
|
| |
|
|
|
|
|
|
|
| |
Still needed:
1.) RS and XOR backends need to honor the 'exclude' argument.
2.) Need to add tests.
|
|
|
|
|
|
|
|
|
| |
follows a common format and that it accurately reflects the code.
- Cleaned up liberasurecode_instance_destroy to ensure that the return
code is correct and that it is not trying to free data that has
not been removed from the open instances.
- Updated the tests to assert that the liberasurecode_instance_destroy
method is return correctly.
|