summaryrefslogtreecommitdiff
path: root/hv_func.h
Commit message (Collapse)AuthorAgeFilesLines
* Revert "use a specific define for 64 bit hashing"Yves Orton2017-04-231-12/+8
| | | | | | This reverts commit a4283faf7092ec370914ee3e4e7afeddd0115689. Accidental push. Sorry.
* Revert "Add new hashing and "hash with state" infrastructure"Yves Orton2017-04-231-221/+269
| | | | | | This reverts commit a3bf60fbb1f05cd2c69d4ff0a2ef99537afdaba7. Accidentally pushed work pending unfreeze.
* Revert "Move utility macros to their own file"Yves Orton2017-04-231-1/+82
| | | | | | This reverts commit 259e968485f855f70472c8be9267efceca42b0fb. Accidentally pushed work pending unfreeze.
* Revert "get rid of USE_HASH_SEED_EXPLICIT"Yves Orton2017-04-231-1/+4
| | | | | | This reverts commit dd1b95f812312c85390f487887cdd55282fcd6ce. Accidentally pushed work pending unfreeze.
* get rid of USE_HASH_SEED_EXPLICITYves Orton2017-04-231-4/+1
| | | | | | I think this has been outright broken for a long time, and it doesnt make any sense to since mandatory randomization anyway, so simply remove it.
* Move utility macros to their own fileYves Orton2017-04-231-82/+1
| | | | | | so that hv_func is left with only logic relating to selecting and configuring the hash function we use, not the macros our hash functions use.
* Add new hashing and "hash with state" infrastructureYves Orton2017-04-231-269/+221
| | | | | | | | | | | | | | | | | | | | | This adds support for three new hash functions: StadtX, Zaphod32 and SBOX, and reworks some of our hash internals infrastructure to do so. SBOX is special in that it is designed to be used in conjuction with any other hash function for hashing short strings very efficiently and very securely. It features compile time options on how much memory and startup time are traded off to control the length of keys that SBOX hashes. This also adds support for caching the hash values of single byte characters which can be used in conjuction with any other hash, including SBOX, although SBOX itself is as fast as the lookup cache, so typically you wouldnt use both at the same time. This also *removes* support for Jenkins One-At-A-Time. It has served us well, but it's day is done. This patch adds three new files: zaphod32_hash.h, stadtx_hash.h, sbox32_hash.h
* use a specific define for 64 bit hashingYves Orton2017-04-231-8/+12
|
* use IVSIZE to see if we should use 64bit hashingYves Orton2016-12-091-1/+1
|
* cleanup hv_func.h - remove non-recommended hash variantsYves Orton2016-12-071-443/+16
| | | | | | We dont need to offer so many options in terms of hash function. Many, if not all, of the removed functions are deficient in one way or another.
* use a hybrid hash function, OAATH for short keys, Siphash 1-3 for longer onesYves Orton2016-12-071-56/+184
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Switch to an optimized/unrolled variant of OAATH for keys 16 bytes and less, and use Siphash 1-3 for longer keys. (On 64 bit builds, 32 bit is untouched.) I've done a bunch of benchmarking with Porting/bench.pl to see at what point the 8 byte reads of Siphash 1-3 start to win over the low overhead costs of OAATH. It seems to be around 16 bytes. At the same time, we unroll OAATH so that we can save some instructions per character. The net result is all key lengths get faster as measured by Porting/bench.pl, and hashing longer keys becomes *much* faster. Interestingly the actual crossover point of OAATH being slower than Siphash 1-3 is much higher than might be guessed from bulk testing either in a raw benchmark. For instance, basic benchmarks of the hash function alone predicts that Siphash 1-3 should "win" at around 4 bytes. However, in practice it is four times longer. I believe this is because setting up Siphash blows away a fair amount of CPU state compared to OAATH, which is more important when the hash is going to be used to manage a data structure. So it requires a correspondingly longer string before the larger sized read starts to win. Summarized stats (higher is better): AVERAGE blead yves ------ ------ Ir 100.00 104.47 Dr 100.00 103.97 Dw 100.00 107.63 COND 100.00 112.16 IND 100.00 75.07 COND_m 100.00 125.75 IND_m 100.00 97.42 Ir_m1 100.00 137.11 Dr_m1 100.00 100.10 Dw_m1 100.00 99.62 Ir_mm 100.00 100.00 Dr_mm 100.00 100.00 Dw_mm 100.00 100.00
* Fix const correctness in hv_func.hPetr Písař2016-12-011-11/+11
| | | | | | | | | | | | | | | | Building an XS code with -Wcast-qual yielded warnings about discarding const qualifiers from pointer targets like: $ printf '#include "EXTERN.h"\n#include "perl.h"\n' | gcc -Wcast-qual -I/usr/lib64/perl5/CORE -c -x c - In file included from /usr/lib64/perl5/CORE/hv.h:629:0, from /usr/lib64/perl5/CORE/perl.h:3740, from <stdin>:2: /usr/lib64/perl5/CORE/hv_func.h: In function ‘S_perl_hash_siphash_2_4’: /usr/lib64/perl5/CORE/hv_func.h:213:17: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual] U64TYPE k0 = ((U64TYPE*)seed)[0]; ^ Signed-off-by: Petr Písař <ppisar@redhat.com>
* [perl #126242] make the constant hash seeds unsigned char *Tony Cook2015-10-141-3/+3
| | | | | ...to prevent warnings on most C compilers and build failures on C++ compilers for -DNO_HASH_SEED builds.
* fix NO_HASH_SEED buildDaniel Dragan2015-10-141-0/+2
| | | | | | | | | | | commit b1300a738f added PERL_HASH_FUNC_ONE_AT_A_TIME_HARD algo, which was the first one to introduce 8 byte seeds, previously all the algos used 4 or 16 byte seeds. No case was added to the CPP tree for 8 byte const seeds, so add one now. Otherwise the #error at the end of the tree runs and breaks the build. NO_HASH_SEED define was public API in the past and could be considered to still be public API, see commit f36626324a. My use for NO_HASH_SEED is reducing entropy for tracking down memory corruption.
* Use U64 (a type) instead of U64TYPE (a define).Jarkko Hietaniemi2015-07-291-40/+40
| | | | Builds on top of 16d89be8.
* Follow-up to d229247c: use U64TYPE consistently.Jarkko Hietaniemi2015-07-251-1/+1
| | | | | | | Why exactly we are using U64TYPE here and not U64, I dunno. What I do know is that if U64TYPE is changed to U64 everywhere in hv_func.h, win32 build dies. Maybe hv_func.h gets included before the typedef for U64 in handy.h takes place, and only the U64TYPE define is available.
* Comment detypos.Jarkko Hietaniemi2015-07-251-2/+2
|
* Use UINT64_C.Jarkko Hietaniemi2015-07-251-1/+1
|
* There's no READ_UINT32.Jarkko Hietaniemi2015-07-251-3/+1
| | | | (relic of some earlier code?)
* Replace common Emacs file-local variables with dir-localsDagfinn Ilmari Mannsåker2015-03-221-6/+0
| | | | | | | | | | | | | | | | An empty cpan/.dir-locals.el stops Emacs using the core defaults for code imported from CPAN. Committer's work: To keep t/porting/cmp_version.t and t/porting/utils.t happy, $VERSION needed to be incremented in many files, including throughout dist/PathTools. perldelta entry for module updates. Add two Emacs control files to MANIFEST; re-sort MANIFEST. For: RT #124119.
* Revert "dont compile unused static hash functions"Yves Orton2014-12-251-16/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 09c759bcfa4c2880c571df4da20458b2f781debf. The reverted patch causes us to not compile functions that we might not use. In theory this is a good thing. But any competent compiler is going to exclude them anyway if they aren't used, and we might miss useful warning messages, or whatnot. See b404539126a for an example of a patch that would not have happened, or would only have been partially done had this patch been applied. Also the reverted patch claimed that "It is not safe to have multiple hash funcs in 1 build due to conflicts on the size of PERL_HASH_SEED_BYTES." which is incorrect. The functions do not reference PERL_HASH_SEED_BYTES directly, and are usable by anyone who knows what size of seed they need. Additionally I have on my long-term todo list the following: * Allow Perl to randomly select the hash function at startup * Allow Perl to use the hash function determined by the ENV at startup. This patch would have to be reverted to complete either of those tasks. To recap, I am reverting this patch because it adds no real value, makes our hash functions susceptible to bit-rot, and because it blocks interesting future changes that I plan to work on in the future.
* dont compile unused static hash functionsDaniel Dragan2014-12-231-4/+16
| | | | | | | | | Perl uses only 1 hash func perl build. It is not safe to have multiple hash funcs in 1 build due to conflicts on the size of PERL_HASH_SEED_BYTES. No point in wasting CC's time to compile a static func which will always be tossed by the CC at the end due to no references to the static has func, or by the linker due to no references to the funcs. Related to [perl #123483] .
* silence VC Win64 perl warnings in hv_func.hDaniel Dragan2014-12-231-8/+8
| | | | | | | | | These warnings are in a public header, hv_func.h, so CPAN XS builders will see 8 warnings with every .c file from a CPAN XS module that they will compile. Silence the warnings since headers should never warn unconditionally. See [perl #123483] for details.
* Add MurmurHash64A and MurmurHash64B to hv_func.hYves Orton2014-08-061-0/+138
| | | | | | | | | | | | | Both of these hash functions are by Austin Appleby and are in the public domain. The 64A variant is designed for 64 bit machines. The 64B variant is designed for 32 bit machines. Both use unaligned loads, so are unsuitable for platforms with strict alignment requirements. Both have been converted to use Perls hash function calling conventions, and to return a 32 bit hash instead of a 64 bit hash (low 32 bits)
* Make hash_value() accept an optional seedYves Orton2014-06-161-9/+11
|
* hv_func.h - fix seed initialization in sdbm and djb2 hashing algorithms.Yves Orton2014-06-021-2/+2
| | | | | | | | | | | | | | | | In a previous commit I added code to "mix in" the length of the string into the seed used by these functions, to avoid issues with zero seeds, and with the hope that it makes it harder to create multicollision attacks against these hash functions. Unfortunately when I restructured the seed logic for the inline functions in hv_func.h I messed it up, and these hash functions were broken. I never noticed because they are both such bad hash functions for our needs that I never built with them, and we have no infrastructure to make it easy to test building with non-standard hash functions so it never got automatically tested. Hopefully at some point someone will find a round-tuit and teach Configure about selecting alternate hash functions.
* Use UINT64_C for UV constants.Jarkko Hietaniemi2014-05-281-4/+4
|
* Remove a __BORLANDC__ stowawaySteve Hay2013-09-131-1/+1
| | | | Support for __BORLANDC__ was axed (actually, chainsawed) in 378eeda70c.
* comments only, explain the different ONE_AT_A_TIME algosDaniel Dragan2013-07-291-0/+10
| | | | | | | | Seeing 3 hash algos with the same name is confusing. If they are the same name ("one at a time"), then why are there 3 different ones? What are the differences (without reading the code)? What are the pros and cons of each? INSTALL lists the ONE_AT_A_TIME algos as existing, but doesn't explain why there are 3 of them. Patch with jkeenan suggestions.
* document and improve hash algorithm randomization related build optionsYves Orton2013-05-081-5/+1
| | | | | | Install was a copy of other material, made heavy reference to 5.8.x and and didnt really document what it should have. I reworked it to be more up to date.
* Use PTR2IV() instead of casting a pointer to IV, to avoid warnings.Robin Barker2013-05-061-2/+2
| | | | | | | | | | | The cast to IV was added to avoid problems on platforms where pointers are larger than longs. However, the change instead generates warnings on platforms where IVs are larger than pointers. Instead, use the PTR2IV() macro provided by perl.h to make everyone happy. Also change the type of the variable on that line, because it is cast to STRLEN the first time it is used, and then passed to a macro that assigns it to an int regardless.
* perl-5.17.11 fails to build with x64 mingw64 (gcc-4.7.0)Sisyphus2013-04-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # New Ticket Created by "Sisyphus" # Please include the string: [perl #117687] # in the subject line of all future correspondence about this issue. # <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=117687 > This is a bug report for perl from sisyphus@OWNER-PC311012, generated with the help of perlbug 1.39 running under perl 5.17.11. ----------------------------------------------------------------- [Please describe your issue here] This is essentially a repeat of a post sent to p5p on March 24 2013 titled "[Win32] perl-5.17.10 fails to build with x64 mingw64 (gcc-4.7.0)" The issue still exists for 5.17.11 (for me). The error I get is: In file included from ../hv.h:570:0, from ../perl.h:3480, from perllib.c:10: ../hv_func.h: In function 'U32 S_perl_hash_murmur3(const unsigned char*, const u nsigned char*, STRLEN)': ../hv_func.h:395:20: error: cast from 'const unsigned char*' to 'long int' loses precision [-fpermissive] Signed-off-by: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
* rework ROTL definitions (and add ROTL_UV)Yves Orton2013-03-191-10/+21
|
* default to PERL_FUNC_ONE_AT_A_TIME_HARD for all buildsYves Orton2013-03-191-0/+1
| | | | For testing, but maybe for ever
* silence signed mistmatch in comparison warning in MurmurhashYves Orton2013-03-191-1/+1
| | | | as far as I can tell 'i' can only be positive here.
* add a hardened one-at-a-time hash variantYves Orton2013-03-191-3/+46
| | | | Mix in additional randomness into the final value.
* Split out hash functions into new file and turn into inline static functionsYves Orton2013-03-191-0/+509
This includes various tweaks related to building SipHash and other cleanup.
View Lorry Controller Status