md: Fix hashing for data >= 256 GB

* cipher/hash-common.h (gcry_md_block_ctx): Add "nblocks_high".
* cipher/hash-common.c (_gcry_md_block_write): Bump NBLOCKS_HIGH.
* cipher/md4.c (md4_init, md4_final): Take care of NBLOCKS_HIGH.
* cipher/md5.c (md5_init, md5_final): Ditto.
* cipher/rmd160.c (_gcry_rmd160_init, rmd160_final): Ditto.
* cipher/sha1.c (sha1_init, sha1_final): Ditto.
* cipher/sha256.c (sha256_init, sha224_init, sha256_final): Ditto.
* cipher/sha512.c (sha512_init, sha384_init, sha512_final): Ditto.
* cipher/tiger.c (do_init, tiger_final): Ditto.
* cipher/whirlpool.c (whirlpool_final): Ditto.

* cipher/md.c (gcry_md_algo_info): Add GCRYCTL_SELFTEST.
(_gcry_md_selftest): Return "not implemented" as required.
* tests/hashtest.c: New.
* tests/genhashdata.c: New.
* tests/Makefile.am (TESTS): Add hashtest.
(noinst_PROGRAMS): Add genhashdata
--

Problem found by Denis Corbin and analyzed by Yuriy Kaminskiy.

sha512 and whirlpool should not have this problem because they use 64
bit types for counting the blocks. However, a similar fix has been
employed to allow for really huge sizes - despite that it will be very
hard to test them.

The test vectors have been produced by sha{1,224,256}sum and the
genhashdata tool.  A sequence of 'a' is used for them because a test
using one million 'a' is commonly used for test vectors.  More test
vectors are required.  Running the large tests needs to be done
manual for now:

  ./hashtest --gigs 256

tests all algorithms,

  ./hashtest --gigs 256 sha1 sha224 sha256

only the given ones.  A configure option to include these test in the
standard regression suite will be useful.  The tests will take looong.

Signed-off-by: Werner Koch <wk@gnupg.org>

1 files changed, 9 insertions, 2 deletions

diff --git a/cipher/sha512.c b/cipher/sha512.c
index 97fb2036..14608dcf 100644
--- a/cipher/sha512.c
+++ b/cipher/sha512.c
@@ -97,6 +97,7 @@ sha512_init (void *context)
   hd->h7 = U64_C(0x5be0cd19137e2179);
 
   ctx->bctx.nblocks = 0;
+  ctx->bctx.nblocks_high = 0;
   ctx->bctx.count = 0;
   ctx->bctx.blocksize = 128;
   ctx->bctx.bwrite = transform;
@@ -122,6 +123,7 @@ sha384_init (void *context)
   hd->h7 = U64_C(0x47b5481dbefa4fa4);
 
   ctx->bctx.nblocks = 0;
+  ctx->bctx.nblocks_high = 0;
   ctx->bctx.count = 0;
   ctx->bctx.blocksize = 128;
   ctx->bctx.bwrite = transform;
@@ -515,15 +517,20 @@ sha512_final (void *context)
 {
   SHA512_CONTEXT *hd = context;
   unsigned int stack_burn_depth;
-  u64 t, msb, lsb;
+  u64 t, th, msb, lsb;
   byte *p;
 
   _gcry_md_block_write (context, NULL, 0); /* flush */ ;
 
   t = hd->bctx.nblocks;
+  /* if (sizeof t == sizeof hd->bctx.nblocks) */
+  th = hd->bctx.nblocks_high;
+  /* else */
+  /*   th = hd->bctx.nblocks >> 64; In case we ever use u128  */
+
   /* multiply by 128 to make a byte count */
   lsb = t << 7;
-  msb = t >> 57;
+  msb = (th << 7) | (t >> 57);
   /* add the count */
   t = lsb;
   if ((lsb += hd->bctx.count) < t)