| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* cipher/asm-common-amd64.h (SECTION_RODATA): New.
* cipher/blake2b-amd64-avx2.S: Use read-only section for constant
data.
* cipher/blake2b-amd64-avx512.S: Likewise.
* cipher/blake2s-amd64-avx.S: Likewise.
* cipher/blake2s-amd64-avx512.S: Likewise.
* cipher/poly1305-amd64-avx512.S: Likewise.
* cipher/sha1-avx-amd64.S: Likewise.
* cipher/sha1-avx-bmi2-amd64.S: Likewise.
* cipher/sha1-avx2-bmi2-amd64.S: Likewise.
* cipher/sha1-ssse3-amd64.S: Likewise.
* cipher/sha256-avx-amd64.S: Likewise.
* cipher/sha256-avx2-bmi2-amd64.S: Likewise.
* cipher/sha256-ssse3-amd64.S: Likewise.
* cipher/sha512-avx-amd64.S: Likewise.
* cipher/sha512-avx2-bmi2-amd64.S: Likewise.
* cipher/sha512-avx512-amd64.S: Likewise.
* cipher/sha512-ssse3-amd64.S: Likewise.
* cipher/sha3-avx-bmi2-amd64.S: Likewise.
--
Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* cipher/asm-common-amd64.h (ret_spec_stop): New.
* cipher/arcfour-amd64.S: Use 'ret_spec_stop' for 'ret' instruction.
* cipher/blake2b-amd64-avx2.S: Likewise.
* cipher/blake2s-amd64-avx.S: Likewise.
* cipher/blowfish-amd64.S: Likewise.
* cipher/camellia-aesni-avx-amd64.S: Likewise.
* cipher/camellia-aesni-avx2-amd64.h: Likewise.
* cipher/cast5-amd64.S: Likewise.
* cipher/chacha20-amd64-avx2.S: Likewise.
* cipher/chacha20-amd64-ssse3.S: Likewise.
* cipher/des-amd64.S: Likewise.
* cipher/rijndael-aarch64.S: Likewise.
* cipher/rijndael-amd64.S: Likewise.
* cipher/rijndael-ssse3-amd64-asm.S: Likewise.
* cipher/rijndael-vaes-avx2-amd64.S: Likewise.
* cipher/salsa20-amd64.S: Likewise.
* cipher/serpent-avx2-amd64.S: Likewise.
* cipher/serpent-sse2-amd64.S: Likewise.
* cipher/sha1-avx-amd64.S: Likewise.
* cipher/sha1-avx-bmi2-amd64.S: Likewise.
* cipher/sha1-avx2-bmi2-amd64.S: Likewise.
* cipher/sha1-ssse3-amd64.S: Likewise.
* cipher/sha256-avx-amd64.S: Likewise.
* cipher/sha256-avx2-bmi2-amd64.S: Likewise.
* cipher/sha256-ssse3-amd64.S: Likewise.
* cipher/sha512-avx-amd64.S: Likewise.
* cipher/sha512-avx2-bmi2-amd64.S: Likewise.
* cipher/sha512-ssse3-amd64.S: Likewise.
* cipher/sm3-avx-bmi2-amd64.S: Likewise.
* cipher/sm4-aesni-avx-amd64.S: Likewise.
* cipher/sm4-aesni-avx2-amd64.S: Likewise.
* cipher/twofish-amd64.S: Likewise.
* cipher/twofish-avx2-amd64.S: Likewise.
* cipher/whirlpool-sse2-amd64.S: Likewise.
* mpi/amd64/func_abi.h (CFI_*): Remove, include from "asm-common-amd64.h"
instead.
(FUNC_EXIT): Use 'ret_spec_stop' for 'ret' instruction.
* mpi/asm-common-amd64.h: New.
* mpi/i386/mpih-add1.S: Use 'ret_spec_stop' for 'ret' instruction.
* mpi/i386/mpih-lshift.S: Likewise.
* mpi/i386/mpih-mul1.S: Likewise.
* mpi/i386/mpih-mul2.S: Likewise.
* mpi/i386/mpih-mul3.S: Likewise.
* mpi/i386/mpih-rshift.S: Likewise.
* mpi/i386/mpih-sub1.S: Likewise.
* mpi/i386/syntax.h (ret_spec_stop): New.
--
Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* cipher/sm3-avx-bmi2-amd64.S: Change K0-K63 macros to signed decimal.
--
There are a lot of the following errors compiling with GNU assembler
version 2.27-41:
sm3-avx-bmi2-amd64.S: Assembler messages:
sm3-avx-bmi2-amd64.S:402: Error: 0xf3988a32 out range of signed
32bit displacement
The newer GNU assembler does not have this issue. It can be fixed by
changing K0-K63 macros from hex-format to signed decimal values.
Also remove unused macro 'addl3'.
Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
|
|
|
* cipher/Makefile.am: Add 'sm3-avx-bmi2-amd64.S'.
* cipher/sm3-avx-bmi2-amd64.S: New.
* cipher/sm3.c (USE_AVX_BMI2, ASM_FUNC_ABI, ASM_EXTRA_STACK): New.
(SM3_CONTEXT): Define 'h' as array instead of separate fields 'h1',
'h2', etc.
[USE_AVX_BMI2] (_gcry_sm3_transform_amd64_avx_bmi2)
(do_sm3_transform_amd64_avx_bmi2): New.
(sm3_init): Select AVX/BMI2 transform function if support by HW; Update
to use 'hd->h' as array.
(transform_blk, sm3_final): Update to use 'hd->h' as array.
* configure.ac: Add 'sm3-avx-bmi2-amd64.lo'.
--
Benchmark on AMD Zen3:
Before:
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 2.18 ns/B 436.6 MiB/s 10.59 c/B 4850
After (~43% faster):
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 1.52 ns/B 627.4 MiB/s 7.37 c/B 4850
Benchmark on Intel Skylake:
Before:
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 4.35 ns/B 219.2 MiB/s 13.48 c/B 3098
After (~34% faster):
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 3.24 ns/B 294.4 MiB/s 10.04 c/B 3098
Benchmark on AMD Zen2:
Before:
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 2.73 ns/B 348.9 MiB/s 11.86 c/B 4339
After (~38% faster):
| nanosecs/byte mebibytes/sec cycles/byte auto Mhz
SM3 | 1.97 ns/B 483.0 MiB/s 8.52 c/B 4318
Reviewed-and-tested-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
|
