diff options
author | Ilya Leoshkevich <iii@linux.ibm.com> | 2019-04-02 08:01:02 -0700 |
---|---|---|
committer | Paul Eggert <eggert@cs.ucla.edu> | 2019-04-02 10:30:11 -0700 |
commit | 7a6f9c9c3267185a299ad178607ac5e3716ab4a5 (patch) | |
tree | 3412bffea29a39e29f04766174bcb756e09a24e6 /util.c | |
parent | 38ae6a4ed36a7c86609a6a595add4298d9c202bc (diff) | |
download | gzip-7a6f9c9c3267185a299ad178607ac5e3716ab4a5.tar.gz |
bug#34918: [PATCH] Add support for IBM Z hardware-accelerated deflate
Future versions of IBM Z mainframes will provide DFLTCC instruction,
which implements deflate algorithm in hardware with estimated
compression and decompression performance orders of magnitude faster
than the current gzip and ratio comparable with that of level 1.
This patch adds DFLTCC support to gzip. In order to enable it, the
following build commands should be used:
$ ./configure --enable-dfltcc
$ make
When built like this, gzip would compress in hardware on level 1, and in
software on all other levels. Decompression will always happen in
hardware. In order to enable DFLTCC compression for levels 1-6 (i.e. to
make it used by default) one could either add -DDFLTCC_LEVEL_MASK=0x7e
at compile time, or set the environment variable DFLTCC_LEVEL_MASK to
0x7e at run time.
Two DFLTCC compression calls produce the same results only when they
both are made on machines of the same generation, and when the
respective buffers have the same offset relative to the start of the
page. Therefore care should be taken when using hardware compression
when reproducible results are desired. One such use case - reproducible
software builds - is handled explicitly: when SOURCE_DATE_EPOCH
environment variable is set, the hardware compression is disabled.
This patch tries to add DFLTCC support in a least intrusive way.
All SystemZ-specific code was placed into a separate file, but
unfortunately there is still a noticeable amount of changes in the
main gzip code. Below is the summary of those changes.
DFLTCC will refuse to write an End-of-block Symbol if there is no input
data, thus in some cases it is necessary to do this manually. In order
to achieve this, bi_buf and bi_valid were promoted to extern variables.
lm_init() function moves the input buffer into the window, which is not
desirable for DFLTCC. Therefore, its invocation was moved to
software-only deflate(). In addition to initializing the window, this
function also used to convert compression level to flags, which is still
needed for DFLTCC. This responsibility was handed off to zip() function.
To achieve maximum performance with DFLTCC, inbuf and outbuf must be
256k big and page-aligned. Additionally, for DFLTCC to work at all, the
window must be page-aligned.
In addition to compression, DFLTCC computes CRC-32 checksum, therefore,
whenever it's used, software checksumming needs to be suppressed and its
results replaced by those of dfltcc. This is achieved by introducing the
new getcrc() and setcrc() functions.
Unlike the current software implementation, DFLTCC decompresses data
into the output buffer, and not the window. Therefore, just like
flushing the window, flushing the output buffer must honor the test
flag.
Finally, znew-k test assumes that "znew -K" would not convert the test
.Z file to .gz, which is not the case with DFLTCC. Since this is not the
main point of the test, this assumption was relaxed.
Diffstat (limited to 'util.c')
-rw-r--r-- | util.c | 30 |
1 files changed, 26 insertions, 4 deletions
@@ -96,6 +96,11 @@ static const ulg crc_32_tab[] = { 0x2d02ef8dL }; +/* ======================================================================== + * Shift register contents + */ +static ulg crc = (ulg)0xffffffffL; + /* =========================================================================== * Copy input to output unchanged: zcat == cat with --force. * IN assertion: insize bytes have already been read in inbuf and inptr bytes @@ -126,13 +131,11 @@ int copy(in, out) * Return the current crc in either case. */ ulg updcrc(s, n) - uch *s; /* pointer to bytes to pump through */ + const uch *s; /* pointer to bytes to pump through */ unsigned n; /* number of bytes in s[] */ { register ulg c; /* temporary variable */ - static ulg crc = (ulg)0xffffffffL; /* shift register contents */ - if (s == NULL) { c = 0xffffffffL; } else { @@ -146,6 +149,23 @@ ulg updcrc(s, n) } /* =========================================================================== + * Return a current CRC value. + */ +ulg getcrc() +{ + return crc ^ 0xffffffffL; +} + +/* =========================================================================== + * Set a new CRC value. + */ +void setcrc(c) + ulg c; +{ + crc = c ^ 0xffffffffL; +} + +/* =========================================================================== * Clear input and output buffers */ void clear_bufs() @@ -238,7 +258,9 @@ void flush_outbuf() { if (outcnt == 0) return; - write_buf(ofd, (char *)outbuf, outcnt); + if (!test) { + write_buf(ofd, (char *)outbuf, outcnt); + } bytes_out += (off_t)outcnt; outcnt = 0; } |