delta/postgresql.git/src/tools/pginclude/cpluspluscheck, branch master git.postgresql.org: git/postgresql.git pg_dump: Add support for zstd compression 2023-04-05T19:39:33+00:00 Tomas Vondra tomas.vondra@postgresql.org 2023-04-05T19:38:04+00:00 84adc8e20f54e93a003cd316fa1eb9b03e393288 Allow pg_dump to use the zstd compression, in addition to gzip/lz4. Bulk of the new compression method is implemented in compress_zstd.{c,h}, covering the pg_dump compression APIs. The rest of the patch adds test and makes various places aware of the new compression method. The zstd library (which this patch relies on) supports multithreaded compression since version 1.5. We however disallow that feature for now, as it might interfere with parallel backups on platforms that rely on threads (e.g. Windows). This can be improved / relaxed in the future. This also fixes a minor issue in InitDiscoverCompressFileHandle(), which was not updated to check if the file already has the .lz4 extension. Adding zstd compression was originally proposed in 2020 (see the second thread), but then was reworked to use the new compression API introduced in e9960732a9. I've considered both threads when compiling the list of reviewers. Author: Justin Pryzby Reviewed-by: Tomas Vondra, Jacob Champion, Andreas Karlsson Discussion: https://postgr.es/m/20230224191840.GD1653@telsasoft.com Discussion: https://postgr.es/m/20201221194924.GI30237@telsasoft.com 
Allow pg_dump to use the zstd compression, in addition to gzip/lz4. Bulk
of the new compression method is implemented in compress_zstd.{c,h},
covering the pg_dump compression APIs. The rest of the patch adds test
and makes various places aware of the new compression method.

The zstd library (which this patch relies on) supports multithreaded
compression since version 1.5. We however disallow that feature for now,
as it might interfere with parallel backups on platforms that rely on
threads (e.g. Windows). This can be improved / relaxed in the future.

This also fixes a minor issue in InitDiscoverCompressFileHandle(), which
was not updated to check if the file already has the .lz4 extension.

Adding zstd compression was originally proposed in 2020 (see the second
thread), but then was reworked to use the new compression API introduced
in e9960732a9. I've considered both threads when compiling the list of
reviewers.

Author: Justin Pryzby
Reviewed-by: Tomas Vondra, Jacob Champion, Andreas Karlsson
Discussion: https://postgr.es/m/20230224191840.GD1653@telsasoft.com
Discussion: https://postgr.es/m/20201221194924.GI30237@telsasoft.com
Tighten header pre-inclusions in headerscheck and cpluspluscheck. 2023-03-04T17:11:50+00:00 Tom Lane tgl@sss.pgh.pa.us 2023-03-04T17:11:50+00:00 f62975b2af6ace276a1d564a070b0aef479025af We allow our header files to depend on the appropriate one of postgres.h, postgres_fe.h, or c.h having already been included. However, there are a few headers such as libpq-fe.h that are meant to be used by client applications and therefore must compile without any assumptions about previous inclusions. These test scripts failed to consider that, which seems quite hazardous since we might not immediately notice such a problem otherwise. Hence, adjust these scripts to test relevant libpq and ecpg headers with no prior inclusion. While at it, we can also make an effort to actually use the relevant one of postgres.h, postgres_fe.h, or c.h. I added some rules that guess which one to use based on the first-level src subdirectory, e.g. use postgres_fe.h under src/bin/. These rules are hardly water-tight but they seem to work today, and we can always refine them in the future. These changes don't reveal any live problems today, which is good, but they should make these scripts more able to catch future bugs. Discussion: https://postgr.es/m/2488193.1677863247@sss.pgh.pa.us 
We allow our header files to depend on the appropriate one of
postgres.h, postgres_fe.h, or c.h having already been included.
However, there are a few headers such as libpq-fe.h that are
meant to be used by client applications and therefore must
compile without any assumptions about previous inclusions.
These test scripts failed to consider that, which seems quite
hazardous since we might not immediately notice such a problem
otherwise.  Hence, adjust these scripts to test relevant libpq
and ecpg headers with no prior inclusion.

While at it, we can also make an effort to actually use the
relevant one of postgres.h, postgres_fe.h, or c.h.  I added
some rules that guess which one to use based on the first-level
src subdirectory, e.g. use postgres_fe.h under src/bin/.
These rules are hardly water-tight but they seem to work today,
and we can always refine them in the future.

These changes don't reveal any live problems today, which is good,
but they should make these scripts more able to catch future bugs.

Discussion: https://postgr.es/m/2488193.1677863247@sss.pgh.pa.us
Add LZ4 compression to pg_dump 2023-02-23T20:19:26+00:00 Tomas Vondra tomas.vondra@postgresql.org 2023-02-23T20:19:19+00:00 0da243fed0875932f781aff08df782b56af58d02 Expand pg_dump's compression streaming and file APIs to support the lz4 algorithm. The newly added compress_lz4.{c,h} files cover all the functionality of the aforementioned APIs. Minor changes were necessary in various pg_backup_* files, where code for the 'lz4' file suffix has been added, as well as pg_dump's compression option parsing. Author: Georgios Kokolatos Reviewed-by: Michael Paquier, Rachel Heaton, Justin Pryzby, Shi Yu, Tomas Vondra Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss%3D%40protonmail.com 
Expand pg_dump's compression streaming and file APIs to support the lz4
algorithm. The newly added compress_lz4.{c,h} files cover all the
functionality of the aforementioned APIs. Minor changes were necessary
in various pg_backup_* files, where code for the 'lz4' file suffix has
been added, as well as pg_dump's compression option parsing.

Author: Georgios Kokolatos
Reviewed-by: Michael Paquier, Rachel Heaton, Justin Pryzby, Shi Yu, Tomas Vondra
Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss%3D%40protonmail.com
Introduce a generic pg_dump compression API 2023-02-23T17:33:40+00:00 Tomas Vondra tomas.vondra@postgresql.org 2023-02-23T17:33:30+00:00 e9960732a9618d5f744ff43a09622c9185798760 Switch pg_dump to use the Compression API, implemented by bf9aa490db. The CompressFileHandle replaces the cfp* family of functions with a struct of callbacks for accessing (compressed) files. This allows adding new compression methods simply by introducing a new struct instance with appropriate implementation of the callbacks. Archives compressed using custom compression methods store an identifier of the compression algorithm in their header instead of the compression level. The header version is bumped. Author: Georgios Kokolatos Reviewed-by: Michael Paquier, Rachel Heaton, Justin Pryzby, Tomas Vondra Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss%3D%40protonmail.com 
Switch pg_dump to use the Compression API, implemented by bf9aa490db.

The CompressFileHandle replaces the cfp* family of functions with a
struct of callbacks for accessing (compressed) files. This allows adding
new compression methods simply by introducing a new struct instance with
appropriate implementation of the callbacks.

Archives compressed using custom compression methods store an identifier
of the compression algorithm in their header instead of the compression
level. The header version is bumped.

Author: Georgios Kokolatos
Reviewed-by: Michael Paquier, Rachel Heaton, Justin Pryzby, Tomas Vondra
Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss%3D%40protonmail.com
Update copyright for 2023 2023-01-02T20:00:37+00:00 Bruce Momjian bruce@momjian.us 2023-01-02T20:00:37+00:00 c8e1ba736b2b9e8c98d37a5b77c4ed31baf94147 Backpatch-through: 11 
Backpatch-through: 11
Fix `trap` in a few shell scripts 2022-09-20T16:50:16+00:00 Alvaro Herrera alvherre@alvh.no-ip.org 2022-09-20T16:50:16+00:00 3d53b9ef1a4cda7b6303a0496f8531eceeef0e77 The original `trap` lines in these scripts are incomplete: in case of any signal, they delete the working directory but let the script run to completion, which is useless because it will only proceed to complain about the working directory being removed. Add `exit` there, with the original exit value (not rm's). Since this is mostly just cosmetic, no backpatch. Discussion: https://postgr.es/m/20220913181002.hzsosy7qkemb7ky7@alvherre.pgsql 
The original `trap` lines in these scripts are incomplete: in case of
any signal, they delete the working directory but let the script run to
completion, which is useless because it will only proceed to complain
about the working directory being removed.  Add `exit` there, with the
original exit value (not rm's).

Since this is mostly just cosmetic, no backpatch.

Discussion: https://postgr.es/m/20220913181002.hzsosy7qkemb7ky7@alvherre.pgsql
Fix failure to build gramparse.h standalone in vpath builds 2022-09-14T07:37:19+00:00 John Naylor john.naylor@postgresql.org 2022-09-14T07:33:35+00:00 b0526d271d16d54064666d30bbfa53d4f335c66d Add include directory in a similar fashion as 829906fb6c. Per buildfarm animal crake 
Add include directory in a similar fashion as 829906fb6c.

Per buildfarm animal crake
Move gramparse.h to src/backend/parser 2022-09-14T03:57:13+00:00 John Naylor john.naylor@postgresql.org 2022-09-14T03:41:44+00:00 ecaf7c5df54f7fa9df2fdc7225d2bb4e283f0081 This header is semi-private, being used only in files related to raw parsing, so move to the backend directory where those files live. This allows removal of Makefile rules that symlink gram.h to src/include/parser, since gramparse.h can now include gram.h from within the same directory. This has the side-effect of no longer installing gram.h and gramparse.h, but there doesn't seem to be a good reason to continue doing so. Per suggestion from Andres Freund and Peter Eisentraut Discussion: https://www.postgresql.org/message-id/20220904181759.px6uosll6zbxcum5%40awork3.anarazel.de 
This header is semi-private, being used only in files related to
raw parsing, so move to the backend directory where those files
live. This allows removal of Makefile rules that symlink gram.h to
src/include/parser, since gramparse.h can now include gram.h from
within the same directory. This has the side-effect of no longer
installing gram.h and gramparse.h, but there doesn't seem to be a
good reason to continue doing so.

Per suggestion from Andres Freund and Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/20220904181759.px6uosll6zbxcum5%40awork3.anarazel.de
Adjust header exceptions for 0bd9c6297 2022-09-13T09:57:15+00:00 John Naylor john.naylor@postgresql.org 2022-09-13T09:57:15+00:00 fcf7b3a9d42c3cf778dab0fc644f11f12684d184 Per buildfarm animal crake 
Per buildfarm animal crake
Fix cplusplusscheck in vpath builds 2022-09-06T06:51:15+00:00 John Naylor john.naylor@postgresql.org 2022-09-06T05:46:41+00:00 cec2754fbebee0a40aaf905947deb823cca4fe39 Same solution as 829906fb6. 
Same solution as 829906fb6.