From 0fdab27ad68a059a1663fa5ce48d76333f1bd74c Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Sat, 8 Apr 2023 02:20:01 -0700 Subject: Allow logical decoding on standbys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Unsurprisingly, this requires wal_level = logical to be set on the primary and standby. The infrastructure added in 26669757b6a ensures that slots are invalidated if the primary's wal_level is lowered. Creating a slot on a standby waits for a xl_running_xact record to be processed. If the primary is idle (and thus not emitting xl_running_xact records), that can take a while. To make that faster, this commit also introduces the pg_log_standby_snapshot() function. By executing it on the primary, completion of slot creation on the standby can be accelerated. Note that logical decoding on a standby does not itself enforce that required catalog rows are not removed. The user has to use physical replication slots + hot_standby_feedback or other measures to prevent that. If catalog rows required for a slot are removed, the slot is invalidated. See 6af1793954e for an overall design of logical decoding on a standby. Bumps catversion, for the addition of the pg_log_standby_snapshot() function. Author: "Drouvot, Bertrand" Author: Andres Freund (in an older version) Author: Amit Khandekar (in an older version) Reviewed-by: Andres Freund Reviewed-by: FabrÌzio de Royes Mello Reviewed-by: Amit Kapila Reviewed-By: Robert Haas --- doc/src/sgml/func.sgml | 15 +++++++++++++++ doc/src/sgml/logicaldecoding.sgml | 27 +++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) (limited to 'doc') diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 4211d31f30..bf4c61ccfb 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27074,6 +27074,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with . + + + + pg_log_standby_snapshot + + pg_log_standby_snapshot () + pg_lsn + + + Take a snapshot of running transactions and write it to WAL, without + having to wait bgwriter or checkpointer to log one. This is useful for + logical decoding on standby, as logical slot creation has to wait + until such a record is replayed on the standby. + + diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..ebe0376e3e 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,33 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. + + A logical replication slot can also be created on a hot standby. To prevent + VACUUM from removing required rows from the system + catalogs, hot_standby_feedback should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Then, the + primary may delete system catalog rows that could be needed by the logical + decoding on the standby (as it does not know about the catalog_xmin on the + standby). Existing logical slots on standby also get invalidated if wal_level + on primary is reduced to less than 'logical'. This is done as soon as the + standby detects such a change in the WAL stream. It means, that for walsenders + that are lagging (if any), some WAL records up to the wal_level parameter change + on the primary won't be decoded. + + + + Creation of a logical slot requires information about all the currently + running transactions. On the primary, this information is available + directly, but on a standby, this information has to be obtained from + primary. Thus, slot creation may need to wait for some activity to happen + on the primary. If the primary is idle, creating a logical slot on + standby may take noticeable time. This can be sped up by calling the + pg_log_standby_snapshot on the primary. + + Replication slots persist across crashes and know nothing about the state -- cgit v1.2.1