Allow direct and internal clients to use the replication network

A new header `X-Backend-Use-Replication-Network` is added; if true, use
the replication network instead of the client-data-path network.

Several background daemons are updated to use the replication network:

  * account-reaper
  * container-reconciler
  * container-sharder
  * container-sync
  * object-expirer

Note that if container-sync is being used to sync data within the same
cluster, the replication network will only be used when communicating
with the "source" container; the "destination" traffic will continue to
use the configured realm endpoint.

The direct and internal client APIs still default to using the
client-data-path network; this maintains backwards compatibility for
external tools written against them.

UpgradeImpact
=============

Until recently, servers configured with

  replication_server = true

would only handle REPLICATE (and, in the case of object servers, SSYNC)
requests, and would respond 405 Method Not Allowed to other requests.
When upgrading from Swift 2.25.0 or earlier, remove the config option
and restart services prior to upgrade to avoid a flood of background
daemon errors in logs.

Note that some background daemons find work by querying Swift rather
than walking local drives that should be available on the replication
network:

  * container-reconciler
  * object-expirer

Previosuly these may have been configured without access to the
replication network; ensure they have access before upgrading.

Closes-Bug: #1883302
Related-Bug: #1446873
Related-Change: Ica2b41a52d11cb10c94fa8ad780a201318c4fc87
Change-Id: Ieef534bf5d5fb53602e875b51c15ef565882fbff

1 files changed, 15 insertions, 2 deletions

diff --git a/swift/common/request_helpers.py b/swift/common/request_helpers.py
index c120ac277..c660da74b 100644
--- a/swift/common/request_helpers.py
+++ b/swift/common/request_helpers.py
@@ -40,14 +40,15 @@ from swift.common.utils import split_path, validate_device_partition, \
     close_if_possible, maybe_multipart_byteranges_to_document_iters, \
     multipart_byteranges_to_document_iters, parse_content_type, \
     parse_content_range, csv_append, list_from_csv, Spliterator, quote, \
-    RESERVED
+    RESERVED, config_true_value
 from swift.common.wsgi import make_subrequest
-from swift.container.reconciler import MISPLACED_OBJECTS_ACCOUNT
 
 
 OBJECT_TRANSIENT_SYSMETA_PREFIX = 'x-object-transient-sysmeta-'
 OBJECT_SYSMETA_CONTAINER_UPDATE_OVERRIDE_PREFIX = \
     'x-object-sysmeta-container-update-override-'
+USE_REPLICATION_NETWORK_HEADER = 'x-backend-use-replication-network'
+MISPLACED_OBJECTS_ACCOUNT = '.misplaced_objects'
 
 
 if six.PY2:
@@ -849,3 +850,15 @@ def update_ignore_range_header(req, name):
         raise ValueError('Header name must not contain commas')
     hdr = 'X-Backend-Ignore-Range-If-Metadata-Present'
     req.headers[hdr] = csv_append(req.headers.get(hdr), name)
+
+
+def get_ip_port(node, headers):
+    use_replication_network = False
+    for h, v in headers.items():
+        if h.lower() == USE_REPLICATION_NETWORK_HEADER:
+            use_replication_network = config_true_value(v)
+            break
+    if use_replication_network:
+        return node['replication_ip'], node['replication_port']
+    else:
+        return node['ip'], node['port']