Fix invalid frag_index header in ssync_sender when reverting EC tombstones

Back in d124ce [1] we failed to recognize the situation where a revert
job would have an explicit frag_index key wth the literal value None
which would take precedence over the dict.get's default value of ''.
Later in ssync_receiver we'd bump into the ValueError converting 'None'
to an int (again).

In ssync_sender we now handle literal None's correctly and should
hopefully no longer put this invalid headers on the wire - but for belts
and braces we'll also update ssync_receiver to raise a 400 series error
and ssync_sender to better log the error messages.

1. https://review.openstack.org/#/c/195457/

Co-Author: Clay Gerrard <clay.gerrard@gmail.com>
Co-Author: Alistair Coles <alistair.coles@hp.com>
Change-Id: Ic71ba7cc82487773214030207bb193f425319449
Closes-Bug: 1489546

1 files changed, 13 insertions, 6 deletions

diff --git a/swift/obj/ssync_sender.py b/swift/obj/ssync_sender.py
index cf6fcad6a..83030782d 100644
--- a/swift/obj/ssync_sender.py
+++ b/swift/obj/ssync_sender.py
@@ -133,9 +133,16 @@ class Sender(object):
             # a sync job must use the node's index for the frag_index of the
             # rebuilt fragments instead of the frag_index from the job which
             # will be rebuilding them
-            self.connection.putheader(
-                'X-Backend-Ssync-Frag-Index', self.node.get(
-                    'index', self.job.get('frag_index', '')))
+            frag_index = self.node.get('index', self.job.get('frag_index'))
+            if frag_index is None:
+                # replication jobs will not have a frag_index key;
+                # reconstructor jobs with only tombstones will have a
+                # frag_index key explicitly set to the value of None - in both
+                # cases on the wire we write the empty string which
+                # ssync_receiver will translate to None
+                frag_index = ''
+            self.connection.putheader('X-Backend-Ssync-Frag-Index',
+                                      frag_index)
             # a revert job to a handoff will not have a node index
             self.connection.putheader('X-Backend-Ssync-Node-Index',
                                       self.node.get('index', ''))
@@ -144,10 +151,10 @@ class Sender(object):
                 self.daemon.node_timeout, 'connect receive'):
             self.response = self.connection.getresponse()
             if self.response.status != http.HTTP_OK:
-                self.response.read()
+                err_msg = self.response.read()[:1024]
                 raise exceptions.ReplicationException(
-                    'Expected status %s; got %s' %
-                    (http.HTTP_OK, self.response.status))
+                    'Expected status %s; got %s (%s)' %
+                    (http.HTTP_OK, self.response.status, err_msg))
 
     def readline(self):
         """