Rework additional ssh argument handling

Now we have the following ways to set additional arguments:

1. [ssh_connection]ssh_args in ansible.cfg: global setting, prepended to
   every command line for ssh/scp/sftp. Overrides default ControlPersist
   settings.
2. ansible_ssh_common_args inventory variable. Appended to every command
   line for ssh/scp/sftp. Used in addition to ssh_args, if set above, or
   the default settings.
3. ansible_{sftp,scp,ssh}_extra_args inventory variables. Appended to
   every command line for the relevant binary only. Used in addition to
   #1 and #2, if set above, or the default settings.
3. Using the --ssh-common-args or --{sftp,scp,ssh}-extra-args command
   line options (which are overriden by #2 and #3 above).

This preserves backwards compatibility (for ssh_args in ansible.cfg),
but also permits global settings (e.g. ProxyCommand via _common_args) or
ssh-specific options (e.g. -R via ssh_extra_args).

Fixes #12576

7 files changed, 101 insertions, 42 deletions

diff --git a/docs/man/man1/ansible-playbook.1.asciidoc.in b/docs/man/man1/ansible-playbook.1.asciidoc.in
index 2a1a94c5cd..356ad545e6 100644
--- a/docs/man/man1/ansible-playbook.1.asciidoc.in
+++ b/docs/man/man1/ansible-playbook.1.asciidoc.in
@@ -151,10 +151,23 @@ run operations with su as this user (default=root)
 
 Run operations with sudo (nopasswd) (deprecated, use become)
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-U*, 'SUDO_USER', *--sudo-user=*'SUDO_USER'::
 
diff --git a/docs/man/man1/ansible-pull.1.asciidoc.in b/docs/man/man1/ansible-pull.1.asciidoc.in
index 520a60bf21..c0a5ab9ed2 100644
--- a/docs/man/man1/ansible-pull.1.asciidoc.in
+++ b/docs/man/man1/ansible-pull.1.asciidoc.in
@@ -105,10 +105,23 @@ Purge the checkout after the playbook is run.
 
 Sleep for random interval (between 0 and SLEEP number of seconds) before starting.  This is a useful way ot disperse git requests.
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-t* 'TAGS', *--tags=*'TAGS'::
 
diff --git a/docs/man/man1/ansible.1.asciidoc.in b/docs/man/man1/ansible.1.asciidoc.in
index 7578e8f8be..07172ffd9b 100644
--- a/docs/man/man1/ansible.1.asciidoc.in
+++ b/docs/man/man1/ansible.1.asciidoc.in
@@ -143,10 +143,23 @@ Run operations with su as this user (default=root)
 
 Run the command as the user given by -u and sudo to root.
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-U* 'SUDO_USERNAME', *--sudo-user=*'SUDO_USERNAME'::
 
diff --git a/docsite/rst/intro_inventory.rst b/docsite/rst/intro_inventory.rst
index 353aebe794..d3ac8cfa18 100644
--- a/docsite/rst/intro_inventory.rst
+++ b/docsite/rst/intro_inventory.rst
@@ -212,11 +212,16 @@ SSH connection::
       The ssh password to use (this is insecure, we strongly recommend using --ask-pass or SSH keys)
     ansible_ssh_private_key_file
       Private key file used by ssh.  Useful if using multiple keys and you don't want to use SSH agent.
-    ansible_ssh_args
-      This setting overrides any ``ssh_args`` configured in ``ansible.cfg``.
+    ansible_ssh_common_args
+      This setting is always appended to the default command line for
+      sftp, scp, and ssh. Useful to configure a ``ProxyCommand`` for a
+      certain host (or group).
+    ansible_sftp_extra_args
+      This setting is always appended to the default sftp command line.
+    ansible_scp_extra_args
+      This setting is always appended to the default scp command line.
     ansible_ssh_extra_args
-      Additional arguments for ssh. Useful to configure a ``ProxyCommand`` for a certain host (or group).
-      This is used in addition to any ``ssh_args`` configured in ``ansible.cfg`` or the inventory.
+      This setting is always appended to the default ssh command line.
     ansible_ssh_pipelining
       Determines whether or not to use SSH pipelining. This can override the
       ``pipelining`` setting in ``ansible.cfg``.
diff --git a/lib/ansible/cli/__init__.py b/lib/ansible/cli/__init__.py
index 5aeef380db..a3d128f911 100644
--- a/lib/ansible/cli/__init__.py
+++ b/lib/ansible/cli/__init__.py
@@ -314,8 +314,14 @@ class CLI(object):
                 help="connection type to use (default=%s)" % C.DEFAULT_TRANSPORT)
             parser.add_option('-T', '--timeout', default=C.DEFAULT_TIMEOUT, type='int', dest='timeout',
                 help="override the connection timeout in seconds (default=%s)" % C.DEFAULT_TIMEOUT)
+            parser.add_option('--ssh-common-args', default='', dest='ssh_common_args',
+                help="specify common arguments to pass to sftp/scp/ssh (e.g. ProxyCommand)")
+            parser.add_option('--sftp-extra-args', default='', dest='sftp_extra_args',
+                help="specify extra arguments to pass to sftp only (e.g. -f, -l)")
+            parser.add_option('--scp-extra-args', default='', dest='scp_extra_args',
+                help="specify extra arguments to pass to scp only (e.g. -l)")
             parser.add_option('--ssh-extra-args', default='', dest='ssh_extra_args',
-                help="specify extra arguments to pass to ssh (e.g. ProxyCommand)")
+                help="specify extra arguments to pass to ssh only (e.g. -R)")
 
         if async_opts:
             parser.add_option('-P', '--poll', default=C.DEFAULT_POLL_INTERVAL, type='int', dest='poll_interval',
diff --git a/lib/ansible/playbook/play_context.py b/lib/ansible/playbook/play_context.py
index dfccf7345b..9cadcf9a76 100644
--- a/lib/ansible/playbook/play_context.py
+++ b/lib/ansible/playbook/play_context.py
@@ -140,6 +140,9 @@ class PlayContext(Base):
     _private_key_file = FieldAttribute(isa='string', default=C.DEFAULT_PRIVATE_KEY_FILE)
     _timeout          = FieldAttribute(isa='int', default=C.DEFAULT_TIMEOUT)
     _shell            = FieldAttribute(isa='string')
+    _ssh_common_args  = FieldAttribute(isa='string')
+    _sftp_extra_args  = FieldAttribute(isa='string')
+    _scp_extra_args   = FieldAttribute(isa='string')
     _ssh_extra_args   = FieldAttribute(isa='string')
     _connection_lockfd= FieldAttribute(isa='int')
     _pipelining       = FieldAttribute(isa='bool', default=C.ANSIBLE_SSH_PIPELINING)
@@ -240,6 +243,9 @@ class PlayContext(Base):
 
         self.remote_user = options.remote_user
         self.private_key_file = options.private_key_file
+        self.ssh_common_args = options.ssh_common_args
+        self.sftp_extra_args = options.sftp_extra_args
+        self.scp_extra_args = options.scp_extra_args
         self.ssh_extra_args = options.ssh_extra_args
 
         # privilege escalation
diff --git a/lib/ansible/plugins/connection/ssh.py b/lib/ansible/plugins/connection/ssh.py
index fce231127a..293fa014fd 100644
--- a/lib/ansible/plugins/connection/ssh.py
+++ b/lib/ansible/plugins/connection/ssh.py
@@ -33,6 +33,7 @@ from ansible import constants as C
 from ansible.errors import AnsibleError, AnsibleConnectionFailure, AnsibleFileNotFound
 from ansible.plugins.connection import ConnectionBase
 from ansible.utils.path import unfrackpath, makedirs_safe
+from ansible.utils.vars import combine_vars
 
 SSHPASS_AVAILABLE = None
 
@@ -47,15 +48,21 @@ class Connection(ConnectionBase):
         super(Connection, self).__init__(*args, **kwargs)
 
         self.host = self._play_context.remote_addr
-        self.ssh_extra_args = ''
-        self.ssh_args = ''
+        for v in ['ssh_common_args', 'sftp_extra_args', 'scp_extra_args', 'ssh_extra_args']:
+            setattr(self, v, '')
 
     def set_host_overrides(self, host):
-        v = host.get_vars()
-        if 'ansible_ssh_extra_args' in v:
-            self.ssh_extra_args = v['ansible_ssh_extra_args']
-        if 'ansible_ssh_args' in v:
-            self.ssh_args = v['ansible_ssh_args']
+        # FIXME: The following can only use the variables set directly against
+        # the host ("hostname var=...") or the group ("[group:vars] ...") in the
+        # inventory file, but NOT those read from group_vars/host_vars files or
+        # any other source. That's clearly wrong, but we don't have access to a
+        # VariableManager here, so I don't know how to get at those settings.
+
+        vars = combine_vars(host.get_group_vars(), host.get_vars())
+        for v in ['ssh_common_args', 'sftp_extra_args', 'scp_extra_args', 'ssh_extra_args']:
+            name = 'ansible_%s' % v
+            if name in vars:
+                setattr(self, v, vars[name])
 
     # The connection is created by running ssh/scp/sftp from the exec_command,
     # put_file, and fetch_file methods, so we don't need to do any connection
@@ -151,8 +158,7 @@ class Connection(ConnectionBase):
         if binary == 'sftp' and C.DEFAULT_SFTP_BATCH_MODE:
             self._command += ['-b', '-']
 
-        elif binary == 'ssh':
-            self._command += ['-C']
+        self._command += ['-C']
 
         if self._play_context.verbosity > 3:
             self._command += ['-vvv']
@@ -160,14 +166,10 @@ class Connection(ConnectionBase):
             # Older versions of ssh (e.g. in RHEL 6) don't accept sftp -q.
             self._command += ['-q']
 
-        # Next, we add ansible_ssh_args from the inventory if it's set, or
-        # [ssh_connection]ssh_args from ansible.cfg, or the default Control*
-        # settings.
+        # Next, we add [ssh_connection]ssh_args from ansible.cfg, or the default
+        # Control* settings.
 
-        if self.ssh_args:
-            args = self._split_args(self.ssh_args)
-            self._add_args("inventory set ansible_ssh_args", args)
-        elif C.ANSIBLE_SSH_ARGS:
+        if C.ANSIBLE_SSH_ARGS:
             args = self._split_args(C.ANSIBLE_SSH_ARGS)
             self._add_args("ansible.cfg set ssh_args", args)
         else:
@@ -189,7 +191,7 @@ class Connection(ConnectionBase):
 
         if self._play_context.port is not None:
             self._add_args(
-                "ANSIBLE_REMOTE_PORT/remote_port/ansible_ssh_port set",
+                "ANSIBLE_REMOTE_PORT/remote_port/ansible_port set",
                 ("-o", "Port={0}".format(self._play_context.port))
             )
 
@@ -212,7 +214,7 @@ class Connection(ConnectionBase):
         user = self._play_context.remote_user
         if user and user != pwd.getpwuid(os.geteuid())[0]:
             self._add_args(
-                "ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set",
+                "ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set",
                 ("-o", "User={0}".format(self._play_context.remote_user))
             )
 
@@ -221,16 +223,17 @@ class Connection(ConnectionBase):
             ("-o", "ConnectTimeout={0}".format(self._play_context.timeout))
         )
 
-        # If any extra SSH arguments are specified in the inventory for
-        # this host, or specified as an override on the command line,
-        # add them in.
-
-        if self._play_context.ssh_extra_args:
-            args = self._split_args(self._play_context.ssh_extra_args)
-            self._add_args("command-line added --ssh-extra-args", args)
-        elif self.ssh_extra_args:
-            args = self._split_args(self.ssh_extra_args)
-            self._add_args("inventory added ansible_ssh_extra_args", args)
+        # If the inventory specifies either common or binary-specific arguments
+        # applicable to this host, or they are specified as an override on the
+        # command line, add them in now.
+
+        for opt in ['ssh_common_args', binary + '_extra_args']:
+            if getattr(self._play_context, opt):
+                args = self._split_args(getattr(self._play_context, opt))
+                self._add_args("command-line added --%s" % opt.replace('_', '-'), args)
+            elif getattr(self, opt):
+                args = self._split_args(getattr(self, opt))
+                self._add_args("inventory added ansible_%s" % opt, args)
 
         # Check if ControlPersist is enabled (either by default, or using
         # ssh_args or ssh_extra_args) and add a ControlPath if one hasn't