gdb: Respect the DW_CC_nocall attribute

It is possible for a compiler to optimize a function in a such ways that
the function does not follow the calling convention of the target.  In
such situation, the compiler can use the DW_AT_calling_convention
attribute with the value DW_CC_nocall to tell the debugger that it is
unsafe to call the function.  The DWARF5 standard states, in 3.3.1.1:

  > If the value of the calling convention attribute is the constant
  > DW_CC_nocall, the subroutine does not obey standard calling
  > conventions, and it may not be safe for the debugger to call this
  > subroutine.

Non standard calling convention can affect GDB's assumptions in multiple
ways, including how arguments are passed to the function, how values are
returned, and so on.  For this reason, it is unsafe for GDB to try to do
the following operations on a function with marked with DW_CC_nocall:

- call / print an expression requiring the function to be evaluated,
- inspect the value a function returns using the 'finish' command,
- force the value returned by a function using the 'return' command.

This patch ensures that if a command which relies on GDB's knowledge of
the target's calling convention is used on a function marked nocall, GDB
prints an appropriate message to the user and does not proceed with the
operation which is unreliable.

Note that it is still possible for someone to use a vendor specific
value for the DW_AT_calling_convention attribute for example to indicate
the use of an alternative calling convention.  This commit does not
prevent this, and target dependent code can be adjusted if one wanted to
support multiple calling conventions.

Tested on x86_64-Linux, with no regression observed.

Change-Id: I72970dae68234cb83edbc0cf71aa3d6002a4a540

1 files changed, 5 insertions, 0 deletions

diff --git a/gdb/infcall.c b/gdb/infcall.c
index 8e896296b96..f8c812c8f61 100644
--- a/gdb/infcall.c
+++ b/gdb/infcall.c
@@ -812,6 +812,11 @@ call_function_by_hand_dummy (struct value *function,
   type *values_type;
   CORE_ADDR funaddr = find_function_addr (function, &values_type, &ftype);
 
+  if (is_nocall_function (ftype))
+    error (_("Cannot call the function '%s' which does not follow the "
+	     "target calling convention."),
+	   get_function_name (funaddr, name_buf, sizeof (name_buf)));
+
   if (values_type == NULL)
     values_type = default_return_type;
   if (values_type == NULL)