infrun: step through indirect branch thunks

With version 7.3 GCC supports new options

   -mindirect-branch=<choice>
   -mfunction-return=<choice>

The choices are:

    keep                behaves as before
    thunk               jumps through a thunk
    thunk-external      jumps through an external thunk
    thunk-inline        jumps through an inlined thunk

For thunk and thunk-external, GDB would, on a call to the thunk, step into
the thunk and then resume to its caller assuming that this is an
undebuggable function.  On a return thunk, GDB would stop inside the
thunk.

Make GDB step through such thunks instead.

Before:
    Temporary breakpoint 1, main ()
        at gdb.base/step-indirect-call-thunk.c:37
    37        x = apply (inc, 41);
    (gdb) s
    apply (op=0x80483e6 <inc>, x=41)
        at gdb.base/step-indirect-call-thunk.c:29
    29        return op (x);
    (gdb)
    30      }

After:
    Temporary breakpoint 1, main ()
        at gdb.base/step-indirect-call-thunk.c:37
    37        x = apply (inc, 41);
    (gdb) s
    apply (op=0x80483e6 <inc>, x=41)
        at gdb.base/step-indirect-call-thunk.c:29
    29        return op (x);
    (gdb)
    inc (x=41) at gdb.base/step-indirect-call-thunk.c:23
    23        return x + 1;

This is independent of the step-mode.  In order to step into the thunk,
you would need to use stepi.

When stepping over an indirect call thunk, GDB would first step through
the thunk, then recognize that it stepped into a sub-routine and resume to
the caller (of the thunk).  Not sure whether this is worth optimizing.

Thunk detection is implemented via gdbarch.  I implemented the methods for
IA.  Other architectures may run into unexpected fails.

The tests assume a fixed number of instruction steps to reach a thunk.
This depends on the compiler as well as the architecture.  They may need
adjustments when we add support for more architectures.  Or we can simply
drop those tests that cover being able to step into thunks using
instruction stepping.

When using an older GCC, the tests will fail to build and will be reported
as untested:

    Running .../gdb.base/step-indirect-call-thunk.exp ...
    gdb compile failed, \
    gcc: error: unrecognized command line option '-mindirect-branch=thunk'
    gcc: error: unrecognized command line option '-mfunction-return=thunk'

                    === gdb Summary ===

    # of untested testcases         1

gdb/
	* infrun.c (process_event_stop_test): Call
	gdbarch_in_indirect_branch_thunk.
	* gdbarch.sh (in_indirect_branch_thunk): New.
	* gdbarch.c: Regenerated.
	* gdbarch.h: Regenerated.
	* x86-tdep.h: New.
	* x86-tdep.c: New.
	* Makefile.in (ALL_TARGET_OBS): Add x86-tdep.o.
	(HFILES_NO_SRCDIR): Add x86-tdep.h.
	(ALLDEPFILES): Add x86-tdep.c.
	* arch-utils.h (default_in_indirect_branch_thunk): New.
	* arch-utils.c (default_in_indirect_branch_thunk): New.
	* i386-tdep: Include x86-tdep.h.
	(i386_in_indirect_branch_thunk): New.
	(i386_elf_init_abi): Set in_indirect_branch_thunk gdbarch
	function.
	* amd64-tdep: Include x86-tdep.h.
	(amd64_in_indirect_branch_thunk): New.
	(amd64_init_abi): Set in_indirect_branch_thunk gdbarch function.

testsuite/
	* gdb.base/step-indirect-call-thunk.exp: New.
	* gdb.base/step-indirect-call-thunk.c: New.
	* gdb.reverse/step-indirect-call-thunk.exp: New.
	* gdb.reverse/step-indirect-call-thunk.c: New.

1 files changed, 76 insertions, 0 deletions

diff --git a/gdb/x86-tdep.c b/gdb/x86-tdep.c
new file mode 100644
index 00000000000..3e665c2be4f
--- /dev/null
+++ b/gdb/x86-tdep.c
@@ -0,0 +1,76 @@
+/* Target-dependent code for X86-based targets.
+
+   Copyright (C) 2018 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include "defs.h"
+#include "x86-tdep.h"
+
+
+/* Check whether NAME is included in NAMES[LO] (inclusive) to NAMES[HI]
+   (exclusive).  */
+
+static bool
+x86_is_thunk_register_name (const char *name, const char **names, int lo,
+			    int hi)
+{
+  int reg;
+  for (reg = lo; reg < hi; ++reg)
+    if (strcmp (name, names[reg]) == 0)
+      return true;
+
+  return false;
+}
+
+/* See x86-tdep.h.  */
+
+bool
+x86_in_indirect_branch_thunk (CORE_ADDR pc, const char **register_names,
+			      int lo, int hi)
+{
+  struct bound_minimal_symbol bmfun = lookup_minimal_symbol_by_pc (pc);
+  if (bmfun.minsym == nullptr)
+    return false;
+
+  const char *name = MSYMBOL_LINKAGE_NAME (bmfun.minsym);
+  if (name == nullptr)
+    return false;
+
+  /* Check the indirect return thunk first.  */
+  if (strcmp (name, "__x86_return_thunk") == 0)
+    return true;
+
+  /* Then check a family of indirect call/jump thunks.  */
+  static const char thunk[] = "__x86_indirect_thunk";
+  static const size_t length = sizeof (thunk) - 1;
+  if (strncmp (name, thunk, length) != 0)
+    return false;
+
+  /* If that's the complete name, we're in the memory thunk.  */
+  name += length;
+  if (*name == '\0')
+    return true;
+
+  /* Check for suffixes.  */
+  if (*name++ != '_')
+    return false;
+
+  if (x86_is_thunk_register_name (name, register_names, lo, hi))
+    return true;
+
+  return false;
+}