gdb/python: extend the Python Disassembler API to allow for styling

This commit extends the Python Disassembler API to allow for styling
of the instructions.

Before this commit the Python Disassembler API allowed the user to do
two things:

  - They could intercept instruction disassembly requests and return a
    string of their choosing, this string then became the disassembled
    instruction, or

  - They could call builtin_disassemble, which would call back into
    libopcode to perform the disassembly.  As libopcode printed the
    instruction GDB would collect these print requests and build a
    string.  This string was then returned from the builtin_disassemble
    call, and the user could modify or extend this string as needed.

Neither of these approaches allowed for, or preserved, disassembler
styling, which is now available within libopcodes for many of the more
popular architectures GDB supports.

This commit aims to fill this gap.  After this commit a user will be
able to do the following things:

  - Implement a custom instruction disassembler entirely in Python
    without calling back into libopcodes, the custom disassembler will
    be able to return styling information such that GDB will display
    the instruction fully styled.  All of GDB's existing style
    settings will affect how instructions coming from the Python
    disassembler are displayed in the expected manner.

  - Call builtin_disassemble and receive a result that represents how
    libopcode would like the instruction styled.  The user can then
    adjust or extend the disassembled instruction before returning the
    result to GDB.  Again, the instruction will be styled as expected.

To achieve this I will add two new classes to GDB,
DisassemblerTextPart and DisassemblerAddressPart.

Within builtin_disassemble, instead of capturing the print calls from
libopcodes and building a single string, we will now create either a
text part or address part and store these parts in a vector.

The DisassemblerTextPart will capture a small piece of text along with
the associated style that should be used to display the text.  This
corresponds to the disassembler calling
disassemble_info::fprintf_styled_func, or for disassemblers that don't
support styling disassemble_info::fprintf_func.

The DisassemblerAddressPart is used when libopcodes requests that an
address be printed, and takes care of printing the address and
associated symbol, this corresponds to the disassembler calling
disassemble_info::print_address_func.

These parts are then placed within the DisassemblerResult when
builtin_disassemble returns.

Alternatively, the user can directly create parts by calling two new
methods on the DisassembleInfo class: DisassembleInfo.text_part and
DisassembleInfo.address_part.

Having created these parts the user can then pass these parts when
initializing a new DisassemblerResult object.

Finally, when we return from Python to gdbpy_print_insn, one way or
another, the result being returned will have a list of parts.  Back in
GDB's C++ code we walk the list of parts and call back into GDB's core
to display the disassembled instruction with the correct styling.

The new API lives in parallel with the old API.  Any existing code
that creates a DisassemblerResult using a single string immediately
creates a single DisassemblerTextPart containing the entire
instruction and gives this part the default text style.  This is also
what happens if the user calls builtin_disassemble for an architecture
that doesn't (yet) support libopcode styling.

This matches up with what happens when the Python API is not involved,
an architecture without disassembler styling support uses the old
libopcodes printing API (the API that doesn't pass style info), and
GDB just prints everything using the default text style.

The reason that parts are created by calling methods on
DisassembleInfo, rather than calling the class constructor directly,
is DisassemblerAddressPart.  Ideally this part would only hold the
address which the part represents, but in order to support backwards
compatibility we need to be able to convert the
DisassemblerAddressPart into a string.  To do that we need to call
GDB's internal print_address function, and to do that we need an
gdbarch.

What this means is that the DisassemblerAddressPart needs to take a
gdb.Architecture object at creation time.  The only valid place a user
can pull this from is from the DisassembleInfo object, so having the
DisassembleInfo act as a factory ensures that the correct gdbarch is
passed over each time.  I implemented both solutions (the one
presented here, and an alternative where parts could be constructed
directly), and this felt like the cleanest solution.

Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Reviewed-By: Tom Tromey <tom@tromey.com>

1 files changed, 81 insertions, 13 deletions

diff --git a/gdb/testsuite/gdb.python/py-disasm.exp b/gdb/testsuite/gdb.python/py-disasm.exp
index 5cbf02fc9fe..304393f71ab 100644
--- a/gdb/testsuite/gdb.python/py-disasm.exp
+++ b/gdb/testsuite/gdb.python/py-disasm.exp
@@ -69,6 +69,12 @@ set nop "(nop|nop\t0)"
 set unknown_error_pattern "unknown disassembler error \\(error = -1\\)"
 set addr_pattern "\r\n=> ${curr_pc_pattern} <\[^>\]+>:\\s+"
 set base_pattern "${addr_pattern}${nop}"
+
+# Helper proc to format a Python exception of TYPE with MSG.
+proc make_exception_pattern { type msg } {
+    return "${::addr_pattern}Python Exception <class '$type'>: $msg\r\n\r\n${::unknown_error_pattern}"
+}
+
 set test_plans \
     [list \
 	 [list "" "${base_pattern}\r\n.*"] \
@@ -90,13 +96,40 @@ set test_plans \
 	 [list "RethrowMemoryErrorDisassembler" "${addr_pattern}Cannot access memory at address $hex"] \
 	 [list "ReadMemoryMemoryErrorDisassembler" "${addr_pattern}Cannot access memory at address ${curr_pc_pattern}"] \
 	 [list "ReadMemoryGdbErrorDisassembler" "${addr_pattern}read_memory raised GdbError\r\n${unknown_error_pattern}"] \
-	 [list "ReadMemoryRuntimeErrorDisassembler" "${addr_pattern}Python Exception <class 'RuntimeError'>: read_memory raised RuntimeError\r\n\r\n${unknown_error_pattern}"] \
+	 [list "ReadMemoryRuntimeErrorDisassembler" \
+	      [make_exception_pattern "RuntimeError" \
+		   "read_memory raised RuntimeError"]] \
 	 [list "ReadMemoryCaughtMemoryErrorDisassembler" "${addr_pattern}${nop}\r\n.*"] \
 	 [list "ReadMemoryCaughtGdbErrorDisassembler" "${addr_pattern}${nop}\r\n.*"] \
 	 [list "ReadMemoryCaughtRuntimeErrorDisassembler" "${addr_pattern}${nop}\r\n.*"] \
-	 [list "MemorySourceNotABufferDisassembler" "${addr_pattern}Python Exception <class 'TypeError'>: Result from read_memory is not a buffer\r\n\r\n${unknown_error_pattern}"] \
-	 [list "MemorySourceBufferTooLongDisassembler" "${addr_pattern}Python Exception <class 'ValueError'>: Buffer returned from read_memory is sized $decimal instead of the expected $decimal\r\n\r\n${unknown_error_pattern}"] \
-	 [list "ResultOfWrongType" "${addr_pattern}Python Exception <class 'TypeError'>: Result is not a DisassemblerResult.\r\n.*"]]
+	 [list "MemorySourceNotABufferDisassembler" \
+	      [make_exception_pattern "TypeError" \
+		   "Result from read_memory is not a buffer"]] \
+	 [list "MemorySourceBufferTooLongDisassembler" \
+	      [make_exception_pattern "ValueError" \
+		   "Buffer returned from read_memory is sized $decimal instead of the expected $decimal"]] \
+	 [list "ResultOfWrongType" \
+	      [make_exception_pattern "TypeError" \
+		   "Result is not a DisassemblerResult."]] \
+	 [list "ErrorCreatingTextPart_NoArgs" \
+	      [make_exception_pattern "TypeError" \
+		  "function missing required argument 'style' \\(pos 1\\)"]] \
+	 [list "ErrorCreatingAddressPart_NoArgs" \
+	      [make_exception_pattern "TypeError" \
+		  "function missing required argument 'address' \\(pos 1\\)"]] \
+	 [list "ErrorCreatingTextPart_NoString" \
+	      [make_exception_pattern "TypeError" \
+		  "function missing required argument 'string' \\(pos 2\\)"]] \
+	 [list "ErrorCreatingTextPart_NoStyle" \
+	      [make_exception_pattern "TypeError" \
+		  "function missing required argument 'style' \\(pos 1\\)"]] \
+	 [list "All_Text_Part_Styles" "${addr_pattern}p1p2p3p4p5p6p7p8p9p10\r\n.*"] \
+	 [list "ErrorCreatingTextPart_StringAndParts" \
+	      [make_exception_pattern "ValueError" \
+		  "Cannot use 'string' and 'parts' when creating gdb\\.disassembler\\.DisassemblerResult\\."]] \
+	 [list "Build_Result_Using_All_Parts" \
+	      "${addr_pattern}fake\treg, ${curr_pc_pattern}(?: <\[^>\]+>)?, 123\r\n.*"] \
+	]
 
 # Now execute each test plan.
 foreach plan $test_plans {
@@ -216,13 +249,48 @@ with_test_prefix "Bad DisassembleInfo creation" {
 	     "Error while executing Python code\\."]
 }
 
-# Test that we can't inherit from the DisassemblerResult class.
-gdb_test_multiline "Sub-class a breakpoint" \
-    "python" "" \
-    "class InvalidResultType(gdb.disassembler.DisassemblerResult):" "" \
-    "   def __init__(self):" "" \
-    "     pass" "" \
-    "end" \
+# Some of the disassembler related types should not be sub-typed,
+# check these now.
+with_test_prefix "check inheritance" {
+    foreach_with_prefix type {gdb.disassembler.DisassemblerResult \
+				  gdb.disassembler.DisassemblerPart
+				  gdb.disassembler.DisassemblerTextPart \
+				  gdb.disassembler.DisassemblerAddressPart} {
+	set type_ptn [string_to_regexp $type]
+	gdb_test_multiline "Sub-class a breakpoint" \
+	    "python" "" \
+	    "class InvalidResultType($type):" "" \
+	    "   def __init__(self):" "" \
+	    "     pass" "" \
+	    "end" \
+	    [multi_line \
+		 "TypeError: type '${type_ptn}' is not an acceptable base type" \
+		 "Error while executing Python code\\."]
+    }
+}
+
+
+# Test some error conditions when creating a DisassemblerResult object.
+gdb_test "python result = gdb.disassembler.DisassemblerResult()" \
     [multi_line \
-	 "TypeError: type 'gdb\\.disassembler\\.DisassemblerResult' is not an acceptable base type" \
-	 "Error while executing Python code\\."]
+	 "TypeError: function missing required argument 'length' \\(pos 1\\)" \
+	 "Error while executing Python code\\."] \
+    "try to create a DisassemblerResult without a length argument"
+
+foreach len {0 -1} {
+    gdb_test "python result = gdb.disassembler.DisassemblerResult($len)" \
+	[multi_line \
+	     "ValueError: Length must be greater than 0\\." \
+	     "Error while executing Python code\\."] \
+	"try to create a DisassemblerResult with length $len"
+}
+
+# Check we can't directly create DisassemblerTextPart or
+# DisassemblerAddressPart objects.
+foreach type {DisassemblerTextPart DisassemblerAddressPart} {
+    gdb_test "python result = gdb.disassembler.${type}()" \
+	[multi_line \
+	     "RuntimeError: Cannot create instances of DisassemblerPart\\." \
+	     "Error while executing Python code\\."] \
+	 "try to create an instance of ${type}"
+}