diff options
Diffstat (limited to 'gdb/testsuite/gdb.gdbtk/README')
| -rw-r--r-- | gdb/testsuite/gdb.gdbtk/README | 421 |
1 files changed, 0 insertions, 421 deletions
diff --git a/gdb/testsuite/gdb.gdbtk/README b/gdb/testsuite/gdb.gdbtk/README deleted file mode 100644 index b70488647b4..00000000000 --- a/gdb/testsuite/gdb.gdbtk/README +++ /dev/null @@ -1,421 +0,0 @@ - The Insight Testsuite - --------------------- - Keith Seitz (keiths@cygnus.com) - May 1, 2001 - -RUNNING THE TESTSUITE - -The Insight testsuite is run in much the same way that gdb's testsuites -are run. The one big difference is the environment variable GDB_DISPLAY, -which governs what display should be used for the tests. - -When GDB_DISPLAY is not set in the user's environment, the Insight testsuite -will attempt to run Xvfb, an X server with a virtual frame buffer. Using -Xvfb, the testsuite can run without interuppting the user. - -When Xvfb is not available, the testsuite will mark the Insight tests -"untested" and print out some appropriate warning to the testsuite log -file. - -If GDB_DISPLAY is set in the user's environment, the testsuite will attempt -to use this display for the tests. If this display is a desktop display, -it is very likely that any interaction between the user and his desktop -will interfere with the tests. Some tests warp the cursor, i.e., they -force the mouse to move on the screen. If you don't want this to happen to -you, put Xvfb in your path. - -On Cygwin systems, Xvfb is not supported. Only two choices are available in -this environment: run the testsuites using the desktop or do not run the -testsuite. To run the testsuite on Cygwin, just define the environment -variable GDB_DISPLAY to anything. - -The examples below summarize the usage of the environment variable GDB_DISPLAY -on unix/X-Windows hosts and Cygwin hosts. In all examples, assume that DISPLAY -set to the local workstation's main display (:0). - -To run the testsuite using Xvfb -- unix only (Xvfb must be in PATH): -$ make check - -To run the testsuite using a given display (either the desktop or a peviously -started Xvfb): -$ GDB_DISPLAY=$DISPLAY make check - -To run the testsuite on Cygwin: -$ GDB_DISPLAY=foo make check - - -TESTSUITE INFRASTRUCTURE - -The rest of this document deals with writing tests for Insight. This reading -is only noteworthy for developers and contributors. - -The Insight testsuite consists of two large portions of code: code which is -run in dejagnu and code which runs in Insight's built-in Tcl interpreter. Files -containing dejagnu code (those files ending in ".exp" in the testsuite directory) -are "glue code" between gdb's dejagnu testsuite and Insight's Tcl testsuite. - -Dejagnu testsuite files are considered "drivers" for any particular set of -tests, since they allow dejagnu to control Insight's Tcl testsuite. - - -Dejagnu Testsuite Infrastructure - -The dejagnu code is responsible for doing several things. Some of the more -important responsibilities include: - -o Initializing the display -o Determining if tests should be run -o Results accounting -o Compiling testcases in various languages -o Repoting results to gdb's testsuite - -There are various functions defined to facilitate the writing of tests. These -functions currently reside in gdb's gdb.exp (src/gdb/testsuite/lib/gdb.exp) and -include: - -Pulic functions: -proc gdbtk_initialize_display {} - - gdbtk_initialize_display should be the first function called from the - (dejagnu) test file. It initializes the DISPLAY variable on unix systems - and determines (for all host systems) whether or not the testsuite should - run. It returns 1 if the test should run. If tests should not run, it - marks the test as "untested" and leaves a suitable message about why - the test should not run. If gdbtk_initialize_display returns zero, a test - should simply exit. - -proc gdbtk_start {test} - - This function marks the start of a test and will execute Insight for - testing. The TEST parameter should be the file name of the Tcl test - file to load into Insight's Tcl interpreter. It returns a list of - test results suitable for passing to gdbtk_done or gdbtk_analyze_results. - See gdbtk_analyze_results for more information on the format of results. - - gdbtk_start is responsible for communicating target startup information - to Insight, so that Insight's testsuite may be run on any target supported - by gdb. It does this by setting several environment variables just before - it starts Insight. These environment variables are: - - OBJDIR - The object directory of the dejagnu testsuite (i.e., - objdir/gdb/testsuite). - SRCDIR - The dejagnu source directory in which the tests are located (i.e, - src/gdb/testsuite) - SUBDIR - The dejagnu testsuite subdirectory for the test (i.e., gdb.gdbtk) - DEFS - The location of the testsuite definitions file (i.e., - src/gdb/testsuite/gdb.gdbtk/defs) - - Note that DEFS is converted to abs$lute tcl-style paths. On unix, - this means that DEFS would point to, for example, - /home/keiths/insight/src/gdb/testsuite/gdb.gdbtk/defs. On Cygwin it - would point to C:/cygwin/home/keiths/insight/src/gdb/testsuite/gdb.gdbtk/defs. - This is because of a descrepency between Cygwin's posix paths and Tcl's - not-quite-posix paths. - -proc gdbtk_analyze_results {results} - This function translates the list of results in RESULTS into dejagnu - results, reporting the number of failures, errors, passes, and expected - failures and passes. It currently does not deal with "untested" and other - test statuses from dejagnu since Insight's tcl testsuite does not - issue such results. - - The format of the results expected by gdbtk_analyze_results is simple: - it is a list of {status name description msg}. "status" is the execution - status of one of the tcl tests run. This can be "PASS", "FAIL", "ERROR", - "XFAIL", or "XPASS". - - "name" is the name of the test, and it is reported in all testsuite - results in dejagnu. This speeds location of the failing test. This - "name" may also be given to Insight's testsuite, telling it to - only run this test. See "do_tests" in Tcl Testsuite Infrastructure - for more information. - - "description" is a textual description of the test given by "name". - - "msg" is currently not used. - -proc gdbtk_done {{results {}}} - gdbtk_done takes any RESULTS and passes it gdbtk_analyze_results for - outputting to the dejagnu part of the testsuite. It may be called - without an argument, in which case it will only terminate Xvfb if it - was started. Tests must call gdbtk_done _once_ at the end of their - test drivers. - -Private functions: -proc _gdbtk_export_target_info - This functin exports information about the target into the environment - so that Insight's testsuite may run programs on any supported gdb - target. This target information is passed into the Tcl testsuite - via an environment variable, TARGET_INFO, which is really a Tcl array, - i.e., the array is constructed in tcl and exported into the environment - with Tcl's "array get" command). - - There are four elements to the array: - TARGET_INFO(init) - (optional) A list of commands to execute in gdb - to initialize the session. This usually includes - setting baud rates and remote protocol options. - TARGET_INFO(target) - (required) The complete "target" command to connect - to the given target. - TARGET_INFO(load) - (optional) The complete "load" command to load an - executable into a target. - TARGET_INFO(run) - (required) The complete "run" command, sans arguments, - to start a process on the target. For remote targets, - this is usually just "continue". - -proc _gdbtk_xvfb_init - This procedure actually determines whether the an Insight test should - run and what DISPLAY it should use for that test. It is called by - gdbtk_initialize_display to do most of the dirty work. - - It has a simple heuristic: If GDB_DISPLAY is not set and Xvfb is available - (on unix), it starts Xvfb, using the current process id as the screen number. - If Xvfb is not available and GDB_DISPLAY was not set, it skips the tests. - -proc _gdbtk_xvfb_exit - _gdbtk_xvfb_exit will kill any previously started Xvfb. - -Private globals: -global _xvfb_spawn_id - This variable holds the spawn_id of any Xvfb process started - by the testsuite (or it is left undefined). - -global _using_windows - A global variable which indicates whether the testsuite is running - on cygwin. Unfortunately, as of writing, the global variable - tcl_platform(platform) is "unix" on Cygwin, so it is not possible - to rely on this for platform-dependent operations. - - Instead, this variable is set by gdbtk_initialize_display. The test - it uses to determine if Cygwin is being used: it looks for the program - cygpath in the PATH. Therefore, cygpath is REQUIRED to run the testsuite - on Cygwin. (gdbtk_start also uses cygpath to determine Windows - pathnames for Cygwin.) - - -Testsuite Driver Basics - -Given the above interfaces for connecting Insight's Tcl testsuite and -gdb's dejagnu testsuite, the basic testsuite driver file should look -(minimally) like this: - -File: mytest.exp -1 load_lib "insight-support.exp" -2 if {[gdbtk_initialize_display]} { -3 # We found a display to use -4 gdb_exit; # Make sure any previous gdb is gone -5 set results [gdbtk_start mytest.test] -6 -7 # Done! -8 gdbtk_done [split $results \n] -9 } - -Line 1 loads the insight testsuite support library which contains definitions -for all the procedures used in dejagnu to initialize and run the Insight testsuite. -Line 2 calls gdbtk_initialize_display to ascertain whether there is a display -to use for the test. This could use an existing display (if GDB_DISPLAY is -set in the environment) or gdbk_initialize_display could startup an Xvfb -for use by the testsuite. - -Line 4 forces any previously executing gdb to terminate. - -Line 5 signals the start of the test run. "mytest.test" is the name of the -Tcl test file to execute in Insight's built-in Tcl interpreter. The output -of gdbtk_start_test is all of the results of the Tcl test from Insight, which -is subsequently passed to gdbk_analyze_results via gdbtk_done on Line 8. - -Note how nothing happens if gdbtk_initialize_display returns false. - - -Tcl Testsuite Infrastructure - -The heart of Insight's testsuite is its Tcl testsuite. It is these tests -which run directly in Insight's Tcl interpreter and allow test writers -access to Insight's internals. Tcl testsuite files have the filename suffix -".test" to distinguish them from their driver files, which end in ".exp". - -The design of the Insight Tcl testsuite parallels Tcl's testsuite. It has -many powerful features, including the ability to run ANY test in a given -Tcl test file. See the description of utility routines below for more -information about running any set of tests from a file. - -The bulk of the code implementing the Tcl testsuite infrastructure in -Insight is contained in the testsuite definitions file, "defs", located -in src/gdb/testsuite/gdb.gdbtk. This file contains routines necessary -to write tests for Insight. - -Public functions: -proc gdbtk_read_defs {} - This function, located in Insight's core Tcl library, attempts to load - the testsuite definitions file. If it fails, it will either pop up - a dialog box with the error (if running interactively) or it will - print the error to stderr and exit (if running non-interactively). - - If successful, it will return true. - -proc gdbtk_test_file {filename} - This function is used to load the file given by FILENAME into - Insight. It will automatically append ".exe" to any FILENAME - on Cygwin-hosted systems. - - If successful, it will load the given file into Insight and - return the output of gdb's file command. It will call "error" - if it was succesful, therefore all calls to "gdbtk_test_file" - should be called using "catch". - - Test authors should not use "gdb_cmd {file FILENAME}" to load - files into gdb unless they are testing interface code between - gdb and Insight. - -proc gdbtk_test_run {{prog_args {}}} - gdbtk_test_run runs the previoiusly loaded executable, passing - the given arguments to the inferior. Like Insight's Run button, - it will do whatever is necessary to get the executable running, - including any target initialization (setting baud rate and remote - protocol options), downloading the executable to the target, and - finally starting execution. - - Test authors should NEVER use "gdb_cmd {run ARGUMENTS}" to run an - executable. Doing so will insure that your tests will only run on - a native debugger. - - It returns true if successful or false otherwise. It will report - the error in a dialog box (if running interactively) or it will - print the error to stderr. - -proc gdbtk_test {name description script answer} - This is Tcl testsuite equivalent of "expect". "name" is a canonical - name of the test, usually of the form "shortname-major.minor". This is - the name that is used when running selected tests from a given file. - If "name" starts with an asterisk (*), it designates that the test - is expected to fail. - - "description" is a short textual description of the test to help - humans understand what it does. - - "script" is the actual test script to run. The result of this script - will be compared against "answer" to determine if the test passed - or failed. - - It calls gdbtk_print_verbose to print out the results to the terminal - (if running interactively) or to the log file. - -proc gdbtk_test_done {} - gdbtk_test_done is called at the very end of all tcl tests. It is used - to exit Insight and return control back to the dejagnu driver which - started the tests. - -proc gdbtk_dotests {file args} - Obsolete. - -proc do_test {{file {}} {verbose {}} {tests {}}} - This procedure is used to invoke the Insight test(s) given in FILE - which match the regular expression(s) in TESTS. This is invoked - from Insight's console window to run tests interactively. - - VERBOSE sets the verbosity of the test run. When set to one, - the testsuite will report all results in human readable form. - When set greater than one, it will print out results as a list, - i.e., for passing to gdbtk_analyze_results. If zero, it will only - print errors and failures in human readable form. - -Public global variables: -objdir - The objdir from dejagnu. See gdbtk_start for more information. -srcdir - The srcdir from dejagnu. See gdbtk_start for more information. -test_ran - Indicates whether the last test ran or not. See example below. - -Private functions: -proc gdbtk_test_error {desc} - An internal function used to report a framework error in the testsuite. - "desc" is a description of the error. It calls gdbtk_test_done. - -proc gdbtk_print_verbose {status name description script code answer} - A helper procedure to gdbtk_test which prints out results to the terminal - or the logfile (or both or none). - -Private global variables: -_test - An array used by the testsuite internals. - - -Tcl Test Basics - -Armed with the basic interface described above, it is possible to test Insight's -GUI. Please do not write tests which attempt to imitate a user (moving the -mouse and clicking buttons), unless there is no other way to test the functionality. - -The basic test file (with one test) looks like this (nonsensical one): -File: mytest.test -1 if {![gdbtk_read_defs]} { -2 break -3 } -4 -5 global objdir test_ran -6 set program [file join $objdir mytest] -7 if {[catch {gdbtk_test_file $program} t]} { -8 gdbtk_test_error "loading \"$program\": $t" -9 } -10 if {![gdbtk_test_run]} { exit 1 } -11 -12 global foo -13 set foo 1 -14 -15 # Test: mytest-1.1 -16 # Desc: check if a source window was created -17 gdbtk_test mytest-1.1 {source window created} { -18 set window [ManagedWin::find SrcWin] -19 llength $window -20 set foo 13 -21 } {1} -22 -23 if {$test_ran} { -24 set foo 1 -25 } -26 -27 # Done -28 gdbtk_test_done - -Line 1 calls the Inisght function gdbtk_read_defs to read in the testsuite -definitions file. - -Line 6 then specifies the name of a file (mytest) in the object directory -which is loaded into gdb on Line 7. If loading the file into Insight -failed, gdbtk_test_error is called to publish the error (and terminate the -test run for this file). - -Line 10 runs the executable on the target, and exits if it was unable -to do so. - -Line 13 simply sets a global variable foo to illustrate the purpose -of the global "test_ran". Before the test "mytest-1.1" runs, foo is set to -one. In order to support running specific tests, the state of the debugger -cannot be altered INSIDE gdbtk_test scripts, since the contents of the -script may not be run if the user requested only a specific test to run. - -Line 20 in the middle of the test modifies the global foo. If subsequent -test relied on foo being one, we would have a state violation, since -mytest-1.1 may have (or may have not) run. - -Therefore, we can check if a test ran and reset foo by checking the -global "test_ran". If set, we know that the previous test (mytest-1.1) -was run, and that foo is now thirteen. We reset the result back to one. - -(Aside: Some tests do not follow this rule explicitly: they can assume -that all tests run sequentially. In these cases, running a specific -test in the file will probably fail, since the debugger is not brought -to a known state along the way.) - -Lines 17-21 contain the actual test. The test's name is "mytest-1.1". It -is this name that may be referred to when asking the testsuite to run -a specific test. The description of this test is "source window created", -indicating that mytest-1.1's purpose is to check whether a source window -was created. - -If gdbtk_test determines that this test is to run, it will execute the -next part, lines 18-20, and compare the output of that script (llength -$window) with "1". If the result is not "1", a failure is recorded. If -it is "1", a pass is recorded. - -Finally, the test file is done and exits on line 28. |