4_debugger.pdf

Wind River SystemsTornado Training Workshop © Copyright Wind River Systems 4-1

Chapter

4

CrossWind

Using Tornado’s source-level debugger to debug target programs.


CrossWind

4.1 Overview

Starting A Debugging Session

Basic Debugging

System-Level Debugging

What is CrossWind?


Overview

• Allows debugging at source and assembly levels.

• CrossWind executes on host; code executes on target.

• Based on GNU debugger GDB:

● Debug engine customized for VxWorks multitaskingand Tornado target server communication strategy.

● Graphical user interface added. Windows and UnixGUIs differ.

• CrossWind uses the WTX protocol to communicate with

the target server.

• The Unix version of CrossWind is a separate, stand-alone program. The

Windows version of CrossWind is integrated into the Tornado

development environment. Only the command-line interface is

available as a separate program under Windows.

• Future releases of Tornado may unify the debugger GUIs on Windows

and Unix.

• The debug engine for Tornado is based upon GDB 4.17.

• See the Tornado User’s Guide for information on the CrossWind GUI and

Wind River’s extensions to GDB; see Debugging with GDB for

information on command-line use. Brief descriptions of GDB

commands are also available via GDBs internal help command.


Customization

• The Debugger is easy to customize:

● Extend or modify user interface

● Add new debugger commands

• To customize the user interface: Use Tcl language.

● Put Tcl commands in crosswind.tcl● Interpreted by GUI Tcl interpreter

• To customize the debug engine:

● Put Tcl commands in gdb.tcl● Interpreted by debug engine Tcl interpreter

● Put GDB commands in .gdbinit

• WRS extensions provide access to the WTX protocol via

Tcl functions.

• Much of the Tornado tools code is written in Tcl. This code is provided

under wind/host/resource/tcl/. See the Tornado User’s Guide for some

CrossWind customization examples; others are provided in lab.

• User Tcl customizations such as crosswind.tcl and gdb.tcl should be

put in the .wind subdirectory of your home directory. In Windows, for

this purpose your home directory is given by the environment variable

%HOME%, which you need to set; also, in Windows only, before

searching in this home directory, Tornado looks in the .windsubdirectory of the Tornado installation directory.

• Wind River Tcl extensions are documented in the Tornado API Guide.

• One can have the debugger read GDB commands from a file:

(gdb) source filename


Task vs. System Level Debugging

• Task level debugging:

● Can debug tasks only, not ISRs.

● By default, breakpoints affect only the attached task.Global task breakpoints may also be set.

● When the attached task is stopped, other tasks andISRs in the system continue executing.

● Communication with WDB agent is interrupt driven.

• System level debugging:

● Can debug tasks, ISRs, and pre-kernel execution.

● Default breakpoints stop the whole system. Task-specific breakpoints may be set.

● When system stopped, external WDB agent runswith interrupts locked out. During this time,communication with WDB agent is in polled mode.

• When debugging at task level, GDB may be attached to only a single

task at a time (the attached task), but the debugger may easily be

detached from one task and attached to another. When debugging at

system level, GDB is said to be attached to system.

• To debug at system level over ethernet requires an END network

interface driver, which supports polled mode communication. Tornado

serial drivers also support polled mode and system level debugging.

• The last section of this chapter will focus on issues specific to system

level debugging.


CrossWind

Overview

4.2 Starting A Debugging Session

Basic Debugging


Invoking the debugger

Locating source and object modules

Listing code

Debugging tasks

Getting help


Executing CrossWind

• Compile code with the -g option to generate debug

symbols. Specifying -O0 is also recommended.

• Select a target server.

• To invoke the debugger:

● Use the launcher CrossWind button (UNIX)

● Use Tools => Debugger or button. (Windows)

• To exit the debugger:

● Select File => Quit from the debugger menu. (UNIX)

● Select Debug => Stop Debugging from the Tornado

menu bar, or use button.(Windows)

• CrossWind connects to the selected target server automatically when

invoked from the Launcher (UNIX) or Tornado IDE (Windows).

• To connect manually to the target from the GDB prompt:

(gdb) target wtx targetServerName

The first parameter to the target command specifies the connectionstrategy, here the WTX protocol.

• The GNU compiler allows specifying both debugging info (-g) and

optimization (-O flags). Compiling with -g and no -O flags is equivalent

to compiling with -g -O0. Compiling with aggressive optimization (-O2or higher) when debugging can result in confusion, both to the

debugger program and the debugger user! Except in rare cases when

timing changes from compiler optimizations are involved in the

problems you see, it is best to debug code with no optimization.


Locating Code

• CrossWind needs to access both object and source code

for modules to be debugged symbolically. Usually it can

locate such modules automatically.

• Modules may be downloaded to the target via

● Project Facility Download button or menu item.

● WindSh ld() command, CrossWind load command,or any other Tornado tool which can load modules.

• When GDB starts, it querries the target server for the file

path to each object module loaded on the target.

● The object module contains symbolic debuginformation if compiled with -g.

● If debug information is found, CrossWind searchesits source path for the corresponding source code.

• In the Windows version of Tornado, CrossWind is informed of the

locations of modules downloaded after it starts, so it rarely has

difficulty locating them. In the UNIX versions of Tornado, if you

download modules onto the target using a tool other than CrossWind

after CrossWind has been started, it is necessary to inform CrossWind of

the module before one can debug it. This may be done with the add-symbol-file command: In this case, tell GDB to load the module’s debug

information using the add-symbol-file command:

(gdb) add-sym /path/to/module.o• If you specify only the module name without path, GDB searches its

source path for the module. This list of directories is also used when

searching for source code. It may be configured using the GDB dircommand, or (under Windows) with Debug => Source Search Path... .

The current source path may be displayed using show dir.


Listing Source

list location

• Displays the C or/and assembly source code centered

at location.

• Without argument, displays source:

(gdb) list

Used to get source into source display window. Thishappens automatically when GDB attaches to a task.

• Examples, using argument:

(gdb) list foo.c:1 First few lines of foo.c

(gdb) list 25 Line 25 in foo.c

(gdb) list mySub Location can be a function.

• In both Unix and Windows, one may use GUI controls (scrollbars, etc.)

to navigate through source, rather than the list command. The listcommand is useful in Unix to display source in the CrossWind source

panel before attaching to a task, in order to set breakpoints, for instance.

In Windows, CrossWind uses the normal Tornado editor to display files,

so list is less frequently necessary.

• In mixed source/disassembly mode, each source line is followed in the

display by the assembly instructions it generates. Some source lines

generate discontiguous sequences of assembly instructions, which can

be detected in the listing by jumps in the assembly addresses.

• The GDB commands search and reverse may be used in the Unix

versions of CrossWind to search forwards or backwards for lines

matching a regular expression. Under Windows, the normal search

facilities of the Tornado editor (Alt-F3, F3) apply.


Debugging a Task

• Use CrossWind to create a new task tDbgTask that runs

the function to debug:

● (gdb) run function arg1 arg2 ...● Windows users may use Debug->Run or the

button.

● You may set breakpoints before running the task.

• To debug a task already running on the target, GDB

must attach to the task.

● Targets->Attach Task... in CrossWind (UNIX)

● Debug->Attach... in Tornado (Windows)

● (gdb) attach taskId● The task is suspended wherever it is executing.

(Windows users may override this default.)

• The priority and stack size of the new task are controlled by GDB

variables, which may be modified using the set command:

(gdb) set wtx-task-priority 200 default 100

(gdb) set wtx-task-stack-size 5000 default 20000

• Current values of GDB settings may be displayed with the showcommand, e.g.

(gdb) show wtx-task-stack-sizeStack size (in bytes) of tasks created using "run" commandis 5000.

(gdb) show wtx-load-timeoutTimeout in seconds used when loading new objects on targetis 30.

• The help show gives describes the debugger variables GDB can display

with the show command.


Debugging Multiple Tasks

• At task level, a CrossWind instance may be attached to a

single task at a time. To debug multiple tasks, either

detach from one and attach to another, use system level

debugging, or start multiple debugger instances.

• To detach from a task, resuming it, use

● Targets->Detach Task/System (UNIX)

● Detach and Resume (Windows)

● detach from the GDB command line.

• To detach from a task, leaving it in its current state, use

● Debug->Detach (Windows)

● detach auto from the GDB command line.

• (GDB) sdetach detaches, leaving the task suspended.

• The GDB detach resume (same as detach) command resumes the task

which the debugger is detaching from, if it is not already “in flight”, and

if it has not just encountered an exception.

• detach suspend suspends the task from which it detaches, if it is not

already suspended. sdetach is an alias for detach suspend.

• The command sattach taskId is equivalent to sdetach followed by

attach taskId.

• In the Windows environment, at most one CrossWind instance is

allowed within a Tornado IDE session. To start multiple CrossWinds,

one needs to start multiple Tornado sessions.


Automatic Attachment (Windows)

• In Windows, CrossWind may be configured to

automatically attach to a task which generates an

exception or hits a breakpoint.

• Use Tools => Options => Debugger dialog.

• Three choices:

● Never auto-attach.

● Auto-attach only if not already attached.

● Auto-attach even if already attached.

• The last choice, always auto-attach, can lead to flurries

of reattachments when several tasks hit a global

breakpoint.


CrossWind

Overview


4.3 Basic Debugging


How to use CrossWind to carry out common debugging operations.


Debugging Tools

• Examples of debugger functionality:

● Set/delete breakpoints.

● Step to next line of code.

● Step over a function call.

● Continue program execution.

● Finish current subroutine.

● Move up or down stack frames.

● Monitor variables or other expressions.

● De-reference pointers.

● Call up editor (UNIX).

• For a complete list of debugger tools and GUI access

methods, see the Tornado User’s Guide.

• The Tcl implementations underlying WindSh built-in commands can be

executed from the debugger.

• To execute the Tcl code underlying a WindSh built-in from GDB, enter

wind- followed by the name of the built-in. If the WindSh command is

composed of several words, change all uppercase characters to

lowercase and separate each word with a hyphen (-):

(gdb) wind-mem-show

(gdb) wind-show [gdbEvalScalar mySem]

For a complete list of WindSh commands, enter:

(gdb) wind-<TAB><TAB> (UNIX)

• This facility does not provide all the functionality of WindSh. For

instance, automatic allocation of target memory (for string arguments,

etc.) and conversion of symbol names to values are not supported.


Make it Go! Make it Stop!

step

next

continue

finish

interrupt

Windows UNIX

• The next command is commonly called step over, since it steps over a

function call.

• The step and next commands step by assembly instruction when

assembly is shown (mixed or assembly display mode, or when there is

no source). The GDB command for stepping a single assembly

instruction is si. The step command will step over a function call to a

function for which debug information is not available.

• The continue command continues execution until either a breakpoint is

hit, the interrupt button is used, or the task exits.

• The finish command continues until execution leaves the selected stack

frame (see next page), or until a breakpoint is hit, etc..

• The interrupt button may be used to stop the attached task or system if it

is executing without hitting any breakpoints.


A View of the StackClick on desired frame toselect it.

The selected frame is usedin informational commandssuch asinfo locals.

It also affects the stoppingaddress offinish.

• The debugger provides a variety of stack manipulation routines:

● backtrace Trace the stack.

● info frame Display complete info on selected frame.

● info locals Display values of all local variables.

● info args Display arguments.

● up or Move up one frame number.

● down or Move down one frame number.

● frame n Move to a particular numbered frame. Withno argument, centers program display atcontext point.

● finish Continue until exiting the selected frame.

• The appearance of the context pointer triangle changes (yellow to green

on Windows, solid to shaded on UNIX) when the current frame is not

the frame in which the program is executing.


Breakpoints

• Default breakpoints:

● Stop attached task, or whole system in system mode.

• Global breakpoints:

● Stop any breakable task. Good for stopping tasksspawned by the task being debugged.

• Hardware breakpoints on some architectures.

● Halts on instruction access or various sorts of dataaccess to specified address. Limited in number.

● PPC 603, 603e, 604, 860, 403; Mips R4650; x86;i960 CA, JX, HX

• Temporary breakpoints:

● Effective once, then deleted (default) or disabled.

• Example GDB breakpoint commands:

(gdb) b hello.c:23 Set breakpoint on line 23 of hello.c

(gdb) b 105 Set breakpoint on line 105 of current file

(gdb) tbreak cleanup Temp. breakpoint on cleanup function.

(gdb) gbreak *0x3fb798 Global breakpoint at address 0x3fb798.

• Breakpoints set by WindSh are global by default.

• A context menu is a menu which appears when you right-click with the

mouse. Typically it contains operations relevant to the item or area in

which the mouse was clicked, so that it is context sensitive.

• Hardware breakpoints may be set using the WindSh command bh() and

the Unix CrossWind GUI. See the reference entry for bh() and the

Programmer’s Guide architecture-specific appendices for more

information.


Breakpoints, Continued.

• Conditional breakpoints:

● Effective if specified expression is true, otherwisetask hitting breakpoint is automatically resumed.

● (gdb) break snafu.c:228 if fbCount > 4● Debug => Breakpoints => Advanced (Windows)

• Listing breakpoints:

● GDB info breakpoints (i b) command.

● Debug->Breakpoints dialog.(Windows)

• A condition may be applied to an existing breakpoint with the gdb

condition command, and an ignore count for a breakpoint may be

specified with the ignore command. In Windows, the AdvancedBreakpoint dialog may be used here; a conditional expression may need

to be quoted from the Tcl interpreter by enclosing it in braces {}.

• A breakpoint may be deleted by location with the clear command, or by

its GDB breakpoint number (listed in info breakpoints output) with the

delete command.

• The maximum number of breakpoints managed by the WDB agent is

configurable via the WDB_BP_MAX parameter of the WDB breakpointscomponent. It defaults to 50.


Information

• Displaying variables and expressions:

● Watch windows - extendable, editable, dockable,hierarchical display. (Windows)

● Inspect windows - hierarchical display. Followingpointers opens additional inspect windows. (UNIX)

● (GDB) display /W expr● (GDB) print expr

• Other display windows:

● Backtrace, Registers (UNIX, Windows)

● Locals, Memory dump (Windows)

• The GDB print command may be used to evaluate even expressions

with side effects, such as assignments or function calls. Be cautious

using this feature.

• Common GDB information commands:

● info locals local variables of selected frame

● info args arguments of selected frame

● info stack or backtrace stack frame listing

● info breakpoints gdb breakpoints and their status

● info registers non-floating-point registers

● info all-registers registers including floating point.

● info threads system level debug list of tasks

● info addr dump memory in various formats,including disassembly.

See Debugging with GDB or help info for more information.


UNIX: The Graphical InterfaceMenu BarToolbar

ProgramDisplayPanel

CommandPanel

• The Menu Bar has the following menus:

● File - Download, or Quit the debugger.

● Source - Select source, assembly, or mixed code display

● Target - Select a target, attach to task or system.

● Tcl - Re-initialize GUI Tcl code.

● Windows - Create windows for viewing a task’s registers or stack.

• Toolbar allows you to execute common debugging commands like

setting breakpoints, stepping through code, or displaying a variable.

• Program-Display Panel displays source and/or assembly code, and

allows you to set breakpoints.

• Enter any GDB command in the Command Panel.


PC: The Graphical Interface

• The Debug menu gives access to debugging facilities. Icons in the Debugtoolbar can also be used.

• Watch, Variables, Register, Backtrace and Memory windows may be

conveniently docked at the periphery of the main window.

• Source and/or assembly code is displayed in a normal editor window.

The left margin of the window displays breakpoints and the contextpointer (which indicates where the execution point is in the current

frame).

• Commands can be entered in the GDB command window (Debug=>

Debug Windows=>Debug Command-line ). The command window is both

less necessary and less convenient in Windows than in UNIX. If you use

it, you may need to quote arguments to GDB commands from the Tcl

interpreter by enclosing them in braces {}.


CrossWind

Overview


Basic Debugging

4.4 System-Level Debugging

System-level debugging


Overview

• System-level debugging supports:

● Debugging ISRs.

● Debugging before the kernel is started.

● Multitask debugging.

• In system mode, VxWorks and all application tasks and

ISRs stop when a breakpoint is hit.

• The external WDB agent runs with interrupts locked

out when VxWorks is stopped.

• VxWorks configuration must include the WDB systemdebugging component.

• The WindSh and other Tornado tools besides CrossWind may be used in

system mode, with some restriction of functionality.

• System level debugging requires a communications path which

supports polled-mode communication by the external WDB agent. To

debug over ethernet requires an END driver, which supports this mode.

• If you are using a serial connection, use the highest possible data rate

supported by both host and target.

• Unless you are debugging before the kernel is started, the WDB taskdebugging component may also be included.

• See the end of the Projects chapter of the Tornado User’s Guide for

information on how to do system level debugging before the VxWorks

kernel has started.


Debugging In System Mode

• To enter system mode:

(gdb) attach systemAttaching to system.0x407ae in wdbSuspendSystemHere ()

• To exit system mode:

(gdb) detach

• While in system mode, one can use normal debugger

features:

● Step/continue

● Set breakpoints

● Display expressions

• wdbSuspendSystemHere( ) is a routine which transfers control to the

external WDB agent.

• You may also use the menu items Debug->Attach system (Windows) or

Targets->Attach System (UNIX) to enter system level debugging.

• The WindSh commands sysSuspend() and sysResume() may be used to

suspend or resume the system, transferring control to or from the

external WDB agent. One should be cautious using the WindSh to start

or stop the system this way under circumstances in which the debugger

might think itself in control of the system state.


Selecting a Thread To Debug• To display all threads (the GDB term for VxWorks

contexts), use the info threads command:

(gdb) i th5 task 0x3e8984 tExcTask 0x72e50 in qJobGet()4 task 0x3e605c tLogTask 0x72e50 in qJobGet()3 task 0x3b349c + tNetTask 0x38e44 in

wdbSuspendSystemHere()2 task 0x389de4 tPortmapd 0x3c36a in noSemQPutEvt()1 task 0x3881c0 tWdbTask 0x3c36a in noSemQPutEvt()

• To select the current thread, for information display:

(gdb) thread 4[Switching to task 0x3e605c tLogTask ]#0 0x72e50 in qJobGet ()

• In the threads list:

+ indicates the active thread where VxWorks is currently stopped, andfrom which execution will continue.

* indicates the GDB’s notion of the current thread, implicitly referencedby such commands as info locals, backtrace, and so on. This thread maybe changed with the thread command. You cannot step or continuethis thread unless it is the same as the active thread.

• Use the thread number, which is the small integer on the left of the

threads list, to refer to a thread.


Examining a Thread

• Use normal debugger tools to examine a thread.

• To set a thread-specific breakpoint:

(gdb) break location thread threadNumber● If the breakpoint is hit by a different thread, the

system is resumed automatically.

• Resume execution of the system with Step , Next , Finish ,

or Continue from the debugger menu.

● When you continue the system, VxWorks controlsscheduling.

● You may not be in the same task after a step, next, orfinish command completes. Such commands mayinvolve setting a breakpoint and resuming thesystem.

• If the system is in flight and is not hitting any breakpoints, you may

interrupt it using the Interrupt debugger button.

• Thread-specific breakpoints can also be made conditional:

(gdb) break printf thread 3 if (myVal < 12)

• To apply a command to a list of threads:

(gdb) thread apply [all |listOfThreads] yourCommand

For example, to print out the program counter for all threads:

(gdb) thread apply all print/x $pc


Caveats

• Cannot create a new task with run.

• Can only step or continue from where system is

stopped.

• Cannot use multiple debuggers per target.

• The WindSh built-in function sp() may be used to spawn a task while

debugging in system mode. The task is spawned by the exception task

when the system is continued. You may wish to examine the shell Tcl

code for the sp( ) command in wind/host/resource/tcl/shell.tcl to see

how this is done.


Summary

• Tornado supports a source-code debugger which:

● Executes on host while code executes on target

● Provides source-level debugging

• Code must be compiled with -g option. For easiest

debugging, also specify -O0 (or no -O flags).

• System-level debugging:

● Is useful for debugging ISRs and BSPs.

● Is useful for multitask debugging.

● VxWorks and application freeze when breakpoint ishit.

● WDB agent uses polled-mode communication whensystem suspended, as interrupts are locked out.

4_debugger.pdf

Documents