Chapter 14. Tools for Programmers

14.1.1. Tracing a Program

papaya$ trymh < image00.pgm > image00.pbm 
Segmentation fault (core dumped)

[52]The sample programs in this section are not programs you're likely to run into anywhere; they were thrown together by the authors for the purpose of demonstration.

Some of the automatic optimizations performed by gcc can be confusing when using a debugger. To turn off all optimization (even optimizations performed without specifying -O), use the -O0 (that's dash-oh-zero) option with gcc.

Now we can fire up gdb to see what the problem might be:

papaya$ gdb trymh 
GNU gdb 20010316
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-suse-linux".
(gdb)

Now gdb is waiting for a command. (The command help displays information on the available commands.) The first thing we want to do is start running the program so that we can observe its behavior. However, if we immediately use the run command, the program simply executes until it exits or crashes.

(gdb) list 
12        main( ) { 
13 
14         FloatImage inimage; 
15         FloatImage outimage; 
16         BinaryImage binimage; 
17         int i,j; 
18 
19         inimage = (FloatImage)imLoadF(IMAGE_FLOAT,stdin); 
20         outimage = laplacian_float(inimage); 
21  
(gdb) break 19 
Breakpoint 1 at 0x289c: file trymh.c, line 19. 
(gdb)

(gdb) run < image00.pgm > image00.pfm 
Starting program: /amd/dusk/d/mdw/vis/src/trymh < image00.pgm > image00.pfm 

Breakpoint 1, main ( ) at trymh.c:19 
19         inimage = (FloatImage)imLoadF(IMAGE_FLOAT,stdin); 
(gdb)

As expected, the breakpoint is reached immediately at the first line of code. We can now take over.

imLoadF is a function that loads an image from a disk file. We know this function is not at fault (you'll have to trust us on that one), so we wish to step over it using the next command:

(gdb) next 
20         outimage = laplacian_float(inimage); 
(gdb)

Here, we are interested in tracing into the suspicious-looking laplacian_float function, so we use the step command:

(gdb) step 
laplacian_float (fim=0x0) at laplacian.c:21 
21          i = 20.0; 
(gdb)

Let's use the list command to get some idea of where we are:

(gdb) list  
16        FloatImage laplacian_float(FloatImage fim) {  
17  
18          FloatImage mask;  
19          float i;  
20  
21          i = 20.0;  
22          mask=(FloatImage)imNew(IMAGE_FLOAT,3,3);  
23          imRef(mask,0,0) = imRef(mask,2,0) = imRef(mask,0,2) = 1.0;  
24          imRef(mask,2,2) = 1.0; imRef(mask,1,0) = imRef(mask,0,1) = i/5;  
25          imRef(mask,2,1) = imRef(mask,1,2) = i/5; imRef(mask,1,1) = -i;  
(gdb) list  
26  
27          return convolveFloatWithFloat(fim,mask);  
28        }  
(gdb)

As you can see, using list multiple times just displays more of the code. Because we don't want to step manually through this code, and we're not interested in the imNew function on line 22, let's continue execution until line 27. For this, we use the until command:

(gdb) until 27 
laplacian_float (fim=0x0) at laplacian.c:27 
27          return convolveFloatWithFloat(fim,mask);  
(gdb)

(gdb) print mask 
$1 = (struct {...} *) 0xe838 
(gdb) print fim 

$2 = (struct {...} *) 0x0 
(gdb)

mask looks fine, but fim, the input image, is null. Obviously, laplacian_float was passed a null pointer instead of a valid image. If you have been paying close attention, you noticed this as we entered laplacian_float earlier.

(gdb) finish 
Run till exit from #0  laplacian_float (fim=0x0) at laplacian.c:27 
0x28c0 in main ( ) at trymh.c:20 
20       outimage = laplacian_float(inimage); 
Value returned is $3 = (struct {...} *) 0x0 
(gdb)

Now we're back in main. To determine the source of the problem, let's examine the values of some variables:

(gdb) list 
15       FloatImage outimage; 
16       BinaryImage binimage; 
17       int i,j; 
18       
19       inimage = (FloatImage)imLoadF(IMAGE_FLOAT,stdin); 
20       outimage = laplacian_float(inimage); 
21       
22       binimage = marr_hildreth(outimage); 
23       if  (binimage =  = NULL) { 
24         fprintf(stderr,"trymh: binimage returned NULL\n"); 
(gdb) print inimage 
$6 = (struct {...} *) 0x0 
(gdb)

The variable inimage, containing the input image returned from imLoadF, is null. Passing a null pointer into the image manipulation routines certainly would cause a core dump in this case. However, we know imLoadF to be tried and true because it's in a well-tested library, so what's the problem?

As it turns out, our library function imLoadF returns NULL on failure — if the input format is bad, for example. Because we never checked the return value of imLoadF before passing it along to laplacian_float, the program goes haywire when inimage is assigned NULL. To correct the problem, we simply insert code to cause the program to exit with an error message if imLoadF returns a null pointer.

(gdb) quit 
The program is running.  Quit anyway (and kill it)? (y or n) y 
papaya$

14.1.2. Examining a Core File

Other errors that result in core files are so-called "bus errors" and "floating-point exceptions." Bus errors result from using incorrectly aligned data and are therefore rare on the Intel architecture, which does not pose the strong alignment conditions that other architectures do, such as SPARC. Floating-point exceptions point to a severe problem in a floating-point calculation like an overflow, but the most usual case is a division by zero.

However, not all such memory errors will cause immediate crashes. For example, you may overwrite memory in some way, but the program continues to run, not knowing the difference between actual data and instructions or garbage. Subtle memory violations can cause programs to behave erratically. One of the authors once witnessed a bug that caused the program to jump randomly around, but without tracing it with gdb, it still appeared to work normally. The only evidence of a bug was that the program returned output that meant, roughly, that two and two did not add up to four. Sure enough, the bug was an attempt to write one too many characters into a block of allocated memory. That single-byte error caused hours of grief.

ulimit -c unlimited

probably in your .bashrc initialization file. You can specify a maximum size for core files other than unlimited, but truncated core files may not be of use when debugging applications.

Also, in order for a core file to be useful, the program must be compiled with debugging code enabled, as described in the previous section. Most binaries on your system will not contain debugging code, so the core file will be of limited value.

Our example for using gdb with a core file is yet another mythical program called cross. Like trymh in the previous section, cross takes an image file as input, does some calculations on it, and outputs another image file. However, when running cross, we get a segmentation fault:

papaya$ cross < image30.pfm > image30.pbm 
Segmentation fault (core dumped) 
papaya$

To invoke gdb for use with a core file, you must specify not only the core filename, but also the name of the executable that goes along with that core file. This is because the core file does not contain all the information necessary for debugging:

papaya$ gdb cross core 
GDB is free software and you are welcome to distribute copies of it
 under certain conditions; type "show copying" to see the conditions.
There is absolutely no warranty for GDB; type "show warranty" for details.
GDB 4.8, Copyright 1993 Free Software Foundation, Inc...
Core was generated by 'cross'.
Program terminated with signal 11, Segmentation fault.
#0  0x2494 in crossings (image=0xc7c8) at cross.c:31
31              if ((image[i][j] >= 0) &&
(gdb)

In this case, signal 11 was sent to the running cross process by the kernel when cross attempted to read or write to memory to which it did not have access. This signal caused cross to die and dump core. gdb says that the illegal memory reference occurred on line 31 of the source file cross.c:

(gdb) list  
26          xmax = imGetWidth(image)-1; 
27          ymax = imGetHeight(image)-1; 
28         
29          for (j=1; j<xmax; j++) { 
30            for (i=1; i<ymax; i++) { 
31              if ((image[i][j] >= 0) && 
32                  (image[i-1][j-1] < 0) || 
33                  (image[i-1][j] < 0) || 
34                  (image[i-1][j+1] < 0) || 
35                  (image[i][j-1] < 0) || 
(gdb)

Here, we see several things. First of all, there is a loop across the two index variables i and j, presumably in order to do calculations on the input image. Line 31 is an attempt to reference data from image[i][j], a two-dimensional array. When a program dumps core while attempting to access data from an array, it's usually a sign that one of the indices is out of bounds. Let's check them:

(gdb) print i 
$1 = 1 
(gdb) print j 
$2 = 1194 
(gdb) print xmax 
$3 = 1551 
(gdb) print ymax 
$4 = 1194 
(gdb)

Here we see the problem. The program was attempting to reference element image[1][1194], however, the array extends only to image[1550][1193] (remember that arrays in C are indexed from 0 to max-1). In other words, we attempted to read the 1195th row of an image that has only 1194 rows.

If we look at lines 29 and 30, we see the problem: the values xmax and ymax are reversed. The variable j should range from 1 to ymax (because it is the row index of the array), and i should range from 1 to xmax. Fixing the two for loops on lines 29 and 30 corrects the problem.

The call stack is the list of functions that led up to the current one. For example, if the program starts in function main, which calls function foo, which calls bamf, the call stack looks like this:

(gdb) backtrace 
#0  0x1384 in bamf ( ) at goop.c:31 
#1  0x4280 in foo ( ) at goop.c:48 
#2  0x218 in main ( ) at goop.c:116 
(gdb)

In the following example, we are looking at a core file for an X-based animation program. Using backtrace gives us:

(gdb) backtrace  
#0  0x602b4982 in _end ( ) 
#1  0xbffff934 in _end ( ) 
#2  0x13c6 in stream_drawimage (wgt=0x38330000, sn=4)\
at stream_display.c:94 
#3  0x1497 in stream_refresh_all ( ) at stream_display.c:116 
#4  0x49c in control_update_all ( ) at control_init.c:73 
#5  0x224 in play_timeout (Cannot access memory at address 0x602b7676. 
(gdb)

This is a list of stack frames for the process. The most recently called function is frame 0, which is the "function" _end in this case. Here, we see that play_timeout called control_update_all, which called stream_refresh_all, and so on. Somehow, the program jumped to _end where it crashed.

We can see, however, that the last "real" function called was stream_drawimage, and we might guess that it is the source of the problem. To examine the state of stream_drawimage, we need to select its stack frame (frame number 2), using the frame command:

(gdb) frame 2 
#2  0x13c6 in stream_drawimage (wgt=0x38330000, sn=4)\
at stream_display.c:94 
94        XCopyArea(mydisplay,streams[sn].frames[currentframe],\
XtWindow(wgt), 
(gdb) list 
91 
92        printf("CopyArea frame %d, sn %d, wid %d\n",currentframe,sn,wgt); 
93 
94        XCopyArea(mydisplay,streams[sn].frames[currentframe],\
XtWindow(wgt), 
95                 picGC,0,0,streams[sn].width,streams[sn].height,0,0); 
(gdb)

Well, not knowing anything else about the program at hand, we can't see anything wrong here, unless the variable sn (being used as an index into the array streams) is out of range. From the output of frame, we see that stream_drawimage was called with an sn parameter of 4. (Function parameters are displayed in the output of backtrace, as well as whenever we change frames.)

Let's move up another frame, to stream_refresh_all, to see how stream_display was called. To do this, we use the up command, which selects the stack frame above the current one:

(gdb) up  
#3  0x1497 in stream_refresh_all ( ) at stream_display.c:116 
116         stream_drawimage(streams[i].drawbox,i); 
(gdb) list 
113     void stream_refresh_all(void) { 
114       int i; 
115       for (i=0; i<=numstreams; i++) { 
116         stream_drawimage(streams[i].drawbox,i); 
117 
(gdb) print i 
$2 = 4 
(gdb) print numstreams 
$3 = 4 
(gdb)

Here, we see that the index variable i is looping from 0 to numstreams, and indeed i here is 4, the second parameter to stream_drawimage. However, numstreams is also 4. What's going on?

The for loop on line 115 looks funny; it should read:

for (i=0; i<numstreams; i++) {

The error is in the use of the <= comparison operator. The streams array is indexed from 0 to numstreams-1, not from 0 to numstreams. This simple off-by-one error caused the program to go berserk.

14.1.3. Debugging a Running Program

gdb can also debug a program that is already running, allowing you to interrupt it, examine it, and then return the process to its regularly scheduled execution. This is very similar to running a program from within gdb, and there are only a few new commands to learn.

For example, if you have started the program pgmseq with process ID 254, you can start up gdb with:

papaya$ gdb pgmseq

and once inside gdb, use the command:

(gdb) attach 254 
Attaching program `/home/loomer/mdw/pgmseq/pgmseq', pid 254 
_ _select (nd=4, in=0xbffff96c, out=0xbffff94c, ex=0xbffff92c, tv=0x0) 
    at _ _select.c:22 
_ _select.c:22: No such file or directory. 
(gdb)

You can also start gdb with the command:

papaya$ gdb pgmseq 254

Once gdb attaches to the running process, it temporarily suspends the program and lets you take over, issuing gdb commands. Or you can set a breakpoint or watchpoint (with the break and watch commands) and use continue to cause the program to continue execution until the breakpoint is triggered.

In fact, gdb allows you to debug three programs concurrently: one running directly under gdb, one tracing with a core file, and one running as an independent process. The target command allows you to select which one you wish to debug.

14.1.4. Changing and Examining Data

(gdb) print mydisplay 
$10 = (struct _XDisplay *) 0x9c800 
(gdb)

This displays the value of the variable mydisplay, as well as an indication of its type. Because this variable is a pointer, you can examine its contents by dereferencing the pointer, as you would in C:

(gdb) print *mydisplay  
$11 = {ext_data = 0x0, free_funcs = 0x99c20, fd = 5, lock = 0,  
  proto_major_version = 11, proto_minor_version = 0,  
  vendor = 0x9dff0 "XFree86", resource_base = 41943040,  
  ... 
  error_vec = 0x0, cms = {defaultCCCs = 0xa3d80 "",\
clientCmaps = 0x991a0 "'",  
    perVisualIntensityMaps = 0x0}, conn_checker = 0, im_filters = 0x0} 
(gdb)

mydisplay is an extensive structure used by X programs; we have abbreviated the output for your reading enjoyment.

print can print the value of just about any expression, including C function calls (which it executes on the fly, within the context of the running program):

(gdb) print getpid( ) 
$11 = 138 
(gdb)

Of course, not all functions may be called in this manner. Only those functions that have been linked to the running program may be called. If a function has not been linked to the program and you attempt to call it, gdb will complain that there is no such symbol in the current context.

More complicated expressions may be used as arguments to print as well, including assignments to variables. For example:

(gdb) print mydisplay->vendor = "Linux" 
$19 = 0x9de70 "Linux" 
(gdb)

assigns to the vendor member of the mydisplay structure the value "Linux" instead of "XFree86" (a useless modification, but interesting nonetheless). In this way, you can interactively change data in a running program to correct errant behavior or test uncommon situations.

Note that after each print command, the value displayed is assigned to one of the gdb convenience registers, which are gdb internal variables that may be handy for you to use. For example, to recall the value of mydisplay in the previous example, we need to merely print the value of $10:

(gdb) print $10 
$21 = (struct _XDisplay *) 0x9c800 
(gdb)

You may also use expressions, such as typecasts, with the print command. Almost anything goes.

(gdb) ptype mydisplay  
type = struct _XDisplay { 
    struct _XExtData *ext_data; 
    struct _XFreeFuncs *free_funcs; 
    int fd; 
    int lock; 
    int proto_major_version; 
    ... 
    struct _XIMFilter *im_filters; 
} * 
(gdb)

x also takes a count and a type specification as an optional argument. The count is the number of objects of the given type to display. For example, x/100x 0x4200 displays 100 bytes of data, represented in hexadecimal format, at the address 0x4200. Use help x to get a description of the various output formats.

To examine the value of mydisplay->vendor, we can use:

(gdb) x mydisplay->vendor 
0x9de70 <_end+35376>:   76 'L'  
(gdb) x/6c mydisplay->vendor 
0x9de70 <_end+35376>:   76 'L'  105 'i' 110 'n' 117 'u' 120 'x' 0 '\000' 
(gdb) x/s mydisplay->vendor 
0x9de70 <_end+35376>:    "Linux"  
(gdb)

The first field of each line gives the absolute address of the data. The second represents the address as some symbol (in this case, _end) plus an offset in bytes. The remaining fields give the actual value of memory at that address, first in decimal, then as an ASCII character. As described earlier you can force x to print the data in other formats.

14.1.5. Getting Information

The info command provides information about the status of the program being debugged. There are many subcommands under info; use help info to see them all. For example, info program displays the execution status of the program:

(gdb) info program  
Using the running image of child process 138. 
Program stopped at 0x9e. 
It stopped at breakpoint 1. 
(gdb)

Another useful command is info locals, which displays the names and values of all local variables in the current function:

(gdb) info locals 
inimage = (struct {...} *) 0x2000 
outimage = (struct {...} *) 0x8000 
(gdb)

This is a rather cursory description of the variables. The print or x commands describe them further.

(gdb) info address inimage 
Symbol "inimage" is a local variable at frame offset -20. 
(gdb)

By frame offset, gdb means that inimage is stored 20 bytes below the top of the stack frame.

You can get information on the current frame using the info frame command, as so:

(gdb) info frame  
Stack level 0, frame at 0xbffffaa8: 
 eip = 0x9e in main (main.c:44); saved eip 0x34 
 source language c. 
 Arglist at 0xbffffaa8, args: argc=1, argv=0xbffffabc 
 Locals at 0xbffffaa8, Previous frame's sp is 0x0 

 Saved registers: 
  ebx at 0xbffffaa0, ebp at 0xbffffaa8, esi at 0xbffffaa4, eip at\
0xbffffaac 
(gdb)

14.1.6. Miscellaneous Features

We have barely scratched the surface of what gdb can do. It is an amazing program with a lot of power; we have introduced you only to the most commonly used commands. In this section, we'll look at other features of gdb and then send you on your way.