@c -*-texinfo-*-
@setfilename ../info/debugging
@node Debugging, Streams, Byte Compilation, Top
@chapter Debugging Lisp Programs

  There are three ways to investigate a problem in an Emacs Lisp program,
depending on what you are doing with the program when the problem appears.

@itemize @bullet
@item
If the problem occurs when you run the program, you can use
the Lisp debugger to investigate what is happening during execution.

@item
If the problem is syntactic, so that Lisp cannot even read the program,
you can use the Emacs facilities for editing Lisp to localize it.

@item
If the problem occurs when trying to compile the program with the byte
compiler, you need to know how to examine the compiler's input buffer.
@end itemize

@menu
* Debugger::            How the Emacs Lisp debugger is implemented.
* Syntax Errors::       How to find syntax errors.
* Compilation Errors::  How to find errors that show up in byte compilation.
@end menu

  Another useful debugging tool is a dribble file.  When a dribble file
is open, Emacs copies all keyboard input characters to that file.
Afterward, you can examine the file to find out what input was used.
@xref{Terminal Input}.

  For debugging problems in terminal descriptions, the
@code{open-termscript} function can be useful.  @xref{Terminal Output}.

@node Debugger, Syntax Errors, Debugging, Debugging
@section The Lisp Debugger
@cindex debugger
@cindex Lisp debugger
@cindex break

  The @dfn{Lisp debugger} provides you with the ability to suspend
evaluation of a form.  While evaluation is suspended (a state that is
commonly known as a @dfn{break}), you may examine the run time stack,
examine the values of local or global variables, or change those values.
Since a break is a recursive edit, all the usual editing facilities of
Emacs are available; you can even run programs that will enter the
debugger recursively.  @xref{Recursive Editing}.

@menu
* Error Debugging::       Entering the debugger when an error happens.
* Infinite Loops::	  Stopping and debugging a program that doesn't exit.
* Function Debugging::    Entering it when a certain function is called.
* Explicit Debug::        Entering it at a certain point in the program.
* Using Debugger::        What the debugger does; what you see while in it.
* Debugger Commands::     Commands used while in the debugger.
* Invoking the Debugger:: How to call the function @code{debug}.
* Internals of Debugger:: Subroutines of the debugger, and global variables.
@end menu

@node Error Debugging, Infinite Loops, Debugger, Debugger
@subsection Entering the Debugger When an Error Occurs
@cindex error debugging
@cindex debugging errors

  The most important time to enter the debugger is when a Lisp error
happens.  This allows you to investigate the immediate causes of the
error.

  However, entry to the debugger is not a normal consequence of an
error.  Many commands frequently get Lisp errors when invoked in
inappropriate contexts (such as @kbd{C-f} at the end of the buffer) and
during ordinary editing it would be very unpleasant to enter the
debugger each time this happens.  If you want errors to enter the
debugger, set the variable @code{debug-on-error} to non-@code{nil}.

@defopt debug-on-error
This variable determines whether the debugger is called when a error is
signaled and not handled.  If @code{debug-on-error} is non-@code{nil},
then the debugger is called when an error happens.  Otherwise the
debugger is not called for errors.
@end defopt

@node Infinite Loops, Function Debugging, Error Debugging, Debugger
@section Debugging Infinite Loops
@cindex infinite loops
@cindex loops, infinite
@cindex quitting from infinite loop
@cindex stopping an infinite loop

  When a program loops infinitely and fails to return, your first
problem is to stop the loop.  On most operating systems, you can do this
with @kbd{C-g}, which causes quit.  This works if you are not using X
windows, and on Berkeley systems even if you are using X windows.

  On other inferior operating systems, @kbd{C-g} does not work when
using X windows.  This is because these systems do not allow Emacs to
request a signal when input arrives from the X server.  There is nothing
Emacs can do about this.

  However, you can still stop a loop by sending Emacs the @code{SIGINT}
signal.  To do this, go to a shell in another window, use @code{ps} to
find out the @sc{pid} of the Emacs process, and then type @samp{kill
-INT @var{pid}}.

  Ordinary quitting gives no information about why the program was
looping.  To get more information, you can set the variable
@code{debug-on-quit} to non-@code{nil}.  Quitting with @kbd{C-g} is not
considered an error, and @code{debug-on-error} has no effect on the
handling of @kbd{C-g}.  @code{debug-on-quit} has no effect on
errors.@refill

  Once you have the debugger running in the middle of the infinite loop,
you can proceed from the debugger using the stepping commands.  If you
step through the entire loop, you will probably get enough information
to solve the problem.

@defopt debug-on-quit
This variable determines whether the debugger is called when @code{quit}
is signaled and not handled.  If @code{debug-on-quit} is non-@code{nil},
then the debugger is called whenever you quit (that is, type @kbd{C-g}).
If @code{debug-on-quit} is @code{nil}, then the debugger is not called
when you quit.  @xref{Quitting}.
@end defopt

@node Function Debugging, Explicit Debug, Infinite Loops, Debugger
@subsection Entering the Debugger when Some Function is Called
@cindex function call debugging
@cindex debugging specific functions

  To investigate a problem that happens in the middle of a program, one
useful technique is to cause the debugger to be entered when a certain
function is called.  You can do this to the function in which the
problem occurs, and then step through the function, or you can do this
to a function called shortly before the problem, step quickly over
the call to that function, and then step through its caller.

@deffn Command debug-on-entry function-name
  This function requests @var{function-name} to invoke the debugger each time
it is called.  It works by inserting the form @code{(debug 'debug)} into
the function definition as the first form.

  Any function defined as Lisp code may be set to break on entry,
regardless of whether it is interpreted code or compiled code.  Even
functions that are commands may be debugged---they will enter the
debugger when called inside a function, or when called interactively.
Primitive functions (i.e., those written in C) may not be debugged.

  When @code{debug-on-entry} is called interactively, it prompts
for @var{function-name} in the minibuffer.

  If @code{debug-on-entry} is called more than once on the same
function, the second call does nothing.  @code{debug-on-entry} returns
@var{function-name}.

@example
(defun fact (n)
  (if (zerop n) 1
      (* n (fact (1- n)))))
     @result{} fact
(debug-on-entry 'fact)
     @result{} fact
(fact 3)
     @result{} 6

---------- Buffer: *Backtrace* ----------
Entering:
* fact(3)
  eval-region(4870 4878 t)
  byte-code("...")
  eval-last-sexp(nil)
  (let ...)
  eval-insert-last-sexp(nil)
* call-interactively(eval-insert-last-sexp)
---------- Buffer: *Backtrace* ----------

(symbol-function 'fact)
     @result{} (lambda (n)
          (debug (quote debug))
          (if (zerop n) 1 (* n (fact (1- n)))))
@end example
@end deffn

@deffn Command cancel-debug-on-entry function-name
This function undoes the effect of @code{debug-on-entry} on
@var{function-name}.  When called interactively, it prompts for
@var{function-name} in the minibuffer.

If @code{cancel-debug-on-entry} is called more than once on the same
function, the second call does nothing.  @code{cancel-debug-on-entry}
returns @var{function-name}.
@end deffn

@node Explicit Debug, Using Debugger, Function Debugging, Debugger
@subsection Explicit Entry to the Debugger

  You can cause the debugger to be called at a certain point in your
program by writing the expression @code{(debug)} at that point.  To do
this, visit the source file, insert the text @samp{(debug)} at the
proper place, and type @kbd{C-M-x}.  Be sure to undo this insertion
before you save the file!

  The place where you insert @samp{(debug)} must be a place where an
additional form can be evaluated and its value ignored.  (If the value
isn't ignored, it will alter the execution of the program!)  Usually
this means inside a @code{progn} or an implicit @code{progn}
(@pxref{Sequencing}).

@node Using Debugger, Debugger Commands, Explicit Debug, Debugger
@subsection Using the Debugger

  When the debugger is entered, it displays the previously selected
buffer in one window and a buffer named @samp{*Backtrace*} in another
window.  The backtrace buffer contains one line for each level of Lisp
function execution currently going on.  At the beginning of this buffer
is a message describing the reason that the debugger was invoked (such
as the error message and associated data, if it was invoked due to an
error).

  The backtrace buffer is read-only and uses a special major mode,
Debugger mode, in which letters are defined as debugger commands.  The
usual Emacs editing commands are available; thus, you can switch windows
to examine the buffer that was being edited at the time of the error,
switch buffers, visit files, or do any other sort of editing.  However,
the debugger is a recursive editing level (@pxref{Recursive Editing})
and it is wise to go back to the backtrace buffer and exit the debugger
(with the @kbd{q} command) when you you are finished with it.  Exiting
the debugger gets out of the recursive edit and kills the backtrace
buffer.

@cindex current stack frame
  The contents of the backtrace buffer show you the functions that are
executing and the arguments that were given to them.  It also allows
you to specify a stack frame by moving point to the line describing
that frame.  (A stack frame is the place where the Lisp interpreter
records information about a particular invocation of a function.  The
frame whose line point is on is considered the @dfn{current frame}.)
Some of the debugger commands operate on the current frame.

  The debugger itself should always be run byte-compiled, since it
makes assumptions about how many stack frames are used for the
debugger itself.  These assumptions are false if the debugger is
running interpreted.

@node Debugger Commands, Invoking the Debugger, Using Debugger, Debugger
@subsection Debugger Commands
@cindex debugger command list

  Inside the debugger (in Debugger mode), these special commands are
available in addition to the usual cursor motion commands.  (Keep in
mind that all the usual facilities of Emacs, such as switching windows
or buffers, are still available.)

  The most important use of debugger commands is for stepping through
code, so that you can see how control flows.  The debugger can step
through the control structures of an interpreted function, but cannot do
so in a byte-compiled function.  If you would like to step through a
byte-compiled function, replace it with an interpreted definition of the
same function.  (To do this, visit the source file for the function and
type @kbd{C-M-x} on its definition.)

@table @kbd
@item c
Exit the debugger and continue execution.  When continuing is possible,
it resumes execution of the program as if the debugger had never been
entered (aside from the effect of any variables or data structures you
may have changed while inside the debugger).

Continuing is possible after entry to the debugger due to function entry
or exit, explicit invocation, quitting or certain errors.  Most errors
cannot be continued; trying to continue an unsuitable error causes the
same error to occur again.

@item d
Continue execution, but enter the debugger the next time any Lisp
function is called.  This allows you to step through the
subexpressions of an expression, seeing what values the subexpressions
compute, and what else they do.

The stack frame made for the function call which enters the debugger in
this way will be flagged automatically so that the debugger will be
called again when the frame is exited.  You can use the @kbd{u} command
to cancel this flag.

@item b
Flag the current frame so that the debugger will be entered when the
frame is exited.  Frames flagged in this way are marked with stars
in the backtrace buffer.

@item u
Don't enter the debugger when the current frame is exited.  This
cancels a @kbd{b} command on that frame.

@item e
Read a Lisp expression in the minibuffer, evaluate it, and print the
value in the echo area.  This is the same as the command @kbd{M-@key{ESC}},
except that @kbd{e} is not normally disabled like @kbd{M-@key{ESC}}.

@item q
Terminate the program being debugged; return to top-level Emacs
command execution.

If the debugger was entered due to a @kbd{C-g} but you really want
to quit, and not debug, use the @kbd{q} command.

@item r
Return a value from the debugger.  The value is computed by reading an
expression with the minibuffer and evaluating it.

The @kbd{r} command makes a difference when the debugger was invoked due
to exit from a Lisp call frame (as requested with @kbd{b}); then the
value specified in the @kbd{r} command is used as the value of that
frame.

The @kbd{r} also matters in certain cases of errors.  For example,
@code{wrong-type-argument} errors will use the debugger's return value
instead of the invalid argument; @code{no-catch} errors will use the
debugger value as a throw tag instead of the tag that was not found.
If an error was signaled by calling the Lisp function @code{signal},
the debugger's return value is returned as the value of @code{signal}.
@end table

@node Invoking the Debugger, Internals of Debugger, Debugger Commands, Debugger
@subsection Invoking the Debugger

  Here we describe fully the function used to invoke the debugger.

@defun debug &rest debugger-args
  This function enters the debugger.  It switches buffers to a buffer
named @samp{*Backtrace*} (or @samp{*Backtrace*<2>} if it is the second
recursive entry to the debugger, etc.), and fills it with information
about the stack of Lisp function calls.  It then enters a recursive
edit, leaving that buffer in Debugger mode and displayed in the selected
window.

  Debugger mode provides a @kbd{c} command which operates by exiting the
recursive edit, switching back to the previous buffer, and returning to
whatever called @code{debug}.  The @kbd{r} command also returns from
@code{debug}.  These are the only ways the function @code{debug} can
return to its caller.

  If the first of the @var{debugger-args} passed to @code{debug} is
@code{nil} (or if it is not one of the following special values), then
the rest of the arguments to @code{debug} are printed at the top of the
@samp{*Backtrace*} buffer.  This mechanism is used to display a message
to the user.

  However, if the first argument passed to @code{debug} is one of the
following special values, then it has special significance.  Normally,
these values are passed to @code{debug} only by the internals of Emacs
and the debugger, and not by programmers calling @code{debug}.

  The special values are:

@table @code
@item lambda
@cindex @code{lambda} in debug
  When the first argument is @code{lambda}, the debugger displays
@samp{Entering:} as a line of text at the top of the buffer.  This means
that a function is being entered when @code{debug-on-next-call} is
non-@code{nil}.

@item debug
  When the first argument is @code{debug}, the debugger displays
@samp{Entering:} just as in the @code{lambda} case.  However,
@code{debug} as the argument indicates that the reason for entering the
debugger is that a function set to debug on entry is being entered.

  In addition, @code{debug} as the first argument directs the debugger
to mark the function that called @code{debug} so that it will invoke the
debugger when exited.  (When @code{lambda} is the first argument, the
debugger does not do this, because it has already been done by the
interpreter.)

@item t
  When the first argument is @code{t}, the debugger displays
@samp{Beginning evaluation of function call form:} as the top line in
the buffer, to indicate that it was entered due to the evaluation of a
list form at a time when @code{debug-on-next-call} is non-@code{nil}.

@item exit
  When the first argument is @code{exit}, it indicates the exit of a stack
frame previously marked to invoke the debugger on exit.  The debugger
displays @samp{Return value:} on the top line of the buffer, followed by
the value being returned from the frame.

@item error
@cindex @code{error} in debug
  When the first argument is @code{error}, the debugger indicates that
it is being entered because an error or @code{quit} was signaled and not
handled, by displaying @samp{Signaling:} followed by the error signaled
and any arguments to @code{signal}.  For example,

@example
(let ((debug-on-error t))
     (/ 1 0))

---------- Buffer: *Backtrace* ----------
Signaling: (arith-error)
  /(1 0)
...
@end example

  If an error was signaled, presumably the variable
@code{debug-on-error} is non-@code{nil}.  If @code{quit} was signaled,
then presumably the variable @code{debug-on-quit} is non-@code{nil}.

@item nil
  Use @code{nil} as the first of the @var{debugger-args} when you want
to enter the debugger explicitly.  The rest of the @var{debugger-args}
are printed on the top line of the buffer.  You can use this feature to
display messages---for example, to remind yourself of the conditions
under which @code{debug} is called.
@end table
@end defun

@node Internals of Debugger,, Invoking the Debugger, Debugger
@subsection Internals of the Debugger

  This section describes functions and variables used internally by the
debugger.

@defvar debugger
  The value of this variable is the function to call to invoke the
debugger.  Its value must be a function of any number of arguments (or,
more typically, the name of a function).  Presumably this function will
enter some kind of debugger.  The default value of the variable is
@code{debug}.

  The first argument that Lisp hands to the function indicates why it
was called.  The convention for arguments is detailed in the description
of @code{debug}.
@end defvar

@deffn Command backtrace
@cindex run time stack
@cindex call stack
  This function prints a trace of Lisp function calls currently active.
This is the function used by @code{debug} to fill up the
@samp{*Backtrace*} buffer.  It is written in C, since it must have access
to the stack to determine which function calls are active.  The return
value is always @code{nil}.

  In the following example, @code{backtrace} is called explicitly in a
Lisp expression.  When the expression is evaluated, the backtrace is
printed to the stream @code{standard-output}: in this case, to the
buffer @samp{backtrace-output}.  Each line of the backtrace represents
one function call.  If the arguments of the function call are all known,
they are displayed; if they are being computed, that fact is stated.
The arguments of special forms are elided.

@example
(with-output-to-temp-buffer "backtrace-output"
  (let ((var 1))
    (save-excursion
      (setq var (eval '(progn
                         (1+ var)
                         (list 'testing (backtrace))))))))

     @result{} nil
----------- Buffer: backtrace-output ------------
  backtrace()
  (list ...computing arguments...)
  (progn ...)
  eval((progn (1+ var) (list (quote testing) (backtrace))))
  (setq ...)
  (save-excursion ...)
  (let ...)
  (with-output-to-temp-buffer ...)
  eval-region(1973 2142 #<buffer *scratch*>)
  byte-code("...  for eval-print-last-sexp ...")
  eval-print-last-sexp(nil)
* call-interactively(eval-print-last-sexp)
----------- Buffer: backtrace-output ------------
@end example
@end deffn

@defopt stack-trace-on-error
@cindex stack trace
  This variable controls whether Lisp automatically displays a
backtrace buffer after every error that is not handled.  A quit signal
counts as an error for this variable.  If it is non-@code{nil} then a
backtrace is shown in a pop-up buffer named @samp{*Backtrace*} on every
error.  If it is @code{nil}, then a backtrace is not shown.

  When a backtrace is shown, that buffer is not selected.  If either
@code{debug-on-quit} or @code{debug-on-error} is also non-@code{nil}, then
a backtrace is shown in one buffer, and the debugger is popped up in
another buffer with its own backtrace.

  We consider this feature to be obsolete and superseded by the debugger
itself.
@end defopt

@defvar debug-on-next-call
@cindex @code{eval}, and debugging
@cindex @code{apply}, and debugging
@cindex @code{funcall}, and debugging
This variable determines whether the debugger is called before the
next @code{eval}, @code{apply} or @code{funcall}.  It is automatically
reset to @code{nil} when the debugger is entered.  

The @kbd{d} command in the debugger works by setting this variable.
@end defvar

@defun backtrace-debug level flag
  This function sets the debug-on-exit flag of the eval frame
@var{level} levels down to @var{flag}.  If @var{flag} is non-@code{nil},
this will cause the debugger to be entered when that frame exits.

  The debug-on-exit flag is an entry in the stack frame of a
function call.  This flag is examined on every exit from a function.

  Normally, this function is only called by the debugger.
@end defun

@node Syntax Errors, Compilation Errors, Debugger, Debugging
@section Debugging Invalid Lisp Syntax

  It is easy to make a syntax error in an Emacs Lisp program by omitting
a parenthesis.  The Lisp reader will detect an error, but cannot say
where the real problem is.  For example, if a close parenthesis is
omitted, the reader will detect an imbalance at the end of the file, but
it cannot tell anything about where the close parenthesis should have
been.  However, you can use the following techniques to figure out
where.

  If the problem is not simply an imbalance of parentheses, a useful
technique is to try @kbd{C-M-e} at the beginning of each defun, and see
if it goes to the place where that defun appears to end.  If it does
not, there is a problem in that defun.

  However, unmatched parentheses are the most common syntax errors in
Lisp, and we can give further advice for those cases.

@menu
* Excess Open::     How to find a spurious open paren or missing close.
* Excess Close::    How to find a spurious close paren or missing open.
@end menu

@node Excess Open, Excess Close, Syntax Errors, Syntax Errors
@subsection Excess Open Parentheses

  The first step is to find the defun that is unbalanced.  If there is
an excess open parenthesis, the way to do this is to insert a
close parenthesis at the end of the file and type @kbd{C-M-b}
(@code{backward-sexp}).  This will move you to the beginning of the
defun that is unbalanced.  (Then type @kbd{C-@key{SPC} C-_ C-u
C-@key{SPC}} to set the mark there, undo the insertion of the
close parenthesis, and finally return to the mark.)

  The next step is to determine precisely what is wrong.  There is no
way to be sure of this except to study the program, but often the
existing indentation is a clue to where the parentheses should have
been.  The easiest way to use this clue is to reindent with @kbd{C-M-q}
and see what moves.

  Before you do this, make sure the defun has enough close parentheses.
Otherwise, @kbd{C-M-q} will get an error, or will reindent all the rest
of the file until the end.  So move to the end of the defun and insert a
close parenthesis there.  Don't use @kbd{C-M-e} to move there, since
that too will fail to work until the defun is balanced.

  Then go to the beginning of the defun and type @kbd{C-M-q}.  Usually
all the lines from a certain point to the end of the function will shift
to the right.  There is probably a missing close parenthesis, or a
superfluous open parenthesis, near that point.  (However, don't assume
this is true; study the code to make sure.)  Once you have found the
discrepancy, undo the @kbd{C-M-q}, since the old indentation is probably
appropriate to the intended parentheses.

  After you think you have fixed the problem, use @kbd{C-M-q} again.  It
should not change anything, if the problem is really fixed.

@node Excess Close,, Excess Open, Syntax Errors
@subsection Excess Close Parentheses

  To deal with an excess close parenthesis, first insert an
open parenthesis at the beginning of the file and type @kbd{C-M-f} to
find the end of the unbalanced defun.  (Then type @kbd{C-@key{SPC} C-_
C-u C-@key{SPC}} to set the mark there, undo the insertion of the
open parenthesis, and finally return to the mark.)

  Then find the actual matching close parenthesis by typing @kbd{C-M-f}
at the beginning of the defun.  This will leave you somewhere short of
the place where the defun ought to end.  It is possible that you will
find a spurious close parenthesis in that vicinity.

  If you don't see a problem at that point, the next thing to do is to
type @kbd{C-M-q} at the beginning of the defun.  A range of lines will
probably shift left; if so, the missing open parenthesis or spurious
close parenthesis is probably near the first of those lines.  (However,
don't assume this is true; study the code to make sure.)  Once you have
found the discrepancy, undo the @kbd{C-M-q}, since the old indentation
is probably appropriate to the intended parentheses.

@node Compilation Errors,, Syntax Errors, Debugging
@section Debugging Problems in Compilation

  When an error happens during byte compilation, it is normally due to
an error in the program you are compiling.  The compiler itself can't tell
you where in the file the error occurred, so here is how to find out.

  What you should do is switch to the buffer @w{@samp{ *Compiler Input*}}.
(Note that the buffer name starts with a space, so it will not show
up in @kbd{M-x list-buffers}.)  This buffer contains the program being
compiled, and point shows how far the byte compiler was able to read.

  If the error was due to invalid Lisp syntax, point shows exactly where
the invalid syntax was @emph{detected}.  The cause of the error is not
necessarily near by!  Use the techniques in the previous section to find
the error.

  If the error was detected while compiling a form that had been read
successfully, then point is located at the end of the form.  In this
case, it can't localize the error precisely, but can still show you
which function to check.
