Assorted tools of varying usefulness exist to debug the kernel. By far
the best debugging tool is the human brain. As Linus has said:

...
I'm afraid that I've seen too many people fix bugs
by looking at debugger output, and that almost
inevitably leads to fixing the symptoms rather than
the underlying problems.
...
"Use the Source, Luke, use the Source. Be one with
the code.". Think of Luke Skywalker discarding the
automatic firing system when closing on the deathstar,
and firing the proton torpedo (or whatever) manually.
_Then_ do you have the right mindset for fixing kernel
bugs.
...

Having said that, sometimes reading the source is not enough. The
following tools exist in the IKD patch:

Debug kernel stack overflows
Detect software lockups
Kernel tracer (show logic flow through procedures)
Memleak detector

Written by Ingo Molnar <mingo@pc7537.hil.siemens.at>. Currently
maintained by Mike Galbraith <mikeg@weiden.de>.

Print-EIP on video ram

Improved by Andrea Arcangeli.

Kernel stack meter
Kernel real profiling
Semaphore deadlock detector

Developed by Andrea Arcangeli.

NMI oopser

Written by Ingo Molnar.
Integration into IKD and fixes for newer kernels
by Andrea Arcangeli.

kdb
Written by Scott Lurndal (SGI)
Integration into IKD by Andrea Arcangeli.

The original merge of debugging tools into a single patch set (IKD)
is been done by Keith Owens <kaos@ocs.com.au>.
PGP 917/C817FEC9.
Fingerprint 2B 25 0A 31 02 AE CA F7 73 0C 28 69 4A 7B 65 27

Currently the IKD patch is maintained by Andrea Arcangeli <andrea@suse.de>
and is dowloadable at:

ftp://ftp.*.kernel.org/pub/linux/kernel/people/andrea/ikd/