backtrace - how do I get local and formal parametes - kernel is compiled -O0.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Got by 1st panic crash, crash seems to be working but
I don't feel comfortable as with gdb. I can't see
the parameters on the stack as variables as with
kgdb nor can I see the local variables on the stack.
Seems I should be able to get crash working with
gdb so I can see the details that gdb can see.

I compiled the kernel -O0 and without the framepointer
as it seem crash prefers (rather odd). But want to
see the locals and formals on the stack and walk up and
down the stack and have the context know to crash just
as with gdb.

I'll re-read your crash paper.

Any suggestions?

- -piet

- -------------------------------------------------
crash> bt -l
PID: 1478   TASK: c7ed4570  CPU: 0   COMMAND:
"savproxy"
~ #0 [d76bbbb4] crash_kexec at c01ecb41
~    include/asm/system.h: 259
~ #1 [d76bbc24] panic at c018bc33
~    /nethome/piet/src/India-2.6.16.57/linux/kernel/panic.c:
89
~ #2 [d76bbc44] skb_tcp_zero_copy_iovec at c06d67a0
~   
/nethome/piet/src/India-2.6.16.57/linux/net/core/skbuff.c:
517
~ #3 [d76bbcb4] tcp_zerocopy_recvmsg at c07e6c14
~    /nethome/piet/src/India-2.6.16.57/linux/net/ipv4/tcp.c:
1792
~ #4 [d76bbd8c] receive_tcp_zero_copy_buffers at c06beb89
~    /nethome/piet/src/India-2.6.16.57/linux/net/socket.c:
3573
~ #5 [d76bbf2c] sys_zcopy_sockop at c06bc077
~    /nethome/piet/src/India-2.6.16.57/linux/net/socket.c:
2536
~ #6 [d76bbfb8] sysenter_entry at c0981544
~    include/asm/system.h: 279
~    EAX: 000000bd  EBX: 00000005  ECX: 00000006  EDX:
08228cd8
~    DS:  007b      ESI: 00000100  ES:  007b      EDI:
00000001
~    SS:  007b      ESP: b59767d0  EBP: b5976828
~    CS:  0073      EIP: ffffe410  ERR: 000000bd  EFLAGS:
00000246
crash> bt
PID: 1478   TASK: c7ed4570  CPU: 0   COMMAND:
"savproxy"
~ #0 [d76bbbb4] crash_kexec at c01ecb41
~ #1 [d76bbc24] panic at c018bc33
~ #2 [d76bbc44] skb_tcp_zero_copy_iovec at c06d67a0
~ #3 [d76bbcb4] tcp_zerocopy_recvmsg at c07e6c14
~ #4 [d76bbd8c] receive_tcp_zero_copy_buffers at c06beb89
~ #5 [d76bbf2c] sys_zcopy_sockop at c06bc077
~ #6 [d76bbfb8] sysenter_entry at c0981544
~    EAX: 000000bd  EBX: 00000005  ECX: 00000006  EDX:
08228cd8
~    DS:  007b      ESI: 00000100  ES:  007b      EDI:
00000001
~    SS:  007b      ESP: b59767d0  EBP: b5976828
~    CS:  0073      EIP: ffffe410  ERR: 000000bd  EFLAGS:
00000246
crash>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFH0g9fJICwm/rv3hoRAhI3AJ9veuvkY6Nfrycz+xoGwGaTNyijwQCf
RzdQ
TKiaPBx3+B4XNCa0n/mfUCc=
=/Pcx
-----END PGP SIGNATURE-----

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Pete/Piet Delaney wrote:
> Got by 1st panic crash, crash seems to be working but
> I don't feel comfortable as with gdb. I can't see
> the parameters on the stack as variables as with
> kgdb nor can I see the local variables on the stack.
> Seems I should be able to get crash working with
> gdb so I can see the details that gdb can see.
> 
> I compiled the kernel -O0 and without the framepointer
> as it seem crash prefers (rather odd). But want to
> see the locals and formals on the stack and walk up
and
> down the stack and have the context know to crash just
> as with gdb.
> 
> I'll re-read your crash paper.
> 
> Any suggestions?

Nope, other than using "bt -f" to dump each
frame's data,
and figuring it out from there...

Dave

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Anderson wrote:
| Pete/Piet Delaney wrote:
|> Got by 1st panic crash, crash seems to be working but
|> I don't feel comfortable as with gdb. I can't see
|> the parameters on the stack as variables as with
|> kgdb nor can I see the local variables on the stack.
|> Seems I should be able to get crash working with
|> gdb so I can see the details that gdb can see.
|>
|> I compiled the kernel -O0 and without the
framepointer
|> as it seem crash prefers (rather odd). But want to
|> see the locals and formals on the stack and walk up
and
|> down the stack and have the context know to crash
just
|> as with gdb.
|>
|> I'll re-read your crash paper.
|>
|> Any suggestions?
|
| Nope, other than using "bt -f" to dump each
frame's data,
| and figuring it out from there...

When I was using gdb on SunOS 4.1.4 I was able to set $sp
and
$fp to switch processes. When I took the SP and FP and used
a gdb set $sp and %fp it didn't seen to allow me to do
this:
-
-----------------------------------------------------------
crash> gdb set $sp = 0xd76bbbb4
No registers.

crash> gdb set $fp = 0xd76bbc24
No registers.
-
-----------------------------------------------------------
Why is is saying 'No registers'?


Also why don't you set up gdb with the stack info for the
current
task and either allow the user to do a 'gdb bt' or do it
with a
crash 'bt' option?

Will gdb work directly with the kdump core file? If not when
did
it break and why? Docs with 2.6.16 suggest to use gdb of the
core
file.

I recall the crash backtrack proving the gdb info back a few
years
ago when I worked on LKCD and crash with you. Is my memory
wrong
and this visibility of the formals and local variables has
never
never been possible? It's hard to recall for sure with my
using
kgdb on the kernel now for so long.

- -piet


|
| Dave
|
|

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFH1ZC9JICwm/rv3hoRArUYAJ4lJAwjGqd1LkwnHZpdWnxUepxdWQCf
buTB
wYelbR3Ffwj6byka/3VQQ5E=
=I0YU
-----END PGP SIGNATURE-----

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Pete/Piet Delaney wrote:
 > -----BEGIN PGP SIGNED MESSAGE-----
 > Hash: SHA1
 >
 > Dave Anderson wrote:
 > | Pete/Piet Delaney wrote:
 > |> Got by 1st panic crash, crash seems to be
working but
 > |> I don't feel comfortable as with gdb. I can't
see
 > |> the parameters on the stack as variables as
with
 > |> kgdb nor can I see the local variables on the
stack.
 > |> Seems I should be able to get crash working
with
 > |> gdb so I can see the details that gdb can see.
 > |>
 > |> I compiled the kernel -O0 and without the
framepointer
 > |> as it seem crash prefers (rather odd). But want
to
 > |> see the locals and formals on the stack and walk
up and
 > |> down the stack and have the context know to
crash just
 > |> as with gdb.
 > |>
 > |> I'll re-read your crash paper.
 > |>
 > |> Any suggestions?
 > |
 > | Nope, other than using "bt -f" to dump
each frame's data,
 > | and figuring it out from there...
 >
 > When I was using gdb on SunOS 4.1.4 I was able to set
$sp and
 > $fp to switch processes. When I took the SP and FP and
used
 > a gdb set $sp and %fp it didn't seen to allow me to do
this:
 > -
-----------------------------------------------------------
 > crash> gdb set $sp = 0xd76bbbb4
 > No registers.
 >
 > crash> gdb set $fp = 0xd76bbc24
 > No registers.
 > -
-----------------------------------------------------------
 > Why is is saying 'No registers'?

I'm not positive, but the gdb sources seem to display that
message
whenever (!target_has_registers), and that's #define'd like
so:

   /* Does the target have registers?  (Exec files don't.) 
*/

   #define target_has_registers    
        (current_target.to_has_registers)

And since it's being invoked as "gdb vmlinux", it
only knows
about the exec file, and has no clue about registers.

 >
 >
 > Also why don't you set up gdb with the stack info for
the current
 > task and either allow the user to do a 'gdb bt' or do
it with a
 > crash 'bt' option?
 >

I don't even come close to understanding gdb internals well
enough
to know what "set up gdb with the stack info"
entails.  Send me
a patch when you've got it working -- for all arches, for
all tasks,
and without requiring any vmcore file argument...  

 > Will gdb work directly with the kdump core file? If
not when did
 > it break and why? Docs with 2.6.16 suggest to use gdb
of the core
 > file.

I don't know if it *has* broke...

As I said this morning, I don't know if the latest and
greatest 32-bit
gdb can work with 64-bit ELF kdump vmcores, which kdump uses
by default
for both 32-bit and 64-bit architectures.  The kdump.txt
file says this:

   * By default, the ELF headers are stored in ELF64 format
to support
     systems with more than 4GB memory. The
--elf32-core-headers option can
     be used to force the generation of ELF32 headers. This
is necessary
     because GDB currently cannot open vmcore files with
ELF64 headers on
     32-bit systems. ELF32 headers can be used on non-PAE
systems (that is,
     less than 4GB of memory).

But at least with a 32-bit ELF vmcore, gdb should be able to
work.

 >
 > I recall the crash backtrack proving the gdb info back
a few years
 > ago when I worked on LKCD and crash with you. Is my
memory wrong
 > and this visibility of the formals and local variables
has never
 > never been possible? It's hard to recall for sure with
my using
 > kgdb on the kernel now for so long.
 >

Your memory's wrong, especially w/respect to local variables
-- are
you kidding me?

Dave

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Anderson wrote:
| Pete/Piet Delaney wrote:
|  > -----BEGIN PGP SIGNED MESSAGE-----
|  > Hash: SHA1
|  >
|  > Dave Anderson wrote:
|  > | Pete/Piet Delaney wrote:
|  > |> Got by 1st panic crash, crash seems to be
working but
|  > |> I don't feel comfortable as with gdb. I can't
see
|  > |> the parameters on the stack as variables as
with
|  > |> kgdb nor can I see the local variables on the
stack.
|  > |> Seems I should be able to get crash working
with
|  > |> gdb so I can see the details that gdb can
see.
|  > |>
|  > |> I compiled the kernel -O0 and without the
framepointer
|  > |> as it seem crash prefers (rather odd). But
want to
|  > |> see the locals and formals on the stack and
walk up and
|  > |> down the stack and have the context know to
crash just
|  > |> as with gdb.
|  > |>
|  > |> I'll re-read your crash paper.
|  > |>
|  > |> Any suggestions?
|  > |
|  > | Nope, other than using "bt -f" to dump
each frame's data,
|  > | and figuring it out from there...
|  >
|  > When I was using gdb on SunOS 4.1.4 I was able to
set $sp and
|  > $fp to switch processes. When I took the SP and FP
and used
|  > a gdb set $sp and %fp it didn't seen to allow me to
do this:
|  > -
-----------------------------------------------------------
|  > crash> gdb set $sp = 0xd76bbbb4
|  > No registers.
|  >
|  > crash> gdb set $fp = 0xd76bbc24
|  > No registers.
|  > -
-----------------------------------------------------------
|  > Why is is saying 'No registers'?
|
| I'm not positive, but the gdb sources seem to display that
message
| whenever (!target_has_registers), and that's #define'd
like so:
|
|   /* Does the target have registers?  (Exec files don't.) 
*/
|
|   #define target_has_registers    
|        (current_target.to_has_registers)
|
| And since it's being invoked as "gdb vmlinux",
it only knows
| about the exec file, and has no clue about registers.

Perhaps gdb changed since SunOS 4.1.4 in this area. We
could
consider patching gdb to think it has registers just to
allow
crash and/or the user from setting $sp and $fp to
facilitate
better back trace information. As I recall you make
environment
isn't building a virgin gdb.



|  >
|  > Also why don't you set up gdb with the stack info
for the current
|  > task and either allow the user to do a 'gdb bt' or
do it with a
|  > crash 'bt' option?
|  >
|
| I don't even come close to understanding gdb internals
well enough
| to know what "set up gdb with the stack info"
entails.  Send me
| a patch when you've got it working -- for all arches, for
all tasks,
| and without requiring any vmcore file argument...  
|
|  > Will gdb work directly with the kdump core file? If
not when did
|  > it break and why? Docs with 2.6.16 suggest to use
gdb of the core
|  > file.
|
| I don't know if it *has* broke...
|
| As I said this morning, I don't know if the latest and
greatest 32-bit
| gdb can work with 64-bit ELF kdump vmcores, which kdump
uses by default
| for both 32-bit and 64-bit architectures.  The kdump.txt
file says this:
|
|   * By default, the ELF headers are stored in ELF64 format
to support
|     systems with more than 4GB memory. The
--elf32-core-headers option can
|     be used to force the generation of ELF32 headers. This
is necessary
|     because GDB currently cannot open vmcore files with
ELF64 headers on
|     32-bit systems. ELF32 headers can be used on non-PAE
systems (that is,
|     less than 4GB of memory).
|
| But at least with a 32-bit ELF vmcore, gdb should be able
to work.

ok, I'll trying pulling out a DIMM and running on 2GB.

|
|  >
|  > I recall the crash backtrack proving the gdb info
back a few years
|  > ago when I worked on LKCD and crash with you. Is my
memory wrong
|  > and this visibility of the formals and local
variables has never
|  > never been possible? It's hard to recall for sure
with my using
|  > kgdb on the kernel now for so long.
|  >
|
| Your memory's wrong, especially w/respect to local
variables -- are
| you kidding me?

No, I wasn't joking, I thought back traces with crash had
the same
format as with gdb. Last I worked on IA64 and gdb I found
messing
with it's accessing of registers rather convolved. I was
modifying
Andy Kleen's gdb interface to SGI's kdb debugger, it's
called 'SKDB'.
At the time SGI had kdb working on IA64 kernels and I tried
modifying
if from ia32 to IA64. The register storage in gdb was a bit
odd.

Seems to be worthy of consideration to fool gdb into thing
it has
registers just so crash can get gdb to provide backtrace
info.
What happens of you simply change the #define to something
like:

	#define target_has_registers	1

I take it that crash in doing the stack unwind it's self
now
instead of having gdb do it. Have you thought about having
gdb
do it and what's needed?

On Friday you mentioned an embedded  gdb module to hijack
read
attempts and back them to crash. I tried a Google for more
info
but didn't find anything. Could you tell me more about
that?

- -piet

|
| Dave
|
| --
| Crash-utility mailing list
| Crash-utilityredhat.com
| https://www.redhat.com/mailman/listinfo/crash-utility
|

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFH1abFJICwm/rv3hoRAuMSAJ41wk72ITTIIwFJD6uBVlCHSo9jFwCa
A7yb
pCqvmBmt0WEI/8k9835UA+8=
=SPCi
-----END PGP SIGNATURE-----

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Dave Anderson wrote:
> Pete/Piet Delaney wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Dave Anderson wrote:
>> | Pete/Piet Delaney wrote:
>> |> Got by 1st panic crash, crash seems to be
working but
>> |> I don't feel comfortable as with gdb. I can't
see
>> |> the parameters on the stack as variables as
with
>> |> kgdb nor can I see the local variables on the
stack.
>> |> Seems I should be able to get crash working
with
>> |> gdb so I can see the details that gdb can
see.
>> |>
>> |> I compiled the kernel -O0 and without the
framepointer
>> |> as it seem crash prefers (rather odd). But
want to
>> |> see the locals and formals on the stack and
walk up and
>> |> down the stack and have the context know to
crash just
>> |> as with gdb.
>> |>
>> |> I'll re-read your crash paper.
>> |>
>> |> Any suggestions?
>> |
>> | Nope, other than using "bt -f" to dump
each frame's data,
>> | and figuring it out from there...
>>
>> When I was using gdb on SunOS 4.1.4 I was able to
set $sp and
>> $fp to switch processes. When I took the SP and FP
and used
>> a gdb set $sp and %fp it didn't seen to allow me to
do this:
>> -
-----------------------------------------------------------
>> crash> gdb set $sp = 0xd76bbbb4
>> No registers.
>>
>> crash> gdb set $fp = 0xd76bbc24
>> No registers.
>> -
-----------------------------------------------------------
>> Why is is saying 'No registers'?
>
> I'm not positive, but the gdb sources seem to display
that message
> whenever (!target_has_registers), and that's #define'd
like so:
>
>   /* Does the target have registers?  (Exec files
don't.)  */
>
>   #define target_has_registers    
>        (current_target.to_has_registers)
>
> And since it's being invoked as "gdb
vmlinux", it only knows
> about the exec file, and has no clue about registers.

Running ddd+gdb on the crash dump seem to be working
unreasonably
well. The back traces for the task running on CPU's are
available as
well as the registers.

I thought it would be convenient to get the $sp and $fp from
'crash' for a
task that I'm interested in and change the values of $sp and
$fp for one
of the CPU's contexts.  I did a 'set write on' to allow me
to change the 
contents
of the crash dump but it continues to say the $sp isn't an
l-value.

    gdb) set write on
    gdb) set $sp += 4
        Left operand of assignment is not an lvalue.
    (gdb)

 
  (gdb) help set write
     Set writing into executable and core files.
   (gdb)

kgdb use to allow this on live sessions on SunOS 4.1.4.
Seems to a common practice still:

    http://www
.mcs.vuw.ac.nz/cgi-bin/info2www?(gdb)Registers

I'm fine on the current bug that I'm looking into but would
like to be able to use gdb on all of the tasks; not just the
ones
currently running on CPUs. The kdump gdb macros are a bit
of help but extremely slow. I once looked in kgdb running
slow
on gdb macro interpretation and at that time is was a silly
lack
of caching that was causing the gdb 'ps' macro to run so
slowly.

 
-piet
>

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Pete/Piet Delaney wrote:
> |  > When I was using gdb on SunOS 4.1.4 I was able
to set $sp and
> |  > $fp to switch processes. When I took the SP and
FP and used
> |  > a gdb set $sp and %fp it didn't seen to allow
me to do this:
> |  > -
-----------------------------------------------------------
> |  > crash> gdb set $sp = 0xd76bbbb4
> |  > No registers.
> |  >
> |  > crash> gdb set $fp = 0xd76bbc24
> |  > No registers.
> |  > -
-----------------------------------------------------------
> |  > Why is is saying 'No registers'?
> |
> | I'm not positive, but the gdb sources seem to display
that message
> | whenever (!target_has_registers), and that's
#define'd like so:
> |
> |   /* Does the target have registers?  (Exec files
don't.)  */
> |
> |   #define target_has_registers    
> |        (current_target.to_has_registers)
> |
> | And since it's being invoked as "gdb
vmlinux", it only knows
> | about the exec file, and has no clue about
registers.
> 
> Perhaps gdb changed since SunOS 4.1.4 in this area. We
could
> consider patching gdb to think it has registers just to
allow
> crash and/or the user from setting $sp and $fp to
facilitate
> better back trace information. As I recall you make
environment
> isn't building a virgin gdb.

That's exactly what I tried many years ago -- just forcibly
setting
the stack and frame pointer values.  But it still required
more
"user environment" information to work correctly. 
I had some limited
success on the alpha platform, but never on x86.

And that's only the beginning.  To be of any use, it would
have to
be able to deal with in-kernel exception frames, which from
a user
debugger standpoint, it has no idea what they are.  And it
would need
to be able to deal with the stack-switching done by the
various
architectures to handle IRQ's, special-case exceptions, soft
IRQ
handling, dealing with the assembly-code magic performed by
the
entry.S entry points, and son on.  I noted that your crash
example
was a "clean" panic() call -- as opposed to a the
far-more-likely
crash scenario where the kernel is running in kernel mode,
takes
an exception, lays down an exception frame, switches stacks,
etc...

> 
> 
> 
> |  >
> |  > Also why don't you set up gdb with the stack
info for the current
> |  > task and either allow the user to do a 'gdb bt'
or do it with a
> |  > crash 'bt' option?
> |  >
> |
> | I don't even come close to understanding gdb
internals well enough
> | to know what "set up gdb with the stack
info" entails.  Send me
> | a patch when you've got it working -- for all arches,
for all tasks,
> | and without requiring any vmcore file argument... 

> |
> |  > Will gdb work directly with the kdump core
file? If not when did
> |  > it break and why? Docs with 2.6.16 suggest to
use gdb of the core
> |  > file.
> |
> | I don't know if it *has* broke...
> |
> | As I said this morning, I don't know if the latest
and greatest 32-bit
> | gdb can work with 64-bit ELF kdump vmcores, which
kdump uses by default
> | for both 32-bit and 64-bit architectures.  The
kdump.txt file says this:
> |
> |   * By default, the ELF headers are stored in ELF64
format to support
> |     systems with more than 4GB memory. The
--elf32-core-headers option 
> can
> |     be used to force the generation of ELF32 headers.
This is necessary
> |     because GDB currently cannot open vmcore files
with ELF64 headers on
> |     32-bit systems. ELF32 headers can be used on
non-PAE systems (that 
> is,
> |     less than 4GB of memory).
> |
> | But at least with a 32-bit ELF vmcore, gdb should be
able to work.
> 
> ok, I'll trying pulling out a DIMM and running on 2GB.

But you'll still get a 64-bit vmcore unless you deal with
the
kexec/kdump configuration.  And why not just put
"mem=2048m" on
the boot command line?


> | Your memory's wrong, especially w/respect to local
variables -- are
> | you kidding me?
> 
> No, I wasn't joking, I thought back traces with crash
had the same
> format as with gdb. Last I worked on IA64 and gdb I
found messing
> with it's accessing of registers rather convolved. I
was modifying
> Andy Kleen's gdb interface to SGI's kdb debugger, it's
called 'SKDB'.
> At the time SGI had kdb working on IA64 kernels and I
tried modifying
> if from ia32 to IA64. The register storage in gdb was a
bit odd.
> 
> Seems to be worthy of consideration to fool gdb into
thing it has
> registers just so crash can get gdb to provide
backtrace info.
> What happens of you simply change the #define to
something
> like:
> 
>     #define target_has_registers    1
> 
> I take it that crash in doing the stack unwind it's
self now
> instead of having gdb do it. Have you thought about
having gdb
> do it and what's needed?

See above...

> 
> On Friday you mentioned an embedded  gdb module to
hijack read
> attempts and back them to crash. I tried a Google for
more info
> but didn't find anything. Could you tell me more about
that?

That's *my* term for the stock version of gdb-6.1 that I
hacked/married
with the crash sources.  If you look at the gdb sources you
can see
that it already had hooks to "embed" it into
another bimary,
typically into a GUI, or some such thing.  You basically
hijack
the user interface from the gdb prompt, and pass it back to
your
own user interface.

Dave


--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Anderson wrote:
| Pete/Piet Delaney wrote:
|> |  > When I was using gdb on SunOS 4.1.4 I was able
to set $sp and
|> |  > $fp to switch processes. When I took the SP
and FP and used
|> |  > a gdb set $sp and %fp it didn't seen to allow
me to do this:
|> |  > -
-----------------------------------------------------------
|> |  > crash> gdb set $sp = 0xd76bbbb4
|> |  > No registers.
|> |  >
|> |  > crash> gdb set $fp = 0xd76bbc24
|> |  > No registers.
|> |  > -
-----------------------------------------------------------
|> |  > Why is is saying 'No registers'?
|> |
|> | I'm not positive, but the gdb sources seem to
display that message
|> | whenever (!target_has_registers), and that's
#define'd like so:
|> |
|> |   /* Does the target have registers?  (Exec files
don't.)  */
|> |
|> |   #define target_has_registers    
|> |        (current_target.to_has_registers)
|> |
|> | And since it's being invoked as "gdb
vmlinux", it only knows
|> | about the exec file, and has no clue about
registers.
|>
|> Perhaps gdb changed since SunOS 4.1.4 in this area. We
could
|> consider patching gdb to think it has registers just
to allow
|> crash and/or the user from setting $sp and $fp to
facilitate
|> better back trace information. As I recall you make
environment
|> isn't building a virgin gdb.
|
| That's exactly what I tried many years ago -- just
forcibly setting
| the stack and frame pointer values.  But it still required
more
| "user environment" information to work
correctly.  I had some limited
| success on the alpha platform, but never on x86.
|
| And that's only the beginning.  To be of any use, it would
have to
| be able to deal with in-kernel exception frames, which
from a user
| debugger standpoint, it has no idea what they are.  And it
would need
| to be able to deal with the stack-switching done by the
various
| architectures to handle IRQ's, special-case exceptions,
soft IRQ
| handling, dealing with the assembly-code magic performed
by the
| entry.S entry points, and son on.  I noted that your crash
example
| was a "clean" panic() call -- as opposed to a
the far-more-likely
| crash scenario where the kernel is running in kernel mode,
takes
| an exception, lays down an exception frame, switches
stacks, etc...

I'm not sure there is a significant problem there, Tom Reni
at
MontaVista wrote some kgdb enhancements for kgdb added Call
Frame
Instruction (CFI) macros that are added to assembler code
for
exception handlers to tell gdb how to deal with these
assembler
stack frames. I'm going to not only enable these for
CONFIG_KGDB
kernels but also for KEXEC kernels, as these are likely to
generate
crash dumps. I noticed your crash code has CFI support for
stack
unwinding.

I more recent kernels may now support CFI already. I'm
still
on an old 2.6.12 kgdb patch. Jason Wessel is working and
Andrew
Morton to integrate the KGDB patch into the mainstream
kernel.
Last I read it looked like 2.6.25 was the likely target but
I still don't see it in Linus's 2.6.25-rc5 git repo. Sigh!

I'd like to see a consistent debugging environment from live
debugging
with kgdb over to crash analysis with gdb+ddd and crash.
Likely the more
crash is integrated into gdb the better. Perhaps over the
long term
integrating support into the kernel and gdb would be better.
Remember
the old -k option for dbx that Sun used when using dbxtool
on the old
M68000 kernels prior to SPARC?



|> |  >
|> |  > Also why don't you set up gdb with the stack
info for the current
|> |  > task and either allow the user to do a 'gdb
bt' or do it with a
|> |  > crash 'bt' option?
|> |  >
|> |
|> | I don't even come close to understanding gdb
internals well enough
|> | to know what "set up gdb with the stack
info" entails.  Send me
|> | a patch when you've got it working -- for all
arches, for all tasks,
|> | and without requiring any vmcore file argument... 


What's the downside to providing the vmcore file to gdb when
you
start it and then using the internal command line interface
to
set thing like $sp and $fp to get gdb to do it's thing
during a
backtrace? Maybe I can get gdb on core files to allow me to
change $sp and $fp with a minor gdb change. If that works I
would think it easy to get crash to do the same to it's gdb
pipe and if you provide the core to gdb on startup. Once
gdb
has the stack context, like gdb on old SunOS 4.1.4, then
crash
can pass thru cmds to walk up and down the stack.

Am I dreaming? It tried just changing a global variable in
the
core file, starting ddd --write and it core dumped when I
changed
the global variable.


|> |
|> |  > Will gdb work directly with the kdump core
file? If not when did
|> |  > it break and why? Docs with 2.6.16 suggest to
use gdb of the core
|> |  > file.
|> |
|> | I don't know if it *has* broke...
|> |
|> | As I said this morning, I don't know if the latest
and greatest 32-bit
|> | gdb can work with 64-bit ELF kdump vmcores, which
kdump uses by default
|> | for both 32-bit and 64-bit architectures.  The
kdump.txt file says
|> this:
|> |
|> |   * By default, the ELF headers are stored in ELF64
format to support
|> |     systems with more than 4GB memory. The
--elf32-core-headers
|> option can
|> |     be used to force the generation of ELF32
headers. This is necessary
|> |     because GDB currently cannot open vmcore files
with ELF64
|> headers on
|> |     32-bit systems. ELF32 headers can be used on
non-PAE systems
|> (that is,
|> |     less than 4GB of memory).
|> |
|> | But at least with a 32-bit ELF vmcore, gdb should be
able to work.
|>
|> ok, I'll trying pulling out a DIMM and running on
2GB.
|
| But you'll still get a 64-bit vmcore unless you deal with
the
| kexec/kdump configuration.  And why not just put
"mem=2048m" on
| the boot command line?

Yea, that's when I did. The back traces for eight CPU's look
fine,
thought I haven't enabled the KGDB CFI macros yet. Data
browsing
our sockets was great. I showed the guys last night;
reaction from one
of our managers was "Very cool".


- -piet


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFH14LMJICwm/rv3hoRAnJdAJsGDeztKYox1ElDGYIXKc+kjDRDjwCg
hlof
IBZWgi3h0HyZFEIC8KuMdN0=
=js5F
-----END PGP SIGNATURE-----

--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Pete/Piet Delaney wrote:
>
> I'd like to see a consistent debugging environment from
live debugging
> with kgdb over to crash analysis with gdb+ddd and
crash. Likely the more
> crash is integrated into gdb the better. Perhaps over
the long term
> integrating support into the kernel and gdb would be
better. Remember
> the old -k option for dbx that Sun used when using
dbxtool on the old
> M68000 kernels prior to SPARC?

No.

> What's the downside to providing the vmcore file to gdb
when you
> start it and then using the internal command line
interface to
> set thing like $sp and $fp to get gdb to do it's thing
during a
> backtrace? Maybe I can get gdb on core files to allow
me to
> change $sp and $fp with a minor gdb change. If that
works I
> would think it easy to get crash to do the same to it's
gdb
> pipe and if you provide the core to gdb on startup.
Once gdb
> has the stack context, like gdb on old SunOS 4.1.4,
then crash
> can pass thru cmds to walk up and down the stack.

ELF vmcores are only one of several dumpfile formats crash
supports.

With the ever-increasing memory sizes, the use of the post
crash
dump makedumpfile utility to change the kdump ELF file into
a
compressed diskdump-clone dumpfile format will become all
the more
common.  Not to mention the use of compressed diskump
dumpfiles,
the unique format used by xen dom0 kdump dumpfiles, or the
format
used by xen domU dumpfiles, or LKCD, and so on.  Any change
is
is going to have to be able to extract the information
needed
by gdb without having to pass the vmcore file to it. 
AFAICT
gdb pulls whatever info into needs from the NT_PRSTATUS
note
in the ELF header.

> 
> Am I dreaming? It tried just changing a global variable
in the
> core file, starting ddd --write and it core dumped when
I changed
> the global variable.

That's great -- so just use it and be happy.

And when you have a fully-functional patch that covers all
bases,
please post it for review.

Dave


--
Crash-utility mailing list
Crash-utilityredhat.com
https://www.redhat.com/mailman/listinfo/crash-utility