List Info

Thread: nfs on zfs panic




nfs on zfs panic
country flaguser name
France
2007-06-26 11:12:36
Hi,

I'm playing with zfs on vmware on FreeBSD current (world and
kernel of 
today with GENERIC kernel). I have 3 disks (virtuals) in
raidz :

# zpool status
   pool: tank
  state: ONLINE
  scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         tank        ONLINE       0     0     0
           raidz1    ONLINE       0     0     0
             da0     ONLINE       0     0     0
             da1     ONLINE       0     0     0
             da2     ONLINE       0     0     0

errors: No known data errors

with this zfs configuration :

# zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
tank         844M  8.93G  25.3K  none
tank/nfs    24.0K  8.93G  24.0K  /mnt/nfs
tank/obj     436M  8.93G   436M  /usr/obj
tank/ports   244M  8.93G   244M  /usr/ports
tank/src     162M  8.93G   162M  /usr/src

/mnt/nfs is nfs exported (zfs set sharenfs=on tank/nfs) and
when I mount 
it on a Linux box and try a ls from Linux, FreeBSD panic.

panic and debug informations :

# kgdb kernel.debug /var/crash/vmcore.0
[GDB will not be able to debug user-mode threads: 
/usr/lib/libthread_db.so: Undefined symbol
"ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public
License, and you are
welcome to change it and/or distribute copies of it under
certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show
warranty" for details.
This GDB was configured as "i386-marcel-freebsd".


Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x10
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc08d088d
stack pointer           = 0x28:0xd61f88f0
frame pointer           = 0x28:0xd61f8bec
code segment            = base 0x0, limit 0xfffff, type
0x1b
                         = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL =
0
current process         = 722 (nfsd)
panic: from debugger
cpuid = 0
Uptime: 3h46m48s
Physical memory: 499 MB
Dumping 52 MB: 37 21 5

#0  doadump () at pcpu.h:195
195             __asm __volatile("movl %%fs:0,%0"
: "=r" (td));
(kgdb) backtrace
#0  doadump () at pcpu.h:195
#1  0xc074997e in boot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0749c3b in panic (fmt=Variable "fmt" is not
available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc048bf87 in db_panic (addr=Could not find the frame
base for 
"db_panic".
) at /usr/src/sys/ddb/db_command.c:433
#4  0xc048c975 in db_command_loop () at
/usr/src/sys/ddb/db_command.c:401
#5  0xc048e0e5 in db_trap (type=12, code=0) at 
/usr/src/sys/ddb/db_main.c:222
#6  0xc07703d6 in kdb_trap (type=12, code=0, tf=0xd61f88b0)
at 
/usr/src/sys/kern/subr_kdb.c:502
#7  0xc09f31bc in trap_fatal (frame=0xd61f88b0, eva=16) at 
/usr/src/sys/i386/i386/trap.c:861
#8  0xc09f33f3 in trap_pfault (frame=0xd61f88b0, usermode=0,
eva=16) at 
/usr/src/sys/i386/i386/trap.c:784
#9  0xc09f3d92 in trap (frame=0xd61f88b0) at 
/usr/src/sys/i386/i386/trap.c:462
#10 0xc09d9a0b in calltrap () at
/usr/src/sys/i386/i386/exception.s:139
#11 0xc08d088d in nfsrv_readdirplus (nfsd=0xc3bb4500,
slp=0xc3d31900, 
td=0xc3b0be00, mrq=0xd61f8c58) at
/usr/src/sys/nfsserver/nfs_serv.c:3640
#12 0xc08de4b4 in nfssvc (td=0xc3b0be00, uap=0xd61f8cfc) at

/usr/src/sys/nfsserver/nfs_syscalls.c:469
#13 0xc09f36d3 in syscall (frame=0xd61f8d38) at 
/usr/src/sys/i386/i386/trap.c:1006
#14 0xc09d9a70 in Xint0x80_syscall () at 
/usr/src/sys/i386/i386/exception.s:196
#15 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) list *0xc08d088d
0xc08d088d is in nfsrv_readdirplus
(/usr/src/sys/nfsserver/nfs_serv.c:3640).
3635                             */
3636                            if (VFS_VGET(vp->v_mount,
dp->d_fileno, 
LK_EXCLUSIVE,
3637                                &nvp))
3638                                    goto invalid;
3639                            bzero((caddr_t)nfhp,
NFSX_V3FH);
3640                            nfhp->fh_fsid =
3641                                   
nvp->v_mount->mnt_stat.f_fsid;
3642                            /*
3643                             * XXXRW: Assert the
mountpoints are the 
same so that
3644                             * we know that acquiring
Giant based on the
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Re: nfs on zfs panic
country flaguser name
United Kingdom
2007-06-26 12:10:44
It looks like nvp is NULL at the point where it crashed.
Looking at  
the zfs code, zfs_vget always returns zero, even if it
failed to find  
a vnode which matches the given 'inode' number. Try changing
the  
return statement in zfs_vget from 'return (0)' to 'return
(err)'.

PS. the code is in
src/sys/contrib/opensolaris/uts/common/fs/zfs/ 
zfs_vfsops.c - it took me a while to find it.

On 26 Jun 2007, at 17:12, Philippe Pegon wrote:

> Hi,
>
> I'm playing with zfs on vmware on FreeBSD current
(world and kernel  
> of today with GENERIC kernel). I have 3 disks
(virtuals) in raidz :
>
> # zpool status
>   pool: tank
>  state: ONLINE
>  scrub: none requested
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         tank        ONLINE       0     0     0
>           raidz1    ONLINE       0     0     0
>             da0     ONLINE       0     0     0
>             da1     ONLINE       0     0     0
>             da2     ONLINE       0     0     0
>
> errors: No known data errors
>
> with this zfs configuration :
>
> # zfs list
> NAME         USED  AVAIL  REFER  MOUNTPOINT
> tank         844M  8.93G  25.3K  none
> tank/nfs    24.0K  8.93G  24.0K  /mnt/nfs
> tank/obj     436M  8.93G   436M  /usr/obj
> tank/ports   244M  8.93G   244M  /usr/ports
> tank/src     162M  8.93G   162M  /usr/src
>
> /mnt/nfs is nfs exported (zfs set sharenfs=on tank/nfs)
and when I  
> mount it on a Linux box and try a ls from Linux,
FreeBSD panic.
>
> panic and debug informations :
>
> # kgdb kernel.debug /var/crash/vmcore.0
> [GDB will not be able to debug user-mode threads:
/usr/lib/ 
> libthread_db.so: Undefined symbol
"ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public
License,  
> and you are
> welcome to change it and/or distribute copies of it
under certain  
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type
"show warranty" for  
> details.
> This GDB was configured as
"i386-marcel-freebsd".
>
>
> Unread portion of the kernel message buffer:
>
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 0; apic id = 00
> fault virtual address   = 0x10
> fault code              = supervisor read, page not
present
> instruction pointer     = 0x20:0xc08d088d
> stack pointer           = 0x28:0xd61f88f0
> frame pointer           = 0x28:0xd61f8bec
> code segment            = base 0x0, limit 0xfffff, type
0x1b
>                         = DPL 0, pres 1, def32 1, gran
1
> processor eflags        = interrupt enabled, resume,
IOPL = 0
> current process         = 722 (nfsd)
> panic: from debugger
> cpuid = 0
> Uptime: 3h46m48s
> Physical memory: 499 MB
> Dumping 52 MB: 37 21 5
>
> #0  doadump () at pcpu.h:195
> 195             __asm __volatile("movl
%%fs:0,%0" : "=r" (td));
> (kgdb) backtrace
> #0  doadump () at pcpu.h:195
> #1  0xc074997e in boot (howto=260) at
/usr/src/sys/kern/ 
> kern_shutdown.c:409
> #2  0xc0749c3b in panic (fmt=Variable "fmt"
is not available.
> ) at /usr/src/sys/kern/kern_shutdown.c:563
> #3  0xc048bf87 in db_panic (addr=Could not find the
frame base for  
> "db_panic".
> ) at /usr/src/sys/ddb/db_command.c:433
> #4  0xc048c975 in db_command_loop () at
/usr/src/sys/ddb/ 
> db_command.c:401
> #5  0xc048e0e5 in db_trap (type=12, code=0) at
/usr/src/sys/ddb/ 
> db_main.c:222
> #6  0xc07703d6 in kdb_trap (type=12, code=0,
tf=0xd61f88b0) at /usr/ 
> src/sys/kern/subr_kdb.c:502
> #7  0xc09f31bc in trap_fatal (frame=0xd61f88b0, eva=16)
at /usr/src/ 
> sys/i386/i386/trap.c:861
> #8  0xc09f33f3 in trap_pfault (frame=0xd61f88b0,
usermode=0,  
> eva=16) at /usr/src/sys/i386/i386/trap.c:784
> #9  0xc09f3d92 in trap (frame=0xd61f88b0) at
/usr/src/sys/i386/i386/ 
> trap.c:462
> #10 0xc09d9a0b in calltrap () at
/usr/src/sys/i386/i386/exception.s: 
> 139
> #11 0xc08d088d in nfsrv_readdirplus (nfsd=0xc3bb4500, 

> slp=0xc3d31900, td=0xc3b0be00, mrq=0xd61f8c58) at
/usr/src/sys/ 
> nfsserver/nfs_serv.c:3640
> #12 0xc08de4b4 in nfssvc (td=0xc3b0be00,
uap=0xd61f8cfc) at /usr/ 
> src/sys/nfsserver/nfs_syscalls.c:469
> #13 0xc09f36d3 in syscall (frame=0xd61f8d38) at
/usr/src/sys/i386/ 
> i386/trap.c:1006
> #14 0xc09d9a70 in Xint0x80_syscall () at
/usr/src/sys/i386/i386/ 
> exception.s:196
> #15 0x00000033 in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> (kgdb) list *0xc08d088d
> 0xc08d088d is in nfsrv_readdirplus
(/usr/src/sys/nfsserver/ 
> nfs_serv.c:3640).
> 3635                             */
> 3636                            if
(VFS_VGET(vp->v_mount, dp- 
> >d_fileno, LK_EXCLUSIVE,
> 3637                                &nvp))
> 3638                                    goto invalid;
> 3639                            bzero((caddr_t)nfhp,
NFSX_V3FH);
> 3640                            nfhp->fh_fsid =
> 3641                                   
nvp->v_mount->mnt_stat.f_fsid;
> 3642                            /*
> 3643                             * XXXRW: Assert the
mountpoints  
> are the same so that
> 3644                             * we know that
acquiring Giant  
> based on the
> _______________________________________________
> freebsd-currentfreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
> To unsubscribe, send any mail to "freebsd-current-

> unsubscribefreebsd.org"

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Re: nfs on zfs panic
country flaguser name
France
2007-06-27 01:54:42
Doug Rabson wrote:
> It looks like nvp is NULL at the point where it
crashed. Looking at the 
> zfs code, zfs_vget always returns zero, even if it
failed to find a 
> vnode which matches the given 'inode' number. Try
changing the return 
> statement in zfs_vget from 'return (0)' to 'return
(err)'.

That seems to fix the problem 

thanks

> 
> PS. the code is in 
>
src/sys/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c -
it took me 
> a while to find it.
> 
> On 26 Jun 2007, at 17:12, Philippe Pegon wrote:
> 
>> Hi,
>>
>> I'm playing with zfs on vmware on FreeBSD current
(world and kernel of 
>> today with GENERIC kernel). I have 3 disks
(virtuals) in raidz :
>>
>> # zpool status
>>   pool: tank
>>  state: ONLINE
>>  scrub: none requested
>> config:
>>
>>         NAME        STATE     READ WRITE CKSUM
>>         tank        ONLINE       0     0     0
>>           raidz1    ONLINE       0     0     0
>>             da0     ONLINE       0     0     0
>>             da1     ONLINE       0     0     0
>>             da2     ONLINE       0     0     0
>>
>> errors: No known data errors
>>
>> with this zfs configuration :
>>
>> # zfs list
>> NAME         USED  AVAIL  REFER  MOUNTPOINT
>> tank         844M  8.93G  25.3K  none
>> tank/nfs    24.0K  8.93G  24.0K  /mnt/nfs
>> tank/obj     436M  8.93G   436M  /usr/obj
>> tank/ports   244M  8.93G   244M  /usr/ports
>> tank/src     162M  8.93G   162M  /usr/src
>>
>> /mnt/nfs is nfs exported (zfs set sharenfs=on
tank/nfs) and when I 
>> mount it on a Linux box and try a ls from Linux,
FreeBSD panic.
>>
>> panic and debug informations :
>>
>> # kgdb kernel.debug /var/crash/vmcore.0
>> [GDB will not be able to debug user-mode threads: 
>> /usr/lib/libthread_db.so: Undefined symbol
"ps_pglobal_lookup"]
>> GNU gdb 6.1.1 [FreeBSD]
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General
Public License, and 
>> you are
>> welcome to change it and/or distribute copies of it
under certain 
>> conditions.
>> Type "show copying" to see the
conditions.
>> There is absolutely no warranty for GDB.  Type
"show warranty" for 
>> details.
>> This GDB was configured as
"i386-marcel-freebsd".
>>
>>
>> Unread portion of the kernel message buffer:
>>
>>
>> Fatal trap 12: page fault while in kernel mode
>> cpuid = 0; apic id = 00
>> fault virtual address   = 0x10
>> fault code              = supervisor read, page not
present
>> instruction pointer     = 0x20:0xc08d088d
>> stack pointer           = 0x28:0xd61f88f0
>> frame pointer           = 0x28:0xd61f8bec
>> code segment            = base 0x0, limit 0xfffff,
type 0x1b
>>                         = DPL 0, pres 1, def32 1,
gran 1
>> processor eflags        = interrupt enabled,
resume, IOPL = 0
>> current process         = 722 (nfsd)
>> panic: from debugger
>> cpuid = 0
>> Uptime: 3h46m48s
>> Physical memory: 499 MB
>> Dumping 52 MB: 37 21 5
>>
>> #0  doadump () at pcpu.h:195
>> 195             __asm __volatile("movl
%%fs:0,%0" : "=r" (td));
>> (kgdb) backtrace
>> #0  doadump () at pcpu.h:195
>> #1  0xc074997e in boot (howto=260) at 
>> /usr/src/sys/kern/kern_shutdown.c:409
>> #2  0xc0749c3b in panic (fmt=Variable
"fmt" is not available.
>> ) at /usr/src/sys/kern/kern_shutdown.c:563
>> #3  0xc048bf87 in db_panic (addr=Could not find the
frame base for 
>> "db_panic".
>> ) at /usr/src/sys/ddb/db_command.c:433
>> #4  0xc048c975 in db_command_loop () at
/usr/src/sys/ddb/db_command.c:401
>> #5  0xc048e0e5 in db_trap (type=12, code=0) at 
>> /usr/src/sys/ddb/db_main.c:222
>> #6  0xc07703d6 in kdb_trap (type=12, code=0,
tf=0xd61f88b0) at 
>> /usr/src/sys/kern/subr_kdb.c:502
>> #7  0xc09f31bc in trap_fatal (frame=0xd61f88b0,
eva=16) at 
>> /usr/src/sys/i386/i386/trap.c:861
>> #8  0xc09f33f3 in trap_pfault (frame=0xd61f88b0,
usermode=0, eva=16) 
>> at /usr/src/sys/i386/i386/trap.c:784
>> #9  0xc09f3d92 in trap (frame=0xd61f88b0) at 
>> /usr/src/sys/i386/i386/trap.c:462
>> #10 0xc09d9a0b in calltrap () at
/usr/src/sys/i386/i386/exception.s:139
>> #11 0xc08d088d in nfsrv_readdirplus
(nfsd=0xc3bb4500, slp=0xc3d31900, 
>> td=0xc3b0be00, mrq=0xd61f8c58) at
/usr/src/sys/nfsserver/nfs_serv.c:3640
>> #12 0xc08de4b4 in nfssvc (td=0xc3b0be00,
uap=0xd61f8cfc) at 
>> /usr/src/sys/nfsserver/nfs_syscalls.c:469
>> #13 0xc09f36d3 in syscall (frame=0xd61f8d38) at 
>> /usr/src/sys/i386/i386/trap.c:1006
>> #14 0xc09d9a70 in Xint0x80_syscall () at 
>> /usr/src/sys/i386/i386/exception.s:196
>> #15 0x00000033 in ?? ()
>> Previous frame inner to this frame (corrupt
stack?)
>> (kgdb) list *0xc08d088d
>> 0xc08d088d is in nfsrv_readdirplus 
>> (/usr/src/sys/nfsserver/nfs_serv.c:3640).
>> 3635                             */
>> 3636                            if
(VFS_VGET(vp->v_mount, 
>> dp->d_fileno, LK_EXCLUSIVE,
>> 3637                                &nvp))
>> 3638                                    goto
invalid;
>> 3639                           
bzero((caddr_t)nfhp, NFSX_V3FH);
>> 3640                            nfhp->fh_fsid =
>> 3641                                   
nvp->v_mount->mnt_stat.f_fsid;
>> 3642                            /*
>> 3643                             * XXXRW: Assert
the mountpoints are 
>> the same so that
>> 3644                             * we know that
acquiring Giant based 
>> on the
>> _______________________________________________
>> freebsd-currentfreebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
>> To unsubscribe, send any mail to 
>> "freebsd-current-unsubscribefreebsd.org"
> 

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Re: nfs on zfs panic
user name
2007-06-27 02:06:18
On Tue, Jun 26, 2007 at 06:10:44PM +0100, Doug Rabson
wrote:
> It looks like nvp is NULL at the point where it
crashed. Looking at the zfs code, zfs_vget always returns
zero, even if it failed to find a vnode which matches the
given 
> 'inode' number. Try changing the return statement in
zfs_vget from 'return (0)' to 'return (err)'.

Your analysis is correct, can you commit this?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjdFreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I
Am!
Re: nfs on zfs panic
country flaguser name
United Kingdom
2007-06-27 02:37:27
On Wednesday 27 June 2007, Pawel Jakub Dawidek wrote:
> On Tue, Jun 26, 2007 at 06:10:44PM +0100, Doug Rabson
wrote:
> > It looks like nvp is NULL at the point where it
crashed. Looking at
> > the zfs code, zfs_vget always returns zero, even
if it failed to
> > find a vnode which matches the given 'inode'
number. Try changing
> > the return statement in zfs_vget from 'return (0)'
to 'return
> > (err)'.
>
> Your analysis is correct, can you commit this?

I tried to commit it but got the 'need Approved by: re'
message. Do you 
have blanket re approval for this stuff?
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Re: nfs on zfs panic
country flaguser name
Poland
2007-06-27 05:12:50
On Wed, Jun 27, 2007 at 08:37:27AM +0100, Doug Rabson
wrote:
> On Wednesday 27 June 2007, Pawel Jakub Dawidek wrote:
> > On Tue, Jun 26, 2007 at 06:10:44PM +0100, Doug
Rabson wrote:
> > > It looks like nvp is NULL at the point where
it crashed. Looking at
> > > the zfs code, zfs_vget always returns zero,
even if it failed to
> > > find a vnode which matches the given 'inode'
number. Try changing
> > > the return statement in zfs_vget from 'return
(0)' to 'return
> > > (err)'.
> >
> > Your analysis is correct, can you commit this?
> 
> I tried to commit it but got the 'need Approved by: re'
message. Do you 
> have blanket re approval for this stuff?

Nope. Could you ask re for approval? If you don't have time,
I'll
handle this later.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjdFreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I
Am!
Re: nfs on zfs panic
country flaguser name
United Kingdom
2007-06-27 05:50:15
On 27 Jun 2007, at 11:12, Pawel Jakub Dawidek wrote:

> On Wed, Jun 27, 2007 at 08:37:27AM +0100, Doug Rabson
wrote:
>> On Wednesday 27 June 2007, Pawel Jakub Dawidek
wrote:
>>> On Tue, Jun 26, 2007 at 06:10:44PM +0100, Doug
Rabson wrote:
>>>> It looks like nvp is NULL at the point
where it crashed. Looking at
>>>> the zfs code, zfs_vget always returns zero,
even if it failed to
>>>> find a vnode which matches the given
'inode' number. Try changing
>>>> the return statement in zfs_vget from
'return (0)' to 'return
>>>> (err)'.
>>>
>>> Your analysis is correct, can you commit this?
>>
>> I tried to commit it but got the 'need Approved by:
re' message.  
>> Do you
>> have blanket re approval for this stuff?
>
> Nope. Could you ask re for approval? If you don't
have time, I'll
> handle this later.

I'll sort things out with re.

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

[1-7]

about | contact  Other archives ( Real Estate discussion Medical topics )