only 8 LUNs on MPT

we are using LSIs 3801E HBA to connect Infortrends
S16S-G1030+JBOD (e.g. 32
disks, 16 x RAID0, 16 LUNs) to an AMD64 machine.

the Infortrend Box is momentarily configured with 9 LUNs.

When booting only 8 LUNs (da45 to da52) show up

da45 at mpt0 bus 0 target 10 lun 0
da45: <IFT S16S-G1030 361F> Fixed Direct Access SCSI-4
device
da45: 300.000MB/s transfers
da45: Command Queueing Enabled
da45: 1430288MB (2929229824 512 byte sectors: 255H 63S/T
182336C)
da46 at mpt0 bus 0 target 10 lun 1
.....
da52 at mpt0 bus 0 target 10 lun 7
da52: <IFT S16S-G1030 361F> Fixed Direct Access SCSI-4
device
da52: 300.000MB/s transfers
da52: Command Queueing Enabled
da52: 1430288MB (2929229824 512 byte sectors: 255H 63S/T
182336C)

"camcontrol reportluns 2:10"
shows: "9 LUNs found"

how to get more that 8 LUNs to map?

May be, some limitation somewhere in the code for MPT ?

any help would be appreciated.


Dieter


_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

d_elbracht wrote:
> we are using LSIs 3801E HBA to connect Infortrends
S16S-G1030+JBOD (e.g. 32
> disks, 16 x RAID0, 16 LUNs) to an AMD64 machine.
> 
> the Infortrend Box is momentarily configured with 9
LUNs.
> 
> When booting only 8 LUNs (da45 to da52) show up
> 
> da45 at mpt0 bus 0 target 10 lun 0
> da45: <IFT S16S-G1030 361F> Fixed Direct Access
SCSI-4 device
> da45: 300.000MB/s transfers
> da45: Command Queueing Enabled
> da45: 1430288MB (2929229824 512 byte sectors: 255H
63S/T 182336C)
> da46 at mpt0 bus 0 target 10 lun 1
> .....
> da52 at mpt0 bus 0 target 10 lun 7
> da52: <IFT S16S-G1030 361F> Fixed Direct Access
SCSI-4 device
> da52: 300.000MB/s transfers
> da52: Command Queueing Enabled
> da52: 1430288MB (2929229824 512 byte sectors: 255H
63S/T 182336C)
> 
> "camcontrol reportluns 2:10"
> shows: "9 LUNs found"
> 
> how to get more that 8 LUNs to map?
> 
> May be, some limitation somewhere in the code for MPT
?
> 
> any help would be appreciated.
> 

The following sysctl/tunable must be set:

kern.cam.cam_srch_hi=1

This can be set via either /boot/loader.conf,
/etc/sysctl.conf, or 
directly by running the sysctl command.

Scott
_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

adding 

kern.cam.cam_srch_hi=1

to /boot/loader.conf did not change anything.
still da52 remains the highest device. I would expect da53
to show up.

did I miss something ?

Dieter

 

> -----Ursprüngliche Nachricht-----
> Von: owner-freebsd-scsifreebsd.org 
> [mailto:owner-freebsd-scsifreebsd.org] Im Auftrag von
Scott Long
> Gesendet: Freitag, 7. März 2008 15:36
> An: d_elbracht
> Cc: freebsd-scsifreebsd.org
> Betreff: Re: only 8 LUNs on MPT
> 
> d_elbracht wrote:
> > we are using LSIs 3801E HBA to connect Infortrends
S16S-G1030+JBOD 
> > (e.g. 32 disks, 16 x RAID0, 16 LUNs) to an AMD64
machine.
> > 
> > the Infortrend Box is momentarily configured with
9 LUNs.
> > 
> > When booting only 8 LUNs (da45 to da52) show up
> > 
> > da45 at mpt0 bus 0 target 10 lun 0
> > da45: <IFT S16S-G1030 361F> Fixed Direct
Access SCSI-4 device
> > da45: 300.000MB/s transfers
> > da45: Command Queueing Enabled
> > da45: 1430288MB (2929229824 512 byte sectors: 255H
63S/T 182336C)
> > da46 at mpt0 bus 0 target 10 lun 1
> > .....
> > da52 at mpt0 bus 0 target 10 lun 7
> > da52: <IFT S16S-G1030 361F> Fixed Direct
Access SCSI-4 device
> > da52: 300.000MB/s transfers
> > da52: Command Queueing Enabled
> > da52: 1430288MB (2929229824 512 byte sectors: 255H
63S/T 182336C)
> > 
> > "camcontrol reportluns 2:10"
> > shows: "9 LUNs found"
> > 
> > how to get more that 8 LUNs to map?
> > 
> > May be, some limitation somewhere in the code for
MPT ?
> > 
> > any help would be appreciated.
> > 
> 
> The following sysctl/tunable must be set:
> 
> kern.cam.cam_srch_hi=1
> 
> This can be set via either /boot/loader.conf, 
> /etc/sysctl.conf, or directly by running the sysctl
command.
> 
> Scott
> _______________________________________________
> freebsd-scsifreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

> To unsubscribe, send any mail to 
> "freebsd-scsi-unsubscribefreebsd.org"

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

d_elbracht wrote:
> adding 
> 
> kern.cam.cam_srch_hi=1
> 
> to /boot/loader.conf did not change anything.
> still da52 remains the highest device. I would expect
da53 to show up.
> 
> did I miss something ?
> 
> Dieter
> 

Can you send me the output of 'camcontrol inq da45'.  It
might be that
this device is reporting itself as SCSI-2 instead of SCSI-3.
 Also, you
can verify that the tunable got set in the loader by
checking it with
the sysctl command.

Scott

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

the tunable is set:
sysctl -a | grep kern.cam.cam_srch
kern.cam.cam_srch_hi: 1

here is the output from 'camcontrol inq da45':

pass45: <IFT S16S-G1030 361F> Fixed Direct Access
SCSI-4 device
pass45: Serial Number 01F53A00000000472CF136000000
300.000MB/s transfers , Command Queueing Enabled 

Dieter


> -----Ursprüngliche Nachricht-----
> Von: owner-freebsd-scsifreebsd.org 
> [mailto:owner-freebsd-scsifreebsd.org] Im Auftrag von
Scott Long
> Gesendet: Freitag, 7. März 2008 16:34
> An: d_elbracht
> Cc: freebsd-scsifreebsd.org
> Betreff: Re: AW: only 8 LUNs on MPT
> 
> d_elbracht wrote:
> > adding
> > 
> > kern.cam.cam_srch_hi=1
> > 
> > to /boot/loader.conf did not change anything.
> > still da52 remains the highest device. I would
expect da53 
> to show up.
> > 
> > did I miss something ?
> > 
> > Dieter
> > 
> 
> Can you send me the output of 'camcontrol inq da45'. 
It 
> might be that this device is reporting itself as SCSI-2

> instead of SCSI-3.  Also, you can verify that the
tunable got 
> set in the loader by checking it with the sysctl
command.
> 
> Scott
> 
> _______________________________________________
> freebsd-scsifreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

> To unsubscribe, send any mail to 
> "freebsd-scsi-unsubscribefreebsd.org"

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

d_elbracht wrote:
> the tunable is set:
> sysctl -a | grep kern.cam.cam_srch
> kern.cam.cam_srch_hi: 1
> 
> here is the output from 'camcontrol inq da45':
> 
> pass45: <IFT S16S-G1030 361F> Fixed Direct Access
SCSI-4 device
> pass45: Serial Number 01F53A00000000472CF136000000
> 300.000MB/s transfers , Command Queueing Enabled 
> 
> Dieter
> 

What version of FreeBSD is this?  The max lun benhavior
changed a
bit in the SCSI layer a couple of years ago.  The MPT driver
itself
supports up to 256 LUNs.

Scott
_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

sorry about that, should have said that in the first place.

it is FreeBSD 7.0, CVSuped Feb 15, 2008

btw, you noticed it said SCSI-4 ?


Dieter 

> -----Ursprüngliche Nachricht-----
> Von: owner-freebsd-scsifreebsd.org 
> [mailto:owner-freebsd-scsifreebsd.org] Im Auftrag von
Scott Long
> Gesendet: Freitag, 7. März 2008 16:56
> An: d_elbracht
> Cc: freebsd-scsifreebsd.org
> Betreff: Re: AW: AW: only 8 LUNs on MPT
> 
> d_elbracht wrote:
> > the tunable is set:
> > sysctl -a | grep kern.cam.cam_srch
> > kern.cam.cam_srch_hi: 1
> > 
> > here is the output from 'camcontrol inq da45':
> > 
> > pass45: <IFT S16S-G1030 361F> Fixed Direct
Access SCSI-4 device
> > pass45: Serial Number 01F53A00000000472CF136000000
300.000MB/s 
> > transfers , Command Queueing Enabled
> > 
> > Dieter
> > 
> 
> What version of FreeBSD is this?  The max lun benhavior

> changed a bit in the SCSI layer a couple of years ago. 
The 
> MPT driver itself supports up to 256 LUNs.
> 
> Scott
> _______________________________________________
> freebsd-scsifreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

> To unsubscribe, send any mail to 
> "freebsd-scsi-unsubscribefreebsd.org"

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

On Fri, Mar 07, 2008 at 08:55:36 -0700, Scott Long wrote:
> d_elbracht wrote:
> >the tunable is set:
> >sysctl -a | grep kern.cam.cam_srch
> >kern.cam.cam_srch_hi: 1
> >
> >here is the output from 'camcontrol inq da45':
> >
> >pass45: <IFT S16S-G1030 361F> Fixed Direct
Access SCSI-4 device
> >pass45: Serial Number 01F53A00000000472CF136000000
> >300.000MB/s transfers , Command Queueing Enabled 
> >
> >Dieter
> >
> 
> What version of FreeBSD is this?  The max lun benhavior
changed a
> bit in the SCSI layer a couple of years ago.  The MPT
driver itself
> supports up to 256 LUNs.

I think the problem may be:

                cpi->max_lun = 7;

That's from line 3590 in rev 1.63 of mpt_cam.c.  (i.e.
-current)

I ran into the same problem a few days ago in RELENG_7, but
hadn't gotten
around to sending email about it yet.

The symptom I was having is all the LUNs on the target
wouldn't probe
automatically.  I could rescan them manually, but CAM would
stop probing at
7.

I fixed it by setting max_lun to 1024.  (Not sure if
anything would break
if I actually got that high, but my target was only setup
for 79 LUNs.)

Ken
-- 
Kenneth Merry
kenkdm.org
_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

Kenneth D. Merry wrote:
> On Fri, Mar 07, 2008 at 08:55:36 -0700, Scott Long
wrote:
>> d_elbracht wrote:
>>> the tunable is set:
>>> sysctl -a | grep kern.cam.cam_srch
>>> kern.cam.cam_srch_hi: 1
>>>
>>> here is the output from 'camcontrol inq da45':
>>>
>>> pass45: <IFT S16S-G1030 361F> Fixed
Direct Access SCSI-4 device
>>> pass45: Serial Number
01F53A00000000472CF136000000
>>> 300.000MB/s transfers , Command Queueing
Enabled 
>>>
>>> Dieter
>>>
>> What version of FreeBSD is this?  The max lun
benhavior changed a
>> bit in the SCSI layer a couple of years ago.  The
MPT driver itself
>> supports up to 256 LUNs.
> 
> I think the problem may be:
> 
>                 cpi->max_lun = 7;
> 
> That's from line 3590 in rev 1.63 of mpt_cam.c.  (i.e.
-current)
> 
> I ran into the same problem a few days ago in RELENG_7,
but hadn't gotten
> around to sending email about it yet.
> 
> The symptom I was having is all the LUNs on the target
wouldn't probe
> automatically.  I could rescan them manually, but CAM
would stop probing at
> 7.
> 
> I fixed it by setting max_lun to 1024.  (Not sure if
anything would break
> if I actually got that high, but my target was only
setup for 79 LUNs.)
> 
> Ken

Ok, I missed it on my first look since there's code
elsewhere in the
driver that specifically handles up to 16k LUNs as an
initiator and 256
LUNs as a target.  I guess the value in XPT_PATH_INQ is
there so that
CAM won't spend hours at boot-time scanning for all possible
LUNs.  I
could change it to something like 256, but that will still
significantly
increase the boot time.  What really needs to happen is to
have
REPORT_LUNS implemented for the probe.  Until that happens,
what I'd
suggest doing is to just manually scan your high lun.  I'll
see if I can
come up with a REPORT_LUNS patch, if anyone is willing to
play guinea
pig.  For the sake of safety with older broken SCSI devices,
I might
limit it to working only with FC initiators, only only with
devices that
report SCSI-4 or higher.  We'll have to play with that.

Scott

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"

Yes, I am willing to play guinea pig 

the Contoller (LSI 3801E) is not FC, it is SAS

if my recollection works, we had a similar issue (unsolved)
with Qlogic FC
2312, limited to 16 LUNs

the Code there is:
isp_freebsd.c:          cpi->max_lun = ISP_MAX_LUNS(isp)
- 1; 

'camcontrol inq da32' gives 
pass32: <IFT A24F-G2224-1 347B> Fixed Direct Access
SCSI-3 device
pass32: Serial Number 0C04C96131C15A00
200.000MB/s transfers , Command Queueing Enabled

so SCSI-4 Limitation may not be all.

Dieter




> -----Ursprüngliche Nachricht-----
> Von: Scott Long [mailto:scottlsamsco.org] 
> Gesendet: Samstag, 8. März 2008 16:57
> An: Kenneth D. Merry; d_elbracht
> Cc: freebsd-scsifreebsd.org
> Betreff: Re: AW: AW: only 8 LUNs on MPT
> 
> Kenneth D. Merry wrote:
> > On Fri, Mar 07, 2008 at 08:55:36 -0700, Scott Long
wrote:
> >> d_elbracht wrote:
> >>> the tunable is set:
> >>> sysctl -a | grep kern.cam.cam_srch
> >>> kern.cam.cam_srch_hi: 1
> >>>
> >>> here is the output from 'camcontrol inq
da45':
> >>>
> >>> pass45: <IFT S16S-G1030 361F> Fixed
Direct Access SCSI-4 device
> >>> pass45: Serial Number
01F53A00000000472CF136000000 300.000MB/s 
> >>> transfers , Command Queueing Enabled
> >>>
> >>> Dieter
> >>>
> >> What version of FreeBSD is this?  The max lun
benhavior 
> changed a bit 
> >> in the SCSI layer a couple of years ago.  The
MPT driver itself 
> >> supports up to 256 LUNs.
> > 
> > I think the problem may be:
> > 
> >                 cpi->max_lun = 7;
> > 
> > That's from line 3590 in rev 1.63 of mpt_cam.c. 
(i.e. -current)
> > 
> > I ran into the same problem a few days ago in
RELENG_7, but hadn't 
> > gotten around to sending email about it yet.
> > 
> > The symptom I was having is all the LUNs on the
target 
> wouldn't probe 
> > automatically.  I could rescan them manually, but
CAM would stop 
> > probing at 7.
> > 
> > I fixed it by setting max_lun to 1024.  (Not sure
if anything would 
> > break if I actually got that high, but my target
was only 
> setup for 79 
> > LUNs.)
> > 
> > Ken
> 
> Ok, I missed it on my first look since there's code
elsewhere 
> in the driver that specifically handles up to 16k LUNs
as an 
> initiator and 256 LUNs as a target.  I guess the value
in 
> XPT_PATH_INQ is there so that CAM won't spend hours at

> boot-time scanning for all possible LUNs.  I could
change it 
> to something like 256, but that will still
significantly 
> increase the boot time.  What really needs to happen is
to 
> have REPORT_LUNS implemented for the probe.  Until that

> happens, what I'd suggest doing is to just manually
scan your 
> high lun.  I'll see if I can come up with a REPORT_LUNS

> patch, if anyone is willing to play guinea pig.  For
the sake 
> of safety with older broken SCSI devices, I might limit
it to 
> working only with FC initiators, only only with devices
that 
> report SCSI-4 or higher.  We'll have to play with
that.
> 
> Scott
> 

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"