LabResources:FiberChannel

Fiber Channel Tips

Fiber Channel HBA's

Emulex/LSI HBA

Naraapp01 has an LSI-based hba. It's a bit different from the adaptec HBA's in that information regarding the device (wwn, etc) is found in /proc/mpt/* rather than in /proc/scsi/_hba_. The driver that is used to support these cards is the mptscsih shipped w/ RHEL 3. There is no additional management software installed.

[root@naraapp01 ioc0]# cat /proc/mpt/ioc0/info 
ioc0:
  ProductID = 0x1001 (LSIFC919)
  FWVersion = 0x01000600 (fw_size=218452)
  MsgVersion = 0x0101
  FirstWhoInit = 0x00
  EventState = 0x01
  CurrentHostMfaHighAddr = 0x00000000
  CurrentSenseBufferHighAddr = 0x00000000
  MaxChainDepth = 0x4f frames
  MinBlockSize = 0x20 bytes
  RequestFrames @ 0xd52e0000 (Dma @ 0x152e0000)
    {CurReqSz=128} x {CurReqDepth=1023} = 130944 bytes ^= 0x20000
    {MaxReqSz=128}   {MaxReqDepth=2047}
  ReplyFrames   @ 0xd532c000 (Dma @ 0x1532c000)
    {CurRepSz=64} x {CurRepDepth=128} = 8192 bytes ^= 0x2080
    {MaxRepSz=128}   {MaxRepDepth=2047}
  MaxDevices = 32
  MaxBuses = 32
  PortNumber = 1 (of 1)
    LanAddr = 00:06:2B:0C:5A:64
    WWN = 200000062B0C5A64:100000062B0C5A64

On any host with this card, you should also install the MyStorage software from [[1][LSI Logic]]. The card in naraapp01 is the LSIFC919.

EMC LPe111-E 4Gb PCIe Fibre Channel Adapter

Naraapp's11-15 have these installed for chronopolis. Emulex tools: [[2][link]]

Storage Devices

ACNC RAID

Connecting

On narafs01, run 'minicom S0', hit ctrl-D and enter the default password.

Adding lun and wwn wwn-mapping

Select '3 Fibre Params -> 33 SAN Mask -> 333 Edit WWN table'. Edit an unused host mask and add the new WWN. You can use 'View WWN table' to locate an empty entry. After editing, esc back to the main menu and 'NVRAM -> Update NVRAM'. Now, map the lun you want, '3 Fibre Params -> 31 Primary FC -> 315 LUN Map' select lun to map, then array/slice to map to. When done, esc back and save NVRAM.

Now map the host to lun by '3 Fibre Params -> 33 SAN Mask -> 332 SAN Mapping -> 3321 Port1 ->' Select host then lun to allow/disallow. Esc back and save nvram. You can now use view san mapping to see if changes stuck. No raid reboot should be necessary.

AX150

The dual controllers are available on the san network as nara-ax150-1 and nara-ax150-2. Access is through an http interface. The password is the same as the naraapp account.

Adding new devices Before you can assign disks to a server, you must introduce the server to the ax150, this is accomplished by adding the 150 to the lun in linux

 echo "scsi add-single-device 2 0 3 0" > /proc/scsi/scsi

followed by running Navisphere util program. (look in /opt on narafs01 for it)

[root@naraapp01 root]# ./naviserverutilcli 
Welcome to Navisphere Server Utility - version : 6.20.0.3.12

Select from one of the following options: or Select '0' to exit the application

1. Update Server Information - Select this option to send information about the 
server to all connected storage systems

2. Start a Snapshot (Navisphere Express only) - Select this option to start a sn
apshot. Starting a snapshot begins the point-in-time copy of the virtual disk

3. Stop a Snapshot (Navisphere Express only) - Select this option to stop a snap
shot. Stopping a snapshot ends the point-in-time copy of the virtual disk

4. Allow Access to a Snapshot (Navisphere Express only) - Select this option to 
allow access to a snapshot

5. Remove Access to a Snapshot (Navisphere Express only) - Select this option to
 remove access to a snapshot

6. Display help for the application

1

Connected Storage Systems:

HBA/NIC Port  Storage System  SP  Port  SP IP Address  
-------------------------------------------------------
0             FCNPR070300149  A   0     192.168.88.18   


There are currently no volumes from external storage systems.

Please verify the information above.  If it is correct, you can update
the server with the attached storage systems.  If the information
is incorrect you can scan again and then update.

Please select [u]pdate, [s]can, [c]ancel: u

The update succeeded.

You should now see the new server listed on the web interface under Servers of connections. If not, you'll have to manually add it. You can do this by clicking 'Connections', selecting the wwn of the server you are adding, click modify and enter the name and ip of the server.

Kernel Tweaking

In /etc/modules.conf the following line usually needs added to probe all available luns:

options scsi_mod max_scsi_luns=255

After adding that line, or additional hba's to a machine, you should rebuild the initrd so the modules will be available on boot:

[root@naraapp01 ~]# cd /boot
[root@naraapp01 boot]# mv initrd-2.4.21-32.0.1.ELsmp.img initrd-2.4.21-32.0.1.ELsmp.img.bak
[root@naraapp01 boot]# mkinitrd initrd-2.4.21-32.0.1.ELsmp.img 2.4.21-32.0.1.ELsmp