OSX Lion with ALUA support actually broke more than it fixed for us

Question

Level 1

0 points

OSX Lion with ALUA support actually broke more than it fixed for us

Hi

As you guys know Apple introduced ALUA in to Lion and when they bumped up the specs of Xsan. Most people seems to be really happy about this but in our case it actually broke more than it fixed. With our ATTO Celerity ES41 and LSI quad channel fibre cards we only see half of our LUNs. The explanation I got from the ATTO support was:

"The problem with this is that we don't really support connectivity to the LSI Netapp Engenio arrays with Multipathing on Lion with the 4Gb cards. Only the 8Gb cards are supported. The problem is we no longer can control this in Lion. In Snow leopard, there is an nvram switch to disable MPIO...In Lion, we can not do this due to Apple implementing ALUA in the OS itself. Our standard driver hooks into ALUA, and the MP Driver hooks into our own MP subsystem."

So the ATTO cards are out of the picture and ATTO wont come up with a fix. The jury is still out over at LSI and I*m waiting to hear back from them. But what I wonder is exaclty what it was that went wrong when Apple "fixed" this...? Is there a "legacy" mode that I can switch over to in some way? We can't replace all our fibre cards over this, that would be redicilous.

Cheers and thanks in advance

Various MacPros

Posted on Oct 28, 2011 6:32 AM

Reply

Answer 1

NEO_Sweded Author

Level 1

0 points

Oct 29, 2011 5:25 AM in response to NEO_Sweded

The jury is back in with some words from LSI. They won't comment on it or even try to help me since it's an OEM board (Apple). Geez.

Can one even get PRO Xsan support from Apple? Guess I have to try and call em on Monday but I don't have high hopes about it. =(

Reply

Answer 2

receng

Level 1

20 points

Nov 30, 2011 11:27 AM in response to NEO_Sweded

I am really insterested in this since I am considering EMC storage with ATTO cards and Mac OS X Lion. So basically you are forced to buy 8Gb cards from ATTO to solve the issue?

I would try fiddling with this NEW command line option in Lion's cvadmin command:

multipath pool(rotate|static)

Specify how Xsan uses multiple paths to astorage pool.

pool – the name of a storage pool in the activevolume

By the way, I looked at the StorNext command line reference and that option is not there for cvadmin.

Reply

Answer 3

NEO_Sweded Author

Level 1

0 points

Nov 30, 2011 11:39 AM in response to receng

Hello

Well... I have for a month try to get some sort of support from Apple but haven't really managed. When seeing the quality of support I can for sure understand that they are pulling out from the pro-market, if you can't support it you shouldn't be there. =/

According to ATTO their only solution is to change cards. If it wasn't for the case that support is broken even for the OEM LSI HBA I would probably stop nagging about this. But Apple it feels strange that Apple should change somehting that breaks support for one of their OEM HBA's...

Trying to reach the XSAN devs through back channels at the moment but no luck so far. =(((

That's an interesting find there! I will look at that on friday when I'm back at work. Stuffs aren't connected at the moment so I can't remote in to look at it.

Reply

Answer 4

Dec 27, 2011 3:29 PM in response to NEO_Sweded

I'm curious what the latest with this is, in particular if you've been able to get support - even if it's not the best news.

Reply

Answer 5

NEO_Sweded Author

Level 1

0 points

Dec 27, 2011 3:47 PM in response to Christopher Murphy

Hi Christopher

Are you having similar problems?

The Apple support have turned out to be one of the worst ever, makes you understand why they are pulling out of the pro market. =/ it have been so bad thatbthey should be ashamed of themselves and now they want me to buy another care pack or per incident support contract. They can not promise use me a solution even if I do and it feels a bit ridiculous since one of the devs would be able to give the answer to this within the blink of an eye. =(

So at the moment I'm still working some back channels. I have a good lead and am awaiting an answer from someone on the dev team. It have taken quite a while and I'm not sure if I will get an answer or not. =/ i will wait a wile longer and then I have to buy a per incident thingy. =(

I also haven't had the time to test the multipath command suggested above, will do as soon as I get back to work in January.

Happy holidays =)

Reply

Answer 6

Dec 27, 2011 4:19 PM in response to NEO_Sweded

I'm not having similar problems, I'm using it to estimate the viability of investing in a new Xsan setup vs other solutions. Any system can have problems, the question is whether or not there is responsive support and active development, and at the moment I'm just not seeing it with Xsan.

I suspect better support would be possible with Quantum as it is ultimately their technology being used here. I have no idea what Quantum's licensing is like, haven't looked into it, but I understand that Xsan uses the same StorNext clustered file system as Quantum. So possibly there is an upgrade path for an existing Xsan system.

Reply

Answer 7

Dec 27, 2011 4:21 PM in response to NEO_Sweded

See if their per incident support has some guarantee or not. If they can fix the problem, paying for an incident makes sense. If they're just going to tell you it can't be done, yet this isn't documented anywhere in system requirements for Xsan 2.3, it's worth very little. If you pay on an Amex, it's easier to do a chargeback if you don't get what you paid for - i.e. a problem that is solved.

Reply

Answer 8

Dec 27, 2011 10:52 PM in response to receng

receng wrote:

I would try fiddling with this NEW command line option in Lion's cvadmin command:

multipath pool(rotate|static)

Specify how Xsan uses multiple paths to astorage pool.

pool – the name of a storage pool in the activevolume

By the way, I looked at the StorNext command line reference and that option is not there for cvadmin.

That's not new and it doesn't do anything on a Mac. You probably want the mpioutil(1) command line utility. I'm not sure if there's a GUI for its features since I don't have any ALUA capable storage... maybe in Disk Utility?

Message was edited by: Blaidd Drwg Apparently you can't do a x-man-page link here.

Reply

Answer 9

NEO_Sweded Author

Level 1

0 points

Jan 26, 2012 4:56 AM in response to receng

Hello

I have to edit this sice what I posted was strange and wrong. =)

When looking in cvadmin its set to Multipath Metod: Rotate on both Lion and Snow Leopard. And since it works on Snow Leopard this can't be it.

Cheers

Reply

Answer 10

NEO_Sweded Author

Level 1

0 points

Jan 26, 2012 5:15 AM in response to NEO_Sweded

Here's some output from the clients... Let's start with the SNOW LEOPARD client:

mekfcp03s-mac-pro:~ mekfcp03$ sudo cvadmin

Xsan Administrator

Enter command(s)

For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS):

1>*nl01[0] located on 1.53.1.1:32850 (pid 10202)

Select FSM "nl01"

Created : Mon Dec 13 16:47:16 2010

Active Connections: 7

Fs Block Size : 16K

Msg Buffer Size : 4K

Disk Devices : 18

Stripe Groups : 6

Fs Blocks : 1280250440 (19.08 TB)

Fs Blocks Free : 821827938 (12.25 TB) (64%)

Xsanadmin (nl01) > show

Show stripe groups (File System "nl01")

Stripe Group 0 [SD0] Status:Up,MetaData,Exclusive

Total Blocks:1245184 (19.00 GB) Reserved:0 (0.00 B) Free:1155397 (17.63 GB) (92%)

MultiPath Method:Rotate

Primary Stripe [SD0] Read:Enabled Write:Enabled

Stripe Group 1 [SD1] Status:Up,Journal,Exclusive

Total Blocks:66128 (1.01 GB) Reserved:0 (0.00 B) Free:65104 (1017.25 MB) (98%)

MultiPath Method:Rotate

Primary Stripe [SD1] Read:Enabled Write:Enabled

Stripe Group 2 [DATA0] Status:Up

Total Blocks:448245140 (6.68 TB) Reserved:270720 (4.13 GB) Free:230827236 (3.44 TB) (51%)

MultiPath Method:Rotate

Primary Stripe [DATA0] Read:Enabled Write:Enabled

Stripe Group 3 [DATA1] Status:Up

Total Blocks:448245140 (6.68 TB) Reserved:270720 (4.13 GB) Free:312787356 (4.66 TB) (69%)

MultiPath Method:Rotate

Primary Stripe [DATA1] Read:Enabled Write:Enabled

Stripe Group 4 [DATA2] Status:Up

Total Blocks:191880080 (2.86 TB) Reserved:270720 (4.13 GB) Free:114779366 (1.71 TB) (59%)

MultiPath Method:Rotate

Primary Stripe [DATA2] Read:Enabled Write:Enabled

Stripe Group 5 [DATA3] Status:Up

Total Blocks:191880080 (2.86 TB) Reserved:270720 (4.13 GB) Free:163433980 (2.44 TB) (85%)

MultiPath Method:Rotate

Primary Stripe [DATA3] Read:Enabled Write:Enabled

Xsanadmin (nl01) > exit

mekfcp03s-mac-pro:~ mekfcp03$ sudo cvlabel -ls

/dev/rdisk6 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA00" Controller#: 'default' Serial#: '600A0B800056167A000003BB498B39EE' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk7 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA02" Controller#: 'default' Serial#: '600A0B80005045D8000001C5498B43B8' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk24 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA01" Controller#: 'default' Serial#: '600A0B80005616F80000039B498B3C19' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk21 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA03" Controller#: 'default' Serial#: '600A0B80005061D4000001B4498B45E1' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk23 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA04" Controller#: 'default' Serial#: '600A0B800056167A000003BD498B3A78' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk22 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA06" Controller#: 'default' Serial#: '600A0B80005045D8000001CB498B4442' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk25 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA07" Controller#: 'default' Serial#: '600A0B80005061D4000001B6498B4671' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk26 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA05" Controller#: 'default' Serial#: '600A0B80005616F80000039D498B3CA5' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk27 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA08" Controller#: 'default' Serial#: '600A0B800056167A000003BE498B3AFC' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk32 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA10" Controller#: 'default' Serial#: '600A0B80005045D8000001CC498B44CA' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk29 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA11" Controller#: 'default' Serial#: '600A0B80005061D4000001B7498B46FD' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk34 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA09" Controller#: 'default' Serial#: '600A0B80005616F80000039E498B3D37' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk31 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA14" Controller#: 'default' Serial#: '600A0B80005045D8000001CD498B4552' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk28 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA12" Controller#: 'default' Serial#: '600A0B800056167A000003BF498B3B88' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk30 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA15" Controller#: 'default' Serial#: '600A0B80005061D4000001B8498B478B' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk33 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA13" Controller#: 'default' Serial#: '600A0B80005616F80000039F498B3DC7' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

mekfcp03s-mac-pro:~ mekfcp03$ sudo cvadmin

Xsan Administrator

Enter command(s)

For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS):

1>*nl01[0] located on 1.53.1.1:32850 (pid 10202)

Select FSM "nl01"

Created : Mon Dec 13 16:47:16 2010

Active Connections: 7

Fs Block Size : 16K

Msg Buffer Size : 4K

Disk Devices : 18

Stripe Groups : 6

Fs Blocks : 1280250440 (19.08 TB)

Fs Blocks Free : 821827938 (12.25 TB) (64%)

Xsanadmin (nl01) > disks

Disks (File System nl01)

acfs Disk Volumes:

MEK_NL01_DATA00 on device:rdisk6 sectors: 3585963712 sector size: 512

MEK_NL01_DATA01 on device:rdisk24 sectors: 3585963712 sector size: 512

MEK_NL01_DATA02 on device:rdisk7 sectors: 3585963712 sector size: 512

MEK_NL01_DATA03 on device:rdisk21 sectors: 3585963712 sector size: 512

MEK_NL01_DATA04 on device:rdisk23 sectors: 3585963712 sector size: 512

MEK_NL01_DATA05 on device:rdisk26 sectors: 3585963712 sector size: 512

MEK_NL01_DATA06 on device:rdisk22 sectors: 3585963712 sector size: 512

MEK_NL01_DATA07 on device:rdisk25 sectors: 3585963712 sector size: 512

MEK_NL01_DATA08 on device:rdisk27 sectors: 1535042944 sector size: 512

MEK_NL01_DATA09 on device:rdisk34 sectors: 1535042944 sector size: 512

MEK_NL01_DATA10 on device:rdisk32 sectors: 1535042944 sector size: 512

MEK_NL01_DATA11 on device:rdisk29 sectors: 1535042944 sector size: 512

MEK_NL01_DATA12 on device:rdisk28 sectors: 1535042944 sector size: 512

MEK_NL01_DATA13 on device:rdisk33 sectors: 1535042944 sector size: 512

MEK_NL01_DATA14 on device:rdisk31 sectors: 1535042944 sector size: 512

MEK_NL01_DATA15 on device:rdisk30 sectors: 1535042944 sector size: 512

Xsanadmin (nl01) >

Reply

Answer 11

NEO_Sweded Author

Level 1

0 points

Jan 26, 2012 5:15 AM in response to NEO_Sweded

And this is the Lion client:

mekfcp06:~ mekfcp06$ sudo cvadmin

Password:

Xsan Administrator

Enter command(s)

For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS):

1>*nl01[0] located on 1.53.1.1:32850 (pid 10202)

Select FSM "nl01"

Created : Mon Dec 13 16:47:16 2010

Active Connections: 7

Fs Block Size : 16K

Msg Buffer Size : 4K

Disk Devices : 18

Stripe Groups : 6

Fs Blocks : 1280250440 (19.08 TB)

Fs Blocks Free : 821827938 (12.25 TB) (64%)

Xsanadmin (nl01) > show

Show stripe groups (File System "nl01")

Stripe Group 0 [SD0] Status:Up,MetaData,Exclusive

Total Blocks:1245184 (19.00 GB) Reserved:0 (0.00 B) Free:1155397 (17.63 GB) (92%)

MultiPath Method:Rotate

Primary Stripe [SD0] Read:Enabled Write:Enabled

Stripe Group 1 [SD1] Status:Up,Journal,Exclusive

Total Blocks:66128 (1.01 GB) Reserved:0 (0.00 B) Free:65104 (1017.25 MB) (98%)

MultiPath Method:Rotate

Primary Stripe [SD1] Read:Enabled Write:Enabled

Stripe Group 2 [DATA0] Status:Up

Total Blocks:448245140 (6.68 TB) Reserved:270720 (4.13 GB) Free:230827236 (3.44 TB) (51%)

MultiPath Method:Rotate

Primary Stripe [DATA0] Read:Enabled Write:Enabled

Stripe Group 3 [DATA1] Status:Up

Total Blocks:448245140 (6.68 TB) Reserved:270720 (4.13 GB) Free:312787356 (4.66 TB) (69%)

MultiPath Method:Rotate

Primary Stripe [DATA1] Read:Enabled Write:Enabled

Stripe Group 4 [DATA2] Status:Up

Total Blocks:191880080 (2.86 TB) Reserved:270720 (4.13 GB) Free:114779366 (1.71 TB) (59%)

MultiPath Method:Rotate

Primary Stripe [DATA2] Read:Enabled Write:Enabled

Stripe Group 5 [DATA3] Status:Up

Total Blocks:191880080 (2.86 TB) Reserved:270720 (4.13 GB) Free:163433980 (2.44 TB) (85%)

MultiPath Method:Rotate

Primary Stripe [DATA3] Read:Enabled Write:Enabled

Xsanadmin (nl01) > exit

mekfcp06:~ mekfcp06$ sudo cvlabel -ls

/dev/rdisk3 [ DC WD1001FALS-0 5.00K0] EFI Controller#: 'default' Serial#: 'unknown' Sectors: 1953506703. SectorSize: 512.

/dev/rdisk1 [ DC WD1001FALS-0 5.00K0] EFI Controller#: 'default' Serial#: 'unknown' Sectors: 1953506703. SectorSize: 512.

/dev/rdisk2 [ DC WD3200AAJS-4 8.01D0] EFI Controller#: 'default' Serial#: 'unknown' Sectors: 625123983. SectorSize: 512.

/dev/rdisk5 [KingstonDataTraveler G2 1.00] MBR Controller#: 'default' Serial#: 'unknown' Sectors: 15617839. SectorSize: 512.

/dev/rdisk0 [ DC WD1001FALS-0 5.00K0] EFI Controller#: 'default' Serial#: 'unknown' Sectors: 1953506703. SectorSize: 512.

/dev/rdisk10 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA03" Controller#: 'default' Serial#: '600A0B80005061D4000001B4498B45E1' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk6 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA01" Controller#: 'default' Serial#: '600A0B80005616F80000039B498B3C19' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk7 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA05" Controller#: 'default' Serial#: '600A0B80005616F80000039D498B3CA5' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk11 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA07" Controller#: 'default' Serial#: '600A0B80005061D4000001B6498B4671' Sectors: 3585963712. SectorSize: 512. Maximum sectors: 3585963712.

/dev/rdisk8 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA09" Controller#: 'default' Serial#: '600A0B80005616F80000039E498B3D37' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk12 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA11" Controller#: 'default' Serial#: '600A0B80005061D4000001B7498B46FD' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk13 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA15" Controller#: 'default' Serial#: '600A0B80005061D4000001B8498B478B' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

/dev/rdisk9 [ENGENIO INF-01-00 0623] acfs "MEK_NL01_DATA13" Controller#: 'default' Serial#: '600A0B80005616F80000039F498B3DC7' Sectors: 1535042944. SectorSize: 512. Maximum sectors: 1535042944.

mekfcp06:~ mekfcp06$ sudo cvadmin

Xsan Administrator

Enter command(s)

For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS):

1>*nl01[0] located on 1.53.1.1:32850 (pid 10202)

Select FSM "nl01"

Created : Mon Dec 13 16:47:16 2010

Active Connections: 7

Fs Block Size : 16K

Msg Buffer Size : 4K

Disk Devices : 18

Stripe Groups : 6

Fs Blocks : 1280250440 (19.08 TB)

Fs Blocks Free : 821827938 (12.25 TB) (64%)

Xsanadmin (nl01) > disks

Disks (File System nl01)

acfs Disk Volumes:

MEK_NL01_DATA01 on device:rdisk6 sectors: 3585963712 sector size: 512

MEK_NL01_DATA03 on device:rdisk10 sectors: 3585963712 sector size: 512

MEK_NL01_DATA05 on device:rdisk7 sectors: 3585963712 sector size: 512

MEK_NL01_DATA07 on device:rdisk11 sectors: 3585963712 sector size: 512

MEK_NL01_DATA09 on device:rdisk8 sectors: 1535042944 sector size: 512

MEK_NL01_DATA11 on device:rdisk12 sectors: 1535042944 sector size: 512

MEK_NL01_DATA13 on device:rdisk9 sectors: 1535042944 sector size: 512

MEK_NL01_DATA15 on device:rdisk13 sectors: 1535042944 sector size: 512

Xsanadmin (nl01) >

Reply