8 Replies Latest reply: Apr 22, 2010 4:27 PM by MrHoffman
kaidomac Level 1 Level 1 (55 points)
Setup:

1. (Server) Mac Pro w/ Snow Leopard Server (contains SAS RAID storage system)
2. (Workstation) Mac Pro w/ Snow Leopard 10.6.2
3. (Workstation) Mac Pro w/ Leopard 10.5.8

Currently using a Gigabit network; the goal is to increase file transfer speeds. Ideally I'd like to setup a Fiber share between the systems. I've only worked with CAT5 networks in the past, so this is new territory for me. From what I understand, I'll need:

1. Quad-port Fiber card for the Server
2. Dual-port Fiber card for each Workstation
3. Secondary Ethernet network setup for metadata

The end goal is to have a Server shortcut or HDD icon on the desktop for easily & quickly saving large files to. Everything is stored & backed up on the server currently; Gigabit is just a bit slow for transferring HD/2K+ video content. I don't think I need XSAN for this type of setup (Fiber), but I may be wrong. What I'd really like is to have everything stored on the server, with all video editing and file operations done on the server over Fiber. Any insight & guidance would be appreciated, thanks!

Message was edited by: kaidomac

null

Mac Pro, Mac OS X (10.6.2)
  • 1. Re: Fiber Network for File Sharing?
    Strontium90 Level 4 Level 4 (3,140 points)
    kaidomac wrote:
    Setup:

    1. (Server) Mac Pro w/ Snow Leopard Server (contains SAS RAID storage system)
    2. (Workstation) Mac Pro w/ Snow Leopard 10.6.2
    3. (Workstation) Mac Pro w/ Leopard 10.5.8

    Currently using a Gigabit network; the goal is to increase file transfer speeds. Ideally I'd like to setup a Fiber share between the systems. I've only worked with CAT5 networks in the past, so this is new territory for me. From what I understand, I'll need:

    1. Quad-port Fiber card for the Server
    2. Dual-port Fiber card for each Workstation
    3. Secondary Ethernet network setup for metadata

    The end goal is to have a Server shortcut or HDD icon on the desktop for easily & quickly saving large files to. Everything is stored & backed up on the server currently; Gigabit is just a bit slow for transferring HD/2K+ video content. I don't think I need XSAN for this type of setup (Fiber), but I may be wrong. What I'd really like is to have everything stored on the server, with all video editing and file operations done on the server over Fiber. Any insight & guidance would be appreciated, thanks!


    The simple answer is you need Xsan. But you have a number of problems with the gear you have. The SAS storage will not work. You need Fibre storage. Plus, you will need an FC switch to connect it all together. FC peer connections are for target/initiators only, like when you connect a storage device to a host. Trying to connect multiple hosts will result in kernel panic and volume damage. Xsan provides the clustered file system needed to pull this off.

    Have you tried link aggregation on your workstations and server? Jumbo frame support? These will speed your ethernet transactions but not achieve your centralization goals. Xsan in your only solution to centralize the content and achieve significant performance increases.

    Hope this helps
  • 2. Re: Fiber Network for File Sharing?
    MrHoffman Level 6 Level 6 (12,455 points)
    Or mayhap 10 GbE NICs and a matching network switch?
  • 3. Re: Fiber Network for File Sharing?
    kaidomac Level 1 Level 1 (55 points)
    Oops, I omitted the Fiber Switch from my list. To clarify, basically the SAS Extenders are on the main Mac Pro with Snow Leopard Server. It's being upgraded shortly to a 40TB RAID 60 array, so it will be one giant "Hard Drive" on the Mac Pro Server. That's what I meant about SAS Extenders.

    So the idea is to connect two Mac Pro workstations to my Server Mac Pro via Fiber to increase file transfer speed, which is what I'm wondering about - is it possible to hook those three up via Fiber using Fiber cards, a Fiber switch, and a secondary Ethernet network for metadata. I don't know if I'd need a pair of metadata controllers for that setup, like XSAN requires (as I understand), or what, or if that setup is even possible. Basically I'd just like to use Fiber for file transfers from an existing large RAID array.
  • 4. Re: Fiber Network for File Sharing?
    MrHoffman Level 6 Level 6 (12,455 points)
    FC SAN doesn't typically do host to host communications; there was some thought of that off and on, but it never really reached fruition.

    You'll either need to go fully SAN or build out you your network, or both.

    For better or worse, a SAN is a good place to stuff all your storage traffic

    But a SAN isn't a fast general-purpose host-to-host network.
  • 5. Re: Fiber Network for File Sharing?
    kaidomac Level 1 Level 1 (55 points)
    Rats - I figured it wasn't possible due to the lack of information online. XSAN it is! Reading through the manual today, so let's see if I have this right:

    1. One fiber card per system (quad on the server?)
    2. Fiber switch
    3. Fiber cabling
    4. XSAN software
    5. (2) Mac Mini Servers (dual Metadata controllers, need an extra USB Ethernet each?)
    6. Secondary router & wiring for XSAN metadata network

    I'm wondering if I can get away with hosting the metadata controller on the server itself, and foregoing a second server. Just getting my feet wet here; I'll finish reading the manual over the next few days to get more familiar with it.
  • 6. Re: Fiber Network for File Sharing?
    MrHoffman Level 6 Level 6 (12,455 points)
    Fibre Channel SAN is an optical disk and tape connection. Regardless of the N in SAN, it's not a general-purpose network and it cannot be bridged with an Ethernet connection. The sites that do need to reduce their cabling or to extend a SAN tend to go the other way; they bridge the FC over an IP network, and run it over 10 GbE or over SONET or such. And you (still) end up with a FC bridge on each end of that intervening IP connection.

    The Mac Mini boxes are sweet boxes, but they're not known for their storage I/O performance nor for high-bandwidth expansion. In particular, the Mac Mini doesn't have a PCI slot for the HBA out to the FC SAN, and Xsan requires the FC HBA.

    The two boxes that are usually connected to a SAN are Mac Pro and particularly Xserve.

    The connection from those boxes to the (other) clients is then made via GbE or 10 GbE networking; via disk services over Ethernet.

    In addition to the FC SAN and SMB or AFP storage services, there's also iSCSI and other such; you'd need an iSCSI initiator on the Mac boxes, a (fast) network connection, and an iSCSI target on your storage server. This iSCSI stuff is arguably the most analogous to the FC SAN I/O stack while still operating over an Ethernet connection; it's the SCSI protocols over IP, rather then sending the SCSI protocols over the FC SAN connection.

    FWIW, some SAN storage controllers and storage shelves do accept SAS and SATA gear, and some vendors do have an upgrade path for that stuff. That's been an up-sell feature for (some of) the HP SAN gear for eons; buy the SCSI or SAS/SATA shelves and other such, and re-use them on a SAN controller.

    nb: I've not tried running the HP SAN gear with Mac gear, and I don't know that Promise offers this upgrade path.
  • 7. Re: Fiber Network for File Sharing?
    kaidomac Level 1 Level 1 (55 points)
    Just popping back in after doing some research. Sounds like 10Gb Ethernet is the way to go. CAT6 switches will be out later this year from what I've heard, so switching should be pretty easy.

    I know AFP/SMB aren't the fastest transfer methods out there - for files on our LAN, FTP is the fastest transfer method, but that doesn't play nice when opening files within programs. AFP looks like the best option - anyone know of something faster but equally transparent? NFS seems to be about the same performance as well.
  • 8. Re: Fiber Network for File Sharing?
    MrHoffman Level 6 Level 6 (12,455 points)
    There are various file-serving protocols that operate over Ethernet; SMB/CIFS, AFP, etc.

    There are also more block-oriented protocols such as iSCSI, if you have an iSCSI initiator for the host and an iSCSI target for the storage.

    If you go for 10 GbE, you'll obviously need the switch and the NICs throughout your I/O path; the host controllers and storage controllers.

    10 GbE has been around for a while in higher-end boxes and environments; switches are available, and prices are going to continue to drop.