This discussion is locked
richlowe
Level 1 (0 points)

Q: Copying between Volumes very slow

We have two identical XSAN2 volumes each consisting of 2 xserve RAID chassis, 3 LUNS of x7 500GB RAID 5 disks for DATA and one LUN of x2 500GB RAID 1 disks for metadata. Both volumes are hosted on the same controller. When we copy the entire contents of 1 volume to the other it is taking aprox. 12 hours to copy 3.5TB of data. This seems very slow given the bandwidth of the xserves. It is almost as if the copy is happening over ethernet rather than fibre! Does anyone have any thoughts or advice on this?

Thanks in advance

Richard

MacPro, Mac OS X (10.6.3), XSAN 2.1.1, FCP 7, Blackmagic Eclipse

Posted on Jul 21, 2010 10:14 AM

Close
richlowe

Q: Copying between Volumes very slow

  • All replies
  • Helpful answers

  • by RobertKite,

    RobertKite RobertKite Jul 22, 2010 3:51 PM in response to richlowe
    Level 1 (120 points)
    Jul 22, 2010 3:51 PM in response to richlowe
    Hey Richard,

    It is likely that you are experiencing cache problems.

    Check the Activity Monitor on the controller you are using to do your copies. Make sure the amount of inactive memory is no more than 50% of the amount listed as free.

    Bob

  • by richlowe,

    richlowe richlowe Jul 23, 2010 4:10 AM in response to RobertKite
    Level 1 (0 points)
    Jul 23, 2010 4:10 AM in response to RobertKite
    Hi Bob, thanks for the reply. You could be onto something here. The free memory is showing 45MB with inactive memory showing 6.89GB (we have 8GB in total).

    Is there an easy way to clear the cache? Restart?

    I should also mention that both volumes are hosted by the MDC master, the copy is done on the MDC slave. The MDC slave is showing only 45MB free where as the MDC master is showing 6.29GB free (8GB total).

    Message was edited by: RL@ITV

  • by RobertKite,

    RobertKite RobertKite Jul 23, 2010 4:51 AM in response to richlowe
    Level 1 (120 points)
    Jul 23, 2010 4:51 AM in response to richlowe
    OK Richard,

    You need to purge the cache. However, to do this you will need to install xcode as the OS does not have the purge command by default. Xcode will install that tool.

    So,

    1. Install Xcode
    2. Open Terminal and type purge

    It may take a bit to complete but you will notice the inactive memory decreasing and performance improving.

    You can have Danielle James come up and automate this entire process for you. If you don't have her contact information ping the Xsan boys at Waterloo and they can provide the info.

    Cheers,

    Bob

  • by Alex Bourn,

    Alex Bourn Alex Bourn Jul 23, 2010 8:29 AM in response to richlowe
    Level 1 (5 points)
    Jul 23, 2010 8:29 AM in response to richlowe
    How are you doing the copy? Are you using the cvcp command? What kind of data? That timing doesn't necessarily seem so bad

  • by richlowe,

    richlowe richlowe Jul 23, 2010 9:17 AM in response to Alex Bourn
    Level 1 (0 points)
    Jul 23, 2010 9:17 AM in response to Alex Bourn
    I have a scheduled unix task set to run within ARD. We are using the following command

    cp -R -p /Volumes/XSAN1/* /Volumes/XSAN2

    If 3.5TB takes 12 hours to copy, this is a throughput of 81KB/sec. You can get 60MB/sec through gigE.

    I would have thought it would be faster using fibre.

    I have purged the memory as Bob suggested so I will monitor the next backup and see ho long it takes.

    Thanks

    Message was edited by: RL@ITV

  • by richlowe,

    richlowe richlowe Jul 26, 2010 3:41 AM in response to richlowe
    Level 1 (0 points)
    Jul 26, 2010 3:41 AM in response to richlowe
    Well after purging the inactive memory the copy still took 13 hours!

    Any other ideas?

  • by RobertKite,

    RobertKite RobertKite Jul 26, 2010 5:41 AM in response to richlowe
    Level 1 (120 points)
    Jul 26, 2010 5:41 AM in response to richlowe
    OK,

    The next step would be to initiate multiple copies at the same time.

    Check out this article. - http://www.xsanity.com/article.php/20081209162454331

    BTW: Just finished a 23 TB copy in 18 hours.

  • by Alex Bourn,

    Alex Bourn Alex Bourn Jul 26, 2010 9:35 AM in response to RobertKite
    Level 1 (5 points)
    Jul 26, 2010 9:35 AM in response to RobertKite
    Check out using cvcp rather than cp

  • by richlowe,

    richlowe richlowe Jul 28, 2010 2:00 AM in response to Alex Bourn
    Level 1 (0 points)
    Jul 28, 2010 2:00 AM in response to Alex Bourn
    Thanks for everyone's help on this. I will check out the cvcp command, however I have found that the latest version of Mike Bombich's Carbon Copy Cloner works with xsan volumes and can carry out scheduled incremental backups of volumes by adding a property list file in /Library/LaunchDaemons.

    The last incremental backup took 1 hour and 40 minutes.

    A result.

    Richard

  • by Christopher Collins3,

    Christopher Collins3 Christopher Collins3 Aug 10, 2010 9:40 PM in response to richlowe
    Level 1 (25 points)
    Aug 10, 2010 9:40 PM in response to richlowe
    Hey Richard,

    Actually your math is wrong. You're getting 80MB/sec, not KB/sec. 3,500,000(MB) / 43200(seconds) = 81 MB/sec. If it were KB, you'd only transfer about 3 and a half GBs in 12 hours.

    What kind of storage are you using? Promise or Xserve RAID? Keep in mind a RAID controller on an Xserve RAID only has 80MB/sec of theoretical bandwidth so each LUN would only be able to ever go 80 MB/sec anyway. Also, if you have medium to smaller sized files, that usually produces longer transfers then large files with big blocks of data.

    If most of my assumptions are true I really don't see your transfer speed as out of the ordinary. But I am making a lot of assumptions.