Need to back up 7TB RAID. What would you do?

Realistically tape is your only real option IMHO. If you're in the rare situation that you could house another RAID in an off-site location and run fibre to it then it is an outlier solution but not what I would recommend. Each tape format has its adherents but our department has changed formats with each major IT overhaul (VXA, SDLT, LTO3) so I've never been too attached to any certain format.

You don't say much about your infrastructure besides the RAID itself. If you already have a fibre channel hub then I would stick with fc for your back-up device for the speed - especially if you want to run the whole 7TB in a timely manner. SCSI is the main interface for many of the other tape devices and in most cases that means getting a card installed. This is less of a big deal in a tower but can be a pain and/or impossible in an Xserve depending on what you already have installed.

We have a two Xserves, single XRAID set-up with fibre channel hub. We went with the Exabyte 21 tape library with a single LTO3 drive running fibre channel. Exabyte now offers more selection in their FC offerings and they are worth a look. Most of their libraries also offer dual tape drives for better back-up speed. Ours is 3U and I wish they had the smaller form factor (and the smaller price) when I got mine. Apple resells Exabyte's FC options so you can shop there too.

The size of LTO3 (400/800GB) is good and the cost/GB is good as well. The LTO4 spec is about to drop but probably won't become a reality in time for you to go that way. (Assuming you'd use a brand new tape format for long term archiving anyway.)

Whatever solution you turn to don't forget to estimate the throughput of your system and the drive/tape speed and the time it will take to back up the amount of data you have. Doing full 7TB back-ups is not going to be fast and if you do them on a weekly or bi-weekly basis you need to be realistic about how long it will take. And usually any business that can generate 7TB of data could easily generate 7 more over time.

Good Luck,

=Tod

G5/2.0x2, Dual XServes x2, XRAID, beige G3 501Mhz

What kills me is that the VXA Packetloader we have is available in SCSI, but for some reason, they went with FW800, which I have been gathering, is a problem.

If you like the machine is there anyway to replace the interface? Or contact the company and tell them you want to turn it in for an "upgrade"? Just a couple scrimping thoughts...

As for backups, what I'd like to do is one big "archive" backup of the entire RAID and then incremental backups of new material as it's generated.

We have two back-up paths: Internal and Off-site.
-The internal is complete back-up + daily incrementals and this is designed to be able to recover from user error like accidental deletion etc. We do full back-ups at 1/2 our recovery interval and have 3 sets of tapes in rotation.
-Off-site is a complete back-up followed by differential back-ups in rotating sets and this is just for disaster recovery. Differential A gets run and goes off-site and the last B set comes back to get used next week. At worst between the original set and the last differential we are 6 days out. Because off-site back-ups imply that the RAID itself and our local back-ups were destroyed I'm not so worried about that gap because we are looking at replacing hardware etc as well. I do mine on Thursday night and do the transfer on Friday so I go home on the weekend knowing that we're backed-up. There is no set schedule (perhaps there should be) for when fulls get done but generally when the incremental climbs to two tapes (four between both sets) I look for a time to do it.

A couple things to keep in mind as you make a plan:
1) Set a time limit on how long you are willing to be able to "recover" incremental versions of files. If you commit to a "forever" recovery time you need to keep incremental tapes stringing on forever. At least in terms of incremental recovery tell them it is 6 months or a year or whatever but you are setting yourself for heartache if you don't. (Explain it in terms of never-ending tape costs to your boss to help with buy-in.)
2) Test your back-ups before you trust that the data is there. Create files and delete them to see if you can recover them before you save data to tape and erase it from the harddrive. Doing recovery will also help realize what you're in for if you ever need to do it for real and help you develop your plan.
3) Keep a current archive of your recovery software along with documentation about your server set-up with your off-site back-ups. Think about what you would want to have if you ever really needed to go to your disaster back-ups. If you really ever need them you will be glad you did.
4) Think about the security of your off-site data and what data you have there. Is your data valuable to someone else? If I can recover all the server passwords from a back-up tape I can make you very unhappy.

Hate to be doom and gloom but disaster planning is just that way.

Good luck,

=Tod

G5/2.0x2, Dual XServes x2, XRAID, beige G3 501Mhz

A couple more points that I think should be considered:

5) If the backups are for DR (disaster recovery), you can assume that the server that did the backups is no more. Therefore, whatever database of which file is where that the BU software keeps (probably on the root drive) is gone, so you'd be wise to buy at least one, and in colder climates two, drives that match your root drive and clone them in rotation to be sent offsite also. The added benefit is that you just pop in the drive to the new machine and reconfigure a couple of minor things like IP addresses, and you're fully good to go.

6) With 7TB of stuff to back up, the needed backup window will probably be huge. D2D2T (disk to disk to tape, copying from the working filesytem to another filesystem and then writing the data to tape so that the working filesystem isn't tied up any longer than needed) is a definite possibility. The low cost of raid vs. the cost of tape drives could make this much cheaper.

Roger

While I do find our storage needs ever-growing, I'm also noticing that as we grow, more and more data gets classified as "I need this on the server forever, but I'm never gonna actually use it again".

I'd evaluate what sort of load you have on the 7TB of "long term storage" rather than active working data... We use a DVD 'live' archiving/HSM solution for our long term data which also keeps the costs low for that data - and disaster recovery is as easy as duplicating ~400 DVD+R DLs every so often. ANYTHING you can do to age out/archive historical data will in turn reduce the load on your backup/DR solution.

For our backups, we run incremental snapshots every 12 hours from our 7TB SAN to a 14TB backup server hosted at our colo... If we did it any less often it would take forever to complete, but as is, most snapshots take just an hour or two of transfer, plus another hour of file compares.