Traditional Unix backup strategies for OS X

I run a Unix shop with a large number of Sparcstations running Solaris- and two Macs running OS X.

My backup strategy is quite simple and robust for the Solaris machines. Each machine is represented by a subdirectory on a single centralized dedicated backup drive, which contains further subdirectories for Mon-Sat, Week1-Week5, and Monthly. Under cron, each night a level 7 ufsdump is run to put the incremental data into its nightly folder, every Sunday night a level 5 is done into the appropriate weekly folder, and once a month a full level 0 dump is done into the appropriate monthly folder. And after all the monthlies are done, the backup drive gets automatically duped onto an online hot spare, and the newly-duped copy gets manually taken offline and physically swapped through my safe deposit box: keeping a full set of everything available online at all times as well as offsite for disaster prevention. This methodology has paid for itself several times over, over the years.

This has proven to be reliable and bulletproof, as well as fully automated: this is how I want to do business. Unfortunately, there doesn't appear to be any good way to fit the Macs into this structure, since all indications are that the dump utility provided in OS X is intrinsically broken and does not handle the strangenesses of the Apple filesystem.

Are there any plans, either at Apple or in the opensource world, to implement proper dump functionality so that these Unix boxes can be fit in to existing, longstanding Unix backup strategies? I've looked at all the Mac "backup" utilities, and none of them seem to understand that simply snapshotting a filesystem is not enough: incremental restore may be critical in case of issues that go undetected for a few days or weeks. They may have nice pretty GUIs, but they don't do what is really needed, which is to provide a controllable combination of snapshot and incremental functionality. For a 20+ year Unix geek who has no fear of CLI tools and reading man pages, this is extremely frustrating...

Does anybody out there have good old cron-based dump/ufsdump for these machines, with no blinky GUI or excess bells and whistles, working? Or at least something that smells enough like a proper dump strategy that a dinosaur like me would be willing to use it? It seems sad to have Unix boxes that have omitted these most basic Unix utilities...

Posted on Nov 9, 2005 11:58 AM

Reply
21 replies

Nov 9, 2005 12:42 PM in response to Lee Cullens

Thanks, Lee. I've read portions of your article, and I'll reread it fully now- it's clear that you have done significant work in this area, and I thank you for the input.

I'm still hoping for the clean, simple, familiar approach allowed by ufsdump/ufsrestore. Interactive restores of files and directories is just flat trivial with ufsrestore -i, and I'd really love to be able to stay with what I know- that's just the way my mind works. Still, the alternatives you mention using rsync might well be survivable for me, despite the fact that deep down inside my brain still wants to be mounting reels of 1/2" tape... Perhaps there is life in this old dog yet! (;-)

Nov 9, 2005 2:03 PM in response to Lee Cullens

Well, dump is not (or at least has not been) HFS-aware, so it'll fail to properly handle anything with resource forks- or so I've read.

They've now made tar and rsync HFS-aware, but nobody's taken on dump/restore yet for some cuirous reason. And thus my original question, hoping to flush a developer out that is working on them- perhaps from the opensource world...

Nov 9, 2005 4:24 PM in response to Lee Cullens

Another rsync based strategy is Dirvish ( http://www.dirvish.org/). It is written in Perl and makes use of some of the niftier features of rsync. I believe Dirvish does much of what it covered in the Rubel article.

The caveat is that it is not written for Mac OS X, so any respect for HFS would have to come from the HFS friendly properties of rsync (10.4).

S

Nov 14, 2005 6:44 PM in response to Stephen Cristol

should you need HFS+ attributes on <10.4, check out psync.

<a class="jive-link-external-small" href="http://">http://www.dan.co.jp/cases/macosx/psync.html

personally, rsync to a dedicated box with the -E flag is a nice solution.

perhaps similar to dirvish (didn't look at the scripts) and related to incremental backups, check out the following link for an elegant, but not mac-centric, solution.

http://www.mikerubel.org/computers/rsync_snapshots/

-b

Nov 14, 2005 8:21 PM in response to Brant Faircloth

Hi Brant,
I think psync is an excellent suggestion even for Tiger. I understand that Mike Bombich's Carbon Copy Cloner uses psync "under the hood" and his latest version works on Jaguar through Tiger so it probably still uses psync. Given that CCC can make a perfect clone of a disk and both psync and rsync can do incremental backups, it seems that they could do everything that Scott wants.
--
Gary
~~~~
I've Been Moved!

Nov 18, 2005 5:45 PM in response to Scott Griffith1

Scott,

I pleaded with Apple ( http://www.apple.com/feedback/) under both OS X and OS X Server teams.

I would like to suggest everyone chime in with feedback to both teams with a polite well written request for these Darwin utilities to be updated.


Gary,

Has CCC a workaround for the Tiger bug? I have been using Super Duper free evaluation lately and its really neat, though full feature (checkpoint rollbacks) is a 19 dolla purchase.

Jan J.

Nov 18, 2005 7:05 PM in response to Scott Griffith1

Scott,
While CCC is too GUI for your needs Mike Bombich does (or did) explain how his tools work. Clearly ufsdump does a great job on Solaris. Apple did something different (go figure) with blessing system folders. To achieve a full system backup you will need to adapt. (yuk, I know, but most of us adapted from OS 9, now that's a real change!).
I did not see any mention of ditto, which I believe CCC uses as well.

By the way, I appreciate your post. While asking a good question you provided clear detail and usefull information. I may adapt some of your ideas in my shop.

Reese

Nov 18, 2005 9:26 PM in response to Jan Johannsen

Hi Jan,

> Has CCC a workaround for the Tiger bug?

I'm afraid you have the better of me; about what bug are you speaking? I use psync on Tiger and it works fine for me. Are you speaking of the difficulty in installing File::MacOS? That's not a bug. The code hasn't been updated for Tiger and a new option in GetFileInfo causes the test harness to choke. However, there's no problem with the installed code and you can easily get it to install with force. See Fwd: psync for tiger? (SOLVED).
--
Gary
~~~~
Have a nice diurnal anomaly.

Feb 2, 2006 10:18 AM in response to reese_

Well, I've finally found an adequate workaround for the time being, and I'll stay with it until somebody actually does do the port of dump/restore.

I've settled on using rnapshot built through Fink (as described here: http://www.rsnapshot.org/), sitting on top of a modified rsync built by hand (both the Tiger rsync and RsyncX appear to be broken, but a patch here got me one that appears to work adequately: http://www.lartmaker.nl/rsync/ ). Useful setup suggestions for exclude files and the like can be found here: http://www.afp548.com/article.php?story=20050219192044818 and here: http://www.debian-administration.org/articles/217 .

Running rsnapshot nightly/weekly/monthly under cron, very much as if I would normally run ufsdump on my other machines, works very well- even if restore is more an issue of clicking and dragging files than using the CLI as I would prefer.

At least I have decent *and usable* automated backups running now, although I'm still hoping against hope that one of the opensource teams will take on the old utilities for those of us Unix dinosaurs who would prefer to treat our Macs as "just another Unix box"... Some of us are just too old and set in our ways to adapt, seem like. (;-)


G5 2x2GHz, 4Gb RAM Mac OS X (10.4)

Feb 2, 2006 6:13 PM in response to Scott Griffith1

Hi Scott,
I'm kindof old-school myself, because I was taught by vets. However, I lean towards GNU's tar. Actually, I think several of their options fail to work as they should but I'm sold on listed-incremental backups. I think the folks at GNU have put their greatest efforts into that option and I think it's a winner. I use it with my Linux machines at school.

However, I'm not writing to sway you, but to help you improve your own solution. I learned about the patch you mentioned in the thread, rsync -E still broken. Also in this thread, TheQ offered his fixed rsync. During that discussion, he further improved his rsync to preserve access control lists. Combined with the fact Quinton made checksums work and addressed problems not even mentioned on the lartmaker page, I'm hoping that he's created the greatest version of rsync ever. He's started with rsync version 2.6.6 so this is certainly a possibility.

You could help verify that Quinton's rsync lives up to these hopes and even if it doesn't yet, I'm sure that he would respond to feedback. At this point in time you still have to e-mail him to get the latest version but he will definitely reply. He's a great guy and I would certainly love to see his rsync become the Mac standard.
--
Gary
~~~~
What this country needs is a good five cent nickel.

Feb 3, 2006 9:59 AM in response to Scott Griffith1

Scott,

Please keep me (us) posted. I'm revamping my backup::restore piece to be part of a "shared Mac OS X and Debian GNU/Linux experience" site. Thus it will include both OSs and as much as possible common *nix approaches.

One approach I'm thinking of getting back to checking out is rdiff-backup. I know it's available on both platforms and was wondering if anyone had any preliminary comments/cautions/experience on such?

Also, could anyone explain the difference (other than the obvious 🙂 between asr and the *nix ghost takeoffs like g4u/g4l? The *nix ghost takeoffs take forever copying every bit of a volume, where asr zips right along with volume level block transfers. Does asr check for and ignore HFS+ freespace where the *nix ghost takeoffs (mostly ignoring specific file systems) do not? I know one can zero fill freespace with the *nix ghost takeoffs and thus avoid copying it, but that's hardly a time savings.

Thanks,
Lee C


PS: It would be nice to have something common to OS X and Linux like Mondo. On my Linux box I can create bootable DVD system images for restoration, or even just an emergency bootable CD with a few tools.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Traditional Unix backup strategies for OS X

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.