Traditional Unix backup strategies for OS X

I run a Unix shop with a large number of Sparcstations running Solaris- and two Macs running OS X.

My backup strategy is quite simple and robust for the Solaris machines. Each machine is represented by a subdirectory on a single centralized dedicated backup drive, which contains further subdirectories for Mon-Sat, Week1-Week5, and Monthly. Under cron, each night a level 7 ufsdump is run to put the incremental data into its nightly folder, every Sunday night a level 5 is done into the appropriate weekly folder, and once a month a full level 0 dump is done into the appropriate monthly folder. And after all the monthlies are done, the backup drive gets automatically duped onto an online hot spare, and the newly-duped copy gets manually taken offline and physically swapped through my safe deposit box: keeping a full set of everything available online at all times as well as offsite for disaster prevention. This methodology has paid for itself several times over, over the years.

This has proven to be reliable and bulletproof, as well as fully automated: this is how I want to do business. Unfortunately, there doesn't appear to be any good way to fit the Macs into this structure, since all indications are that the dump utility provided in OS X is intrinsically broken and does not handle the strangenesses of the Apple filesystem.

Are there any plans, either at Apple or in the opensource world, to implement proper dump functionality so that these Unix boxes can be fit in to existing, longstanding Unix backup strategies? I've looked at all the Mac "backup" utilities, and none of them seem to understand that simply snapshotting a filesystem is not enough: incremental restore may be critical in case of issues that go undetected for a few days or weeks. They may have nice pretty GUIs, but they don't do what is really needed, which is to provide a controllable combination of snapshot and incremental functionality. For a 20+ year Unix geek who has no fear of CLI tools and reading man pages, this is extremely frustrating...

Does anybody out there have good old cron-based dump/ufsdump for these machines, with no blinky GUI or excess bells and whistles, working? Or at least something that smells enough like a proper dump strategy that a dinosaur like me would be willing to use it? It seems sad to have Unix boxes that have omitted these most basic Unix utilities...

Posted on Nov 9, 2005 11:58 AM

Reply
21 replies

Feb 3, 2006 2:09 PM in response to Lee Cullens

I'd be glad to. I'm also testing yet another patched rsync from here: http://www.quesera.com/reynhout/misc/rsync+hfsmode/ , so I'll just have a shootout of sorts and post the results.

Who knows- with all these diff files to use for guidelines, I might just have to grab my ancient BSD sources and patch up dump/restore myself. That ought to be ready in, oh, about 2016- at the rate that I write code... (;-)

Feb 3, 2006 6:18 PM in response to TheQ

Thanks Q,

I tried unison several months back on just my Mac. I didn't get into it enough at the time and had a bit of a chore just completely removing it from my Mac afterwards. I seem to remember they (some university?) had moved on to other things which gave me the idea that they might not keep up with any Apple tinkering.

I have received comments from some others that they were using it on a Mac successfully, and from even more others that they were using it on Linux platforms successfully.

If my perception is wrong I'd gladly get back into checking it out for inclusion, but right now I'm concentrating on native *nix tools and active open source projects that I think have promise.

Actually, right now I'm off on a tangent picking up enough CSS to make a decent site to offer up for initial comments and hopefully others content contributions. I never was much enthralled at this webbie thing 🙂

All suggestions and comments are greatly appreciated. I'll have passed on before I thoroughly cover and correctly document every decent possibility by myself ;')


User uploaded file Lee C
"Life is judged with all the blindness of life itself." -- George Santayana



Dual 2.5 Power Mac G5 Mac OS X (10.4.4) on LAN with Debian Linux

Feb 14, 2006 3:40 PM in response to Lee Cullens

Just as a followup: I've got both the Reynhout and QDolan versions of rsync working to a degree, and I can report that Q's seems more robust: it produces no error messages and very usable results on my Tiger Server installation. So I'm happy to report that it is now in production use on that machine.

I finally gave up on NFS mounting the dump volume from the existing Solaris servers down in the server room, primarily because it was taking too much time to get the mission-critical backup of my G5/Tiger Server desktop machine running. I acquired a pair of very compact and quiet 300GB Firewire drives, and have now moved the dump volume locally onto the G5 instead. I formatted these external drives as HFS+, so the qdolan rsync can very easily do a tidy local backup of the Tiger machine there, HFS-to-HFS.

The dump volume is then exported from the G5 via Samba and NFS. Thus, the Solaris machines can do ufsdump onto it by simply changing the server name in all my scripts to point at it, and the couple of trivial Windows boxes can do their backups using Acronis TrueImage onto it after a few clicks: all treating it as a normal UFS volume. So I can now downgrade to a much smaller safe deposit box for offsite storage. Those Firewire drives are downright teeny, and since they don't sound quite so much like hovercraft, I can tolerate having one under my desk. (;-) It doesn't matter a bit whether it is down in the server room or not.

I still have a little bit of tweeking to do to the QDolan version to get things working as I want on my Panther 10.3.9 Powerbook (the code currently doesn't build properly under Panther, and the prebuilt version dies with some library errors). But I suspect that I'll get there eventually. Right now, I'm using the Reynhout version with --hfs-mode=appledouble for that machine, since the NFS mount treats the dump volume as UFS. That gives me enormous numbers of error messages about chgrp on symbolic links, but does indeed produce a usable backup.

To answer the unasked question: I'm avoiding setting up SSH and rsync's client/server mode thus far simply because it irks me to have yet another layer of software to have to debug to do a simple dadgum dump. I'm not particularly SSH-savvy, and am going to keep trying to do what I want via the NFS mechanism that I know already, rather than trying to wade into the SSH void- since I also have to try to make a living in here somehow...

So I'll post more when I know more: but there is at least an existence proof that this can be done in a more traditional Unix dinosaur environment.

Feb 14, 2006 6:57 PM in response to Scott Griffith1

Following up on my own post: it would appear that for Panther owners, we'll need to stick with the Reynhout version for at least the time being. The QDolan version currently available depends upon the copyfile() system call, which is apparently absent altogether in Panther, as documented here: http://www.opendarwin.org/pipermail/darwinports/2006-January/031622.html and here: http://bob.pythonmac.org/archives/2005/05/03/mac-os-x-104-tigers-copyfile/ .

The Reynhout version includes a stub version of copyfile() that is used at build time if the expected system call is absent, and allows the resource forks to be copied under Panther. On the other hand, it complains mightily about not being able to chgrp symbolic links...

I've provided this info to Q to see if he might figure out a workaround in his own code, but Panther is rapidly disappearing from the landscape- making it a bit less useful undertaking than furthering Tiger support. But for those of us who are too bleedin' cheap to spring for Tiger on our older machines, the Reynhout version may suffice for the time being. I'll probably upgrade the laptop to Tiger to render this point moot, but others may need to stay with Panther for whatever reasons. Hope that helps!

Feb 19, 2006 12:51 PM in response to Scott Griffith1

Yet another followup to my own post: I spoke too soon about the Reynhout version of rsync working properly under Panther. My incrementals are missing a number of links, which leads to me think that there is a problem with --link-dest in that code.

My current plan is to update the laptop to Tiger so that I can use the QDolan version on both machines (and limit my own confusion). So I will be of little help with finding an optimal Panther solution...

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Traditional Unix backup strategies for OS X

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.