Rsync and extended attributes

Hi all

First time poster, long time lurker.

I have a hopefully simple question about rsync. I'm trying to use it to backup my home computers (1 Windows XP (working okay), 1 MacBook Pro) to a NAS I have built running a rsync server. The NAS is running the FreeBSD-based FreeNAS. All I really need to backup is my User folder; occasionally I do a CCC of the whole drive for disaster recovery.

What I need to know is, is it important to backup extended attributes/resource forks? I understand that Apple managed to add this functionality to rsync, though rather poorly it seems, and it only works when syncing to another Mac running the same version of rsync, or another server running a patched version of rsync. I thought that extended attributes are only used in OS9, and I have no OS9 programs or documents on my Mac, and it seems that without patching the version of rsync on the NAS (something I'm unlikely to be able to do) I won't be able to take advantage of the feature with the setup I have.

If it's important to backup extended attributes, it is possible to use the version of rsync that Quinton Dolan has patched, that saves the extended attribute/resource fork as a separate file, but if I don't need it I'd prefer not to go down this route.

Any guidance gratefully appreciated.

MacBook Pro, Mac OS X (10.4.10)

Posted on Oct 29, 2007 11:56 AM

Reply
5 replies

Oct 30, 2007 10:37 AM in response to AlanNYC

That's funny, I thought that an "@" sign meant that it was a symbolic link? I can't find the post, maybe you could linky it?

Anyway, I've done some research on extended attributes, and I now know what they are (please correct me if I'm wrong); an extra part that can be applied to any file and store any meta data that anyone cares to add. Basically, you can use it to be far more descriptive about a file that what's available in "Get info", and can look it up without having to open the file itself. Network admins seem to think this is a fantastic idea, as they can use the extended attribute to very specifically define who is and who isn't allowed access to files, but there's lots of other applications for this besides security. Here's a link with a pretty good explanation: http://arstechnica.com/reviews/os/macosx-10-4.ars/7

I'm not really sure how widely used extended attributes are. I can't seem to find any just by randomly looking through directories in console, making sure I use "ls -a" to list files with a leading "."; I think they should start with "._" and then the same filename as the file they refer to.

So it seems that extended attributes are different to Mac OS9 resource forks. However, I don't use OS9 anymore, so I'm not going to worry that the BSD-based NAS I'm using doesn't recognise these either. On the whole, I'm going to ignore both issues until someone tells me different!

Which brings me finally to Rsync. Gosh, it's annoying. The Apple implementation is flawed because of it's tacked on support for extended attributes. I didn't even try using it. I downloaded RsyncX and played around with that, even managed to do a backup of my important folders in my Users folder, but not without plenty of mysterious "Code 2" errors stopping the backup every so often (about every 500MB it transferred to the NAS). Once everything was on the NAS, I could do a single rsync batch of folders, and it would pass through the whole thing, only updating the things I'd changed - just like it was supposed to do. I'm going to have to read up more about how to set it up to automate the backup though. I have checked the contents of the backup and it all seems to be there, I can browse my iPhoto library from my PC, and the file sizes all seem about right. So I guess part of the job is done.

However, RsyncX also seems a little flaky, and support seems pretty non existent. I think I'm going to try the patched version from http://www.onthenet.com.au/~q/rsync/ which is apparently patched to work with non-OS X filesystems and copies the extended attributes properly. It may get over the problem of RsyncX stopping every so often too. I think in the end the best solution to backup a Mac, assuming you're trying to back it up onto a server and not an external HD, is to make sure that server is another Mac, and if you're going to use rsync, makes sure that the version is the same in both locations. Then you won't have to worry about extended attributes as it'll support them automatically.

Oct 30, 2007 11:15 AM in response to Ginjaian

Here's the thread: <http://discussions.apple.com/thread.jspa?threadID=1202723>.

ls -e is the trick to seeing the full set of attributes with a directory listing.

The reason I suggested trying the iPhoto library is because it's presented in the Finder as (I think) a package of some sort with some other weird extended attributes. I wasn't worried about the individual components of the package rsync'ing, but I wonder what happens when you restore it back to the Mac.

If the extended attributes are lost, I'll be the library now looks like a plain old directory. And does iPhoto know what to do with it? Does it turn it reapply the attributes, or just work with it as a directory with no ill consequences?

I know that this has been a problem with Quicken data files (i.e., packages).

Oct 30, 2007 6:44 PM in response to AlanNYC

Hi Alan

I've had a look at my iPhoto folder, and it seems just like a normal folder. I can't see any "@" or "+" on the attributes. I did this by using "ls -elR". I think this should show any extended attributes, and by doing it recursively (-R) I viewed the whole iPhoto folder tree. The "-l" shows the attributes. Not sure about "-e", in the "man ls" it says "-e Print the Access Control List (ACL) associated with the file, if present." Here's my iPhoto Library root directory, nothing odd looking to me! It isn't a package, just looks like a regular folder. Remember I'm using 10.4.10 not Leopard, that other thread they're all on Leo.

./iPhoto Library:
total 245288
-rw------- 1 Ian Ian 1434435 Oct 28 22:16 AlbumData.xml
drwx------ 3 Ian Ian 102 May 27 12:33 Data
-rw------- 1 Ian Ian 68 Oct 28 22:15 Dir.data
-rw------- 1 Ian Ian 20 Nov 13 2006 Library.data
-rw------- 1 Ian Ian 20 Nov 13 2006 Library.iPhoto
-rw------- 1 Ian Ian 14821323 Oct 28 22:16 Library6.iPhoto
drwx------ 4 Ian Ian 136 May 27 12:33 Modified
drwx------ 4 Ian Ian 136 May 27 12:33 Originals
-rw------- 1 Ian Ian 15618048 Oct 7 22:05 Thumb32Segment.data
-rw------- 1 Ian Ian 62459904 Oct 7 22:05 Thumb64Segment.data
-rw------- 1 Ian Ian 31232000 Oct 7 22:05 ThumbJPGSegment.data
-rw-r--r-- 1 Ian Ian 48 Oct 7 22:05 iPhoto.ipspot
-rw-rw-rw- 1 Ian Ian 0 Nov 13 2006 iPhotoLock.data

Maybe with Quicken you'll just have to get used to archiving the package to a zip file, then back that up. Were all the @ showing up on your 10.5 system? I notice you've switched back to 10.4.7 for your last post! I won't restore my iPhoto directory at the moment, but I'm going to rebuild another Mac, so I'll try and restore it to that in the next couple of days.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Rsync and extended attributes

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.