Backup using rsync

Question

Level 1

0 points

Backup using rsync

I am using rsync to backup some files from an external drive to my NAS. I am trying to skip hidden files, but it does not seem to be working. I am wondering if the OSX implementation is different from the normal.

rsync -vaE --progress --delete --exclude=".*" /Volumes/Storage/Media/ /Volumes/Backup/Media > /Volumes/Storage/Logs/"$(date "+%m%d%Y%H%M%S") [rsync | Media].log"

When tailing the log file, I see 1000s of hidden files being transferred. Also, they aren't just transferred once, they are being transferred every time I run rsync.

Here is an example like the ones in the log file:

Videos/DSLR/._MVI_5739.MOV

189 100% 9.71kB/s 0:00:00 (xfer#5622, to-check=78/26142)

Does anyone have any advice?

One other thing, diff I believe is showing clear (it is still running, but log is empty so far):

diff -rq --exclude=".*" /Volumes/Storage/Media/ /Volumes/Backup/Media/ > /Volumes/Storage/Logs/"$(date "+%m%d%Y%H%M%S") [diff | Media].log" &

MacBook Pro with Retina display

Posted on Oct 8, 2015 3:39 PM

Reply

Answer 1

Best reply

Mark Jalbert

Level 5

4,694 points

Oct 15, 2015 12:09 PM in response to Tsyras

AppleDouble format metadata (._files) will always be sent with the -E option. You cannot exclude them when you use the -E option.

Reply

Answer 2

Tsyras Author

Level 1

0 points

Oct 13, 2015 10:15 PM in response to Mark Jalbert

Thanks for the reply. I presume all standard metadata will be preserved if I remove the -E option (dates/exif/etc)? The -E from what I have read is for OSX only metadata.

Is there any real reason for me to keep the -E? For standard backup I guess the -a is enough?

Reply

Answer 3

Tony T1

Level 6

10,232 points

Oct 14, 2015 3:51 AM in response to Tsyras

Consider running dot_clean on the NAS drive after you rsync with -E

Reply

Answer 4

BobHarris

Level 9

53,045 points

Oct 14, 2015 6:26 AM in response to Tsyras

Not using the -E will mean you ONLY store the metadata supported by the destination file system. If the destination file system does not support it, you do not get it saved.

Reply

Answer 5

Oct 15, 2015 12:09 PM in response to Tsyras

Inode metadata- permission bits, owner, and group are only preserved if the destination file system supports that type of metadata. The -E option will preserve modification time and birth time better known as creation time. File flags are not preserved.

EXIF metadata is stored within the file and will preserved with or without the -E option.

Without the -E option (short list):

ACL's are not preserved.

finder_info is not preserved (Apple still uses creator codes and file types, some applications may still use them).

Resource forks are not preserved. (Yes, Apple uses them internally for Finder aliases, web archives).

So, you need to determine if it is safe for you not to copy OS X metadata. It depends on how you use the OS. Personally, I just use the -a option.

Reply

Answer 6

Tsyras Author

Level 1

0 points

Oct 14, 2015 9:12 AM in response to Mark Jalbert

Thanks for all replies. I think it is safe for me not to use -E. This is just for personal files, and the only ones I truly consider REALLY important are my photos and videos. I use Lightroom for photos, which sorts on EXIF data. I assume there is something similar buried within the video files (as of this point I haven't organized or edited any of my videos, just dumped then all in a folder to do it later)?

What I have been doing is using the NAS as my workspace and using Synology cloud sync to backup to Google Drive. As this was pretty slow, I bought a Thermaltake enclosure and a WD Red Pro and transferred all files to it. I can already tell how much better doing work over USB3 vs my NAS is. Now I want to do a cron job rsync to my NAS (the same folders I was using before) nightly for backup purposes. Then Synology cloud sync will send it to the cloud.

It is a Synology NAS just as an FYI. I believe its file system is ext4.

Reply