Gnarlodious

Q: rsync not deleting files

I’ve been using rsync for years to backup to an external disk, currently version 3.1.1 in OSX 10.9.5. Some time in the past year rsync stopped deleting nonexistent files, but only in some folders. For example, in my external disk’s ~/Movies/ folder I see files and folders I deleted months ago that are still there. Meanwhile in the Desktop folder files get deleted normally. This is happening on my two backup Firewire disks so it is not peculiar to the disk.

 

The only thing I could think of is that permissions on the Movies folder look like drwxr-x—@ while on the Desktop they look like drwxr-x——. So I thought it was corrupted ACLs, so I booted into Recovery and ran resetpassword and reset permissions on my userfolder but it didn’t help. I searched the internet thinking maybe it was a known breakage but found no such clues. http://rsync.samba.org/ shows no broken OSX syncing, and a bugfix list for a candidate verson doesn’t mention anything.

 

I do have verbose setting so I see what is being deleted, but rsync doesn’t even see files missing on the source that exist on the destination. There are no error messages pertaining to those files. The command I use looks like this:

rsync -aAEHSWX --progress --delete-excluded --exclude-from=/Volumes/$USER/.rsyncBackup /Volumes/$USER/ /Volumes/Backup/

 

I even added --delete-excluded in case something in my exclusion file was matching some files.

 

Does anyone know what’s wrong?

MacBook Pro with Retina display, OS X Mavericks (10.9.5), Thunderbolt display, iPhone 5S

Posted on Oct 29, 2015 7:21 PM

Close

Q: rsync not deleting files

  • All replies
  • Helpful answers

  • by BobHarris,

    BobHarris BobHarris Oct 29, 2015 7:36 PM in response to Gnarlodious
    Level 6 (19,405 points)
    Mac OS X
    Oct 29, 2015 7:36 PM in response to Gnarlodious

    If you want to see ALL the access rights

    /bin/ls -dleO@ /path/to/file/or/directory

    Then see:

    man chmod

    man chflags

    man xattr

  • by Tony T1,Solvedanswer

    Tony T1 Tony T1 Oct 30, 2015 6:34 AM in response to Gnarlodious
    Level 6 (9,249 points)
    Mac OS X
    Oct 30, 2015 6:34 AM in response to Gnarlodious

    Are you getting any rsync errors? (not just pertaining to those files.  I believe rsync will not delete if there are any errors)

    You can try —ignore-errors (delete even if there are I/O errors)

  • by Gnarlodious,

    Gnarlodious Gnarlodious Oct 30, 2015 8:57 AM in response to Tony T1
    Level 4 (3,239 points)
    Mac OS X
    Oct 30, 2015 8:57 AM in response to Tony T1

    Thanks, Tony. Its apparently working using --ignore-errors, mostly. I also ran a command to delete all ACLs and that helped too. Apparently certain softwares install support files setting their own ACL and that excludes rsync access. I even tried running as sudo and it didn’t work right. Maybe I should try running rsync as root user.

     

    I think I can see the end of my rsync usage for HFS backups. It has been a headache for years and there’s no end in sight.

  • by Tony T1,

    Tony T1 Tony T1 Oct 30, 2015 9:26 AM in response to Gnarlodious
    Level 6 (9,249 points)
    Mac OS X
    Oct 30, 2015 9:26 AM in response to Gnarlodious

    Did you apply the following patches to your rsync build:

         fileflags.diff

         crtimes.diff

         hfs-compression.diff

  • by BobHarris,Helpful

    BobHarris BobHarris Oct 30, 2015 11:25 AM in response to Gnarlodious
    Level 6 (19,405 points)
    Mac OS X
    Oct 30, 2015 11:25 AM in response to Gnarlodious

    I think I can see the end of my rsync usage for HFS backups. It has been a headache for years and there’s no end in sight.

    May I suggest using Carbon Copy Cloner.  It is rsync based, and the author keeps the copy of rsync it ships with up-to-date for fixes and OS X specifics.

     

    rsync and ease of use.  What could go wrong

     

    And once you have CCC, you can use it's copy of rsync instead of the OS X shipped version (just be aware that the OS X rsync -E option is different in the CCC rsync version; I think it is -X or --xattrs).