7 Replies Latest reply: Dec 17, 2007 9:36 PM by Scott Radloff
ElPapa Level 1 Level 1 (0 points)
Hey there..

Can anyone give any insights on the aging algorithm that TM uses? I mean this application is going to back up on a regular schedule until all 500GB on the drive is consumed.. but then what? How does it decided what to "get rid" of in order to maintain integrity of the backup data over time?

Mac Pro (Quad Core 2.66Ghz), Mac OS X (10.5.1)
  • 1. Re: Aging algorithm...
    Peggy Lynn Level 4 Level 4 (2,470 points)
    When the drive is full Time Machine starts deleting the oldest backup first keeping all the files necessary for a full restore of the system. If a file is on your computer for less than a day and it isn't in the hourly backup done at midnight or the first backup after it will be deleted in the process of Time Machine thinning the hourly backups to dailies.

    It is not yet absolutely clear what Time Machine keeps in the daily backups when they are thinned to weeklies. To be on the safe side it's probably best to keep files on your drive for eight days before trashing them if you want Time Machine to retain backups of those files until it runs out of disk space.
  • 2. Re: Aging algorithm...
    ElPapa Level 1 Level 1 (0 points)
    Hi Peggy,

    At a high level I figured that what you just described was the case.

    On a side note - what prompted this question was the following article from MacOSXHints... He questions how effective the backups are because the diff between the external backup volume and the harddrive shows they're not identical..

    http://www.macosxhints.com/article.php?story=20071109024812437

    Interesting posting - but IMHO superfluous because the guy has you diff against the soft-link to the "Latest" folder. Which more than likely is only a differential backup from the most recent interval.
  • 3. Re: Aging algorithm...
    BarryXSharp Level 5 Level 5 (7,880 points)
    TM maintains backups as follows

    The initial backup is the first hourly backup.

    1. Performs hourly snapshots for 24 hours (can be a few more than 24 at times)

    2. The first backup that runs after midnight will take the last hourly backup done prior to midnight and save that as a daily backup.

    3. The daily backups are collected up for 30 days (or one month)

    4. After a month and one week or partial week all daily backups for Mon thru Sat are discarded and the Sun daily is kept as a weekly backup.

    5. As each week passes the action in 4) above is performed.

    Soooooo, say after 6 months you will have 5 months of weekly backups (constituting some 20 t0 22 weekly backups) and 30 daily backups and a few hourly backups.

    When the disk fills TM will try and remove some of the old weekly backups to make room. It's unclear to me just how far TM will go to make more room for the current backups. However, if the current backups are relatively small I believe TM will simply remove a few of the oldest weekly backups until it has regained sufficient disk space to complete the current hourly backup.

    TM will always keep important OS related files so that a full OS restore can be done. Typically these OS related files are captured during the initial backup. However, more OS related files can be captured immediately TM backup executes after you've done an OS upgrade such as 10.5.1, 10.5.2, 10.5.3, etc.

    Hope that clarifies things for you.

    Good luck.
  • 4. Re: Aging algorithm...
    pvonk Level 6 Level 6 (13,685 points)
    BarryXSharp wrote:


    When the disk fills TM will try and remove some of the old weekly backups to make room. It's unclear to me just how far TM will go to make more room for the current backups. However, if the current backups are relatively small I believe TM will simply remove a few of the oldest weekly backups until it has regained sufficient disk space to complete the current hourly backup.
    BarryXSharp wrote:


    While I don't think any of us really know what goes on under the hood, I don't think it's as simple as removing the oldest weekly backup. Each backup is a collection of files that are saved for the first time or links to files that were previously backed up. So if the oldest backup includes files that newer backups have links to, I'd be surprised if the real file is deleted.

    One possibility is that, within the oldest backup, the number of references to each "file" is checked, and if only one reference exists, then that file is deleted, otherwise only the link to the file is removed.

    But it would be interesting to read about the algorithm.
  • 5. Re: Aging algorithm...
    ElPapa Level 1 Level 1 (0 points)
    BarryXSharp wrote:
    TM maintains backups as follows



    4. After a month and one week or partial week all daily backups for Mon thru Sat are discarded and the Sun daily is kept as a weekly backup.


    When the disk fills TM will try and remove some of the old weekly backups to make room. It's unclear to me just how far TM will go to make more room for the current backups.


    And this part I understood... And this is where I had the question 'when the hard drive is full.. what does TM do to "make more room" and still retain restoration integrity...?'

    So far no one really knows...
  • 6. Re: Aging algorithm...
    Scott Radloff Level 6 Level 6 (14,490 points)
    ElPapa,

    Actually, we do know....

    Time Machine will begin deleting the oldest backups. Period. It won't "thin" the files in specific backups, it won't skip backups, it will simply begin deleting the very oldest backups, oldest to newer, until it has freed up enough space for any new ones.

    Chances are, this means that at least several of the oldest backups will be deleted, given the multi-linked nature of the Time Machine "snapshots." In any given snapshot, only a relatively small number of files will be unique, so many of these snapshots must be deleted before any real disk space is recovered. At some point, all of the snapshots that are comprised of a given OS version (10.5.0, for example) will have been deleted; it is most likely at this point that some substantial disk space will become available.

    Of course, those that find themselves in this situation the soonest will be those that tend to have a large amount of additional or changing files, a lot of "data churning," as it is called. Fortunately, it will be the backups made under these conditions that will be the easiest to "thin" by deleting old backups.

    Rest assured, however, that at no time will one not have what is required for a complete system restore. The latest backup always links every file necessary for a complete restore.


    Scott
  • 7. Re: Aging algorithm...
    ElPapa Level 1 Level 1 (0 points)
    Scott,

    I think this covers everything. Thanks.