Skip navigation

symbolic links get corrupted by system process?

18981 Views 232 Replies Latest reply: Jan 21, 2014 2:54 AM by robertk1 RSS
  • moitx Calculating status...
    Currently Being Moderated
    Nov 22, 2012 4:26 AM (in response to Jmanis)

    You can use AIDE, available from MacPorts. It runs as a cron job and check the current state of all the files in the filesystem against a database that contains the original signatures, sending you a report of what is changed. See at http://aide.sourceforge.net. Be aware that if your filesystem is very huge, aide will take a lot of time to complete: in this case, use it to check only what is really important by means of a reduced configuration.

  • Brian Best Calculating status...
    Currently Being Moderated
    Nov 27, 2012 12:45 PM (in response to Brian Best)

    So we're now just over 2 weeks after a rebuild of our affected MacPro.  Our RAID card now has two separate volumes which show up in diskutil list as disk0 and disk1.  I think this is significant because someone reported the issue happenning on a large partitioned drive (which would show up like disk0s2, disk0s3).  The boot volume is just under 1TB and we haven't seen any corrupted symlinks since.

     

    Certainly not a "fix" but definitely a do-able "workaround."  Will report back if I simply didn't wait long enough to see recurrence.

     

    YMMV.

  • Iosepho Calculating status...
    Currently Being Moderated
    Nov 27, 2012 12:50 PM (in response to Brian Best)

    Another million dollar question. Could everyone post whether the corruption happened on factory-installed, Apple certified drives, or on configurations not supported by Apple.

     

    My problems happened on a non-certified 3TB WD drive in a late 2009 iMac. Did anyone experience the problem on, say, a brand new iMac with a factory installed 3TB drive?

  • Brian Best Level 1 Level 1 (10 points)
    Currently Being Moderated
    Nov 27, 2012 1:38 PM (in response to Iosepho)

    My affected client was all Apple factory components.

  • Iosepho Level 1 Level 1 (0 points)
    Currently Being Moderated
    Nov 27, 2012 2:04 PM (in response to Brian Best)

    That's good, that means they won't shrug off your report. Have you set up AIDE?

    I'd love to know the results.

  • Brian Best Level 1 Level 1 (10 points)
    Currently Being Moderated
    Nov 27, 2012 2:10 PM (in response to Iosepho)

    Downloaded, but havent compiled it yet.  Looks really cool.  We've been checking it every couple days with manual commands.  No reports of odd issues from the client either.

  • sam.doran Calculating status...
    Currently Being Moderated
    Nov 28, 2012 9:40 PM (in response to ktwalker69)

    My drive is a a single 3TB Hitachi HDS723030ALA640 not originally from Apple. I do not have any RAID configuration.

     

    I have turned off Spotlight and my symlinks have remained intact for a few days now. This seems to reinforce the idea that constant read/write access leads to the corruption.

     

    I noticed that my Spotlight index seems to rebuild a lot. I know there was an issue with the Spotlight index taking a very long time to complete but this was supposedly resolved in 10.8.2, which I am running.

     

    I am hoping for a bug fix in 10.8.3 that doesn't thrash the disk as much. My guess is that the disk is getting a lot of I/O due to some high level bug and this is aggrivating some of the inherant imperfections in HFS+, namely that it tends to become corrupt over time. Or it could be a bug one layer deeper in the logical volume, as suggested by Moitx earlier, that is being aggrivated by the excessive I/O of a higer level bug.

     

    Just guesses.

     

    --

     

    Sam

  • Iosepho Level 1 Level 1 (0 points)
    Currently Being Moderated
    Nov 29, 2012 7:45 AM (in response to sam.doran)

    "and this is aggrivating some of the inherant imperfections in HFS+, namely that it tends to become corrupt over time"

     

    Don't even go there.

     

    There can be no such "inherent imperfection". Becoming corrupt over time is not an "inherent imperfection", it is called a BLOCKING ISSUE. A filesystem that, during normal operation (ie. excluding power outages or magnetic storms), loses data, is not a filesystem.

    HFS+ may be outdated and vulnerable, but implying that it, by itself, just by virtue of being used too much, loses information, is like saying that Mercedes cars spontaneously explode if you drive too much in them. (Thankfully they do not, but if they did, you wouldn't see anyone but insane people driving them.)

     

    Apple computers are widely used in data heavy applications, such as movie and music production, newspaper publishing, etc.

    If this was an "inherent imperfection", these people would have long switched to Windows. (Which uses NTFS, a filesystem not all that much more advanced than HFS+.)

     

    We'll hopefully soon see what this is. It is not an "inherent imperfection", that is certain.

  • moitx Level 1 Level 1 (0 points)
    Currently Being Moderated
    Nov 30, 2012 1:02 PM (in response to sam.doran)

    Hy all,

     

    I have investigated a lot this problem but the solution isn't there (at the moment). What I have found is:

     

    - AIDE seems to confirm that some files were modified during time, also in locations unsuspected;

     

    - I have also enabled the audit daemon, native on OS X, but it has not reported who is the author of the modifications, although was configured to trace all events;

     

    - a strange information coming from the stat command (see a precedent post), as in the following example:

     

    # Broken link

    ./warnings.py:

        Access = Nov 19 21:38:29 2012

        Modified = Nov 19 21:38:29 2012

        Changed = Nov 22 21:53:42 2012 <--

        Birth = Nov 19 21:38:29 2012 <--

        Inode = 2454785

     

    # Was:

    ./warnings.py:

        Access = Nov 19 21:38:29 2012

        Modified = Nov 19 21:38:29 2012

        Changed = Nov 19 21:38:29 2012

        Birth = Nov 19 21:38:29 2012

        Inode = 2454785

     

    Many other files reports similar informations. If you look with attention, the inode was not modified, that is the file was not moved around into the filesystem, but it's contents is modified. I agree with sam.dorat that Spotlight can be be author of this bug. Another very strange thing is the changed timestamp: it doesn't  corresponds to the real time at which the file was modified, as I have seen by the backups (Time Machine) and from AIDE's log.

     

    My machine was assembled by Apple, no modifications at hardware or operating system are in place.

     

    moitx

  • daboulet Calculating status...
    Currently Being Moderated
    Dec 8, 2012 11:32 PM (in response to ktwalker69)

    I have been seeing these symlink corruptions sporadically since sometime this past summer on my mid-2012 Mac Pro. It has three 3TB Seagate ST3000DM001-9YN166 drives (one of which is my boot drive) as well as the came-with-the-computer-when-I-bought-it WDC WD1001FALS-41Y6A1 drive.  I have tried different 3TB drives as my boot drive and still get sporadic corrupted symlinks. I just did a from-scratch re-install of Mountain Lion about a week ago in the hopes that that might solve the problem - no such luck as I just ran across a corrupted symlink tonight.

     

    They seem to happen pretty much all over the place although my Canon MX330 printer drivers stopped working due to corrupted symlinks about once every two to three weeks prior to the OS reinstall. By "stopped working", I mean that the app that I typed command-P at would crash and I would soon discover a corrupted symlink down deep in the Canon printer drivers.

     

    I run SuperDuper! every night to backup my boot drive (in addition to Time Machine).  SuperDuper! aborts when it encounters one of these corrupted symlinks so I sometimes know that one is corrupted quite soon after knowing that it is ok (I had one case where I happened to have used the symlink just a couple of hours before SuperDuper! ran). As indicated in the last comment above, the inode last change and last modification times did not 'suggest' that anything had actually changed regarding the symlink's inode.

  • Jmanis Level 1 Level 1 (5 points)
    Currently Being Moderated
    Dec 9, 2012 12:42 AM (in response to moitx)

    Thanks to motix for the cmdline tips to identify bad symlinks.  Only issue to watch out for for those using those cmds is that relative links that are valid will be present in the results since they are relative from the directory that the link exisits in they appear invalid when checked from the root location the commands are executed.  Best thing is to save the results to a file and monitor changes via diff on the current and previous file.

     

    Jeff

  • Jmanis Level 1 Level 1 (5 points)
    Currently Being Moderated
    Dec 9, 2012 12:53 AM (in response to Jmanis)

    All,

      I've opend bug report 12693117 and recieved a response from Apple requesting I run sysdiagnose and upload the results.  Shortly afterwards they closed the ticked as a duplicate and related my ticket to 12133587.  From the ticket ID it would appear that the issue has been around for a while.  Does the related ticket belong to anyone on the thread?  I've asked to be added to the bug ID so I can see the details and the progress but I'm not sure if they will allow that.

     

    Jeff

  • moitx Level 1 Level 1 (0 points)
    Currently Being Moderated
    Dec 9, 2012 4:06 AM (in response to Jmanis)

    You can save the results to a file with a simple redirect as follow:

     

    {your_command} 1>/tmp/yourlog.txt 2>&1

     

    If you want to run the commands from a different location (not the directory under check), you can substitute the dot (.) with the name of the directory of interest. A small script that can be usefull to check the same directories at different time is the following:

     

    #!/bin/bash

     

    WATCHED_DIRS="/opt/local /System/Library /Applications /Users/max/Library"

    DATE=`date +%Y%m%d%H%M`

    LOG_FILE="/tmp/broken_links-${DATE}.log"

     

    echo "Searching for broken links"

    for DIR in ${WATCHED_DIRS}; do

        echo "Looking into directory ${DIR}"

        /usr/bin/find -x ${DIR} -type l ! -execdir test -e '{}' \; -ls | nl 1>${LOG_FILE} 2>&1

    done

     

    echo "Done. Your log file is ${LOG_FILE}"

     

    If you change the space separated list of WATCHED_DIRS with the names of your interesting directories, you can collect into a log file the current state of all broken links. You can then use this log to control if some links are gone broken from the last check. To use this script, save it into a file (for example check_broken_links.sh) and give it the execution attribute (chmod a+x check_broken_links.sh)

     

    Just for information, I have disabled all the activities of Spotlight but the problem remains. The commands I have used are the following (to be run as root user):

     

    cd /System/Library/LaunchDaemons/

    launchctl unload -w com.apple.metadata.mds.plist

    launchctl unload -w com.apple.metadata.mds.scan.plist

    launchctl unload -w com.apple.metadata.mds.spindump.plist

     

    You can enable again Spotlight with the same commands substituing option unload with load.

     

    moitx

1 ... 6 7 8 9 10 ... 16 Previous Next

Actions

More Like This

  • Retrieving data ...

Bookmarked By (4)

Legend

  • This solved my question - 10 points
  • This helped me - 5 points
This site contains user submitted content, comments and opinions and is for informational purposes only. Apple disclaims any and all liability for the acts, omissions and conduct of any third parties in connection with or related to your use of the site. All postings and use of the content on this site are subject to the Apple Support Communities Terms of Use.