symbolic links get corrupted by system process?

Looks like you're seeing the same issue that a number of us are....file symlink corruption. General symptom appears to occur on machines with RAID partitions greater than 2Tb. I have a developer request in with Apple who's only recommendation is to visit the Genus Bar and check for hardware issues. Do not believe this was addressed in 10.8.3 but plan to test once I've upgraded....hopes not high.

My partition is NOT raided. Seagate 3TB HFS+ journaled encrypted. There are RAIDed volumes but they are not shoeing the problem.

No RAID here either. 3TB Seagate drive. See my posts above. I was able to resolve this by going back to a smaller 1TB drive.

Various folks (including myself) have seen this in contexts which do not involve RAIDs.

I seem to have made the problem go away on my mid-2012 Mac Pro by repartitioning my internal boot drive with partitions which are less than 2TB. Unfortunately, I seem to recall that some people have reported that this does not make the problem go away for them.

From what I have seen and what I have read, this looks like a software problem to me. If it was a hardware problem then one would expect the corruption to be much more random as opposed to only affecting symlinks. One would also not expect a hardware problem to be correlated to partition size since partitions are in a very real sense a software concept in that they are an "invention" of the kernel (and related programs) as opposed to involving any sort of hardware change to the drives (granted the drive contains the information that the kernel uses to know how large and where the partitions are but that doesn't change the fact that partitions are a software "invention" of the kernel).

I've got a first generation Mac Pro running Lion that has been using 3TB drives with 3TB partitions for at least a year or more without any problems at all. This suggests to me that this is a defect which appeared for the first time in Mountain Lion. Has anyone seen anything even remotely like this issue with Lion or other older versions of Mac OS?

I wrote a C program that I automatically run twice a day via the cron facility on my mid-2012 Mac Pro to look for symlinks which contain non-ASCII characters. It used to find problems with alarming regularity before I repartitioned my drive in early January. It still runs twice a day but has not reported a hit since early January.

I can provide the C program in source code form if anyone wants it. I am not interested in being falsely accused of distributing malware so I am not willing to release a compiled version (if you're worried about malware, the program is pretty short and anyone with a reasonably solid background in Unix software development should be able to verify that the only thing that it does is report symlinks which contain non-ASCII characters). You would need to:

1. install XCode

2. use XCode to install the command line tools

3. compile the program using said command line tools

4. setup a cron job to check your drive (or check it manually from time to time)

-Danny

Add me to the list of people experiencing this problem. In my case it happens to (at least) some of the libraries MAMP uses. It has only happened twice, but that's twice too many. The hard drive is fairly new (a month or two), and shows no other issues and no SMART errors. It is a 3 TB drive, set up as a single partition.

daboulet wrote:

...

I wrote a C program that I automatically run twice a day via the cron facility on my mid-2012 Mac Pro to look for symlinks which contain non-ASCII characters. ..

I can provide the C program in source code form if anyone wants it.

I would welcome a copy -- I'm more than a little nervous now, since I'm not sure what else might be affected. A scan like that would certainly help.

I'm so glad I found this thread at last...for 6 months am I experiencing the same issues with broken symlinks on Mountain Lion. All the clues are mentioned here (why haven't I found this earlier? so much wasted time reinstalling stuff over and over with the same result after few days). So I can also confirm these thoughts. It all begins after upgrading my internal hard drive from 1TB to 3TB size. No RAID or JBOD, just one huge HFS+ partition. I do compile a lot of stuff on command line and it's really madness😠. So take me to the list too.

daboulet I also have big interest of the C program source code.

BTW, thanks to all for great work at investigating this "pain in the a.."

Danny,

Can you post the source code on codeplex or git? I'd like to get a copy or email to my username @yahoo.com

Thanks

Jeff

I have posted the source code on github at the following URL:

https://github.com/daboulet/MacOSX-broken-symlink-finder

Please read and make sure that you comprehend this entire message before downloading the software.

There is a README file accompanying the source code which provides a very brief explanation of how to use the program.

I have released the source code into the public domain.

I will not provide compiled versions of the software to anyone because I am not interested in being accused of distributing malware. Please do not ask.

You will need to install Apple's XCode software and then use it to install the command line development tools before you will be able to compile this program. If you don't know how to do this then it might be best to get help from someone you trust and who does know how to install and use XCode.

Anyone interested in using this program needs to understand that I am providing this program because I believe that it might be useful to others. Whether or not it is useful to you is something that you will have to decide for yourself.

When the time comes to actually run the compiled program, you will need to run it with root privileges. Please keep in mind that a program which is run with root privileges has the ability to do REALLY BAD THINGS to your computer. Use appropriate caution. For example, you may want to have someone you trust that understands how to write Unix software in the C programming language review the source code and provide you with guidance and assistance.

It is possible to run the program without root privileges although it will not be able to scan your entire hard drive and you will get a lot of 'noise' error messages.

If you obtain what someone claims is a compiled version of this program from a third party, make sure that you trust that third party enough to run their software on your computer with root privileges.

If you do not understand the implications of any of the above statements, get advice from someone who does BEFORE attempting to use this software.

THIS SOFTWARE IS PROVIDED WITHOUT ANY WARRANTEE OF ANY KIND. USE AT YOUR OWN RISK.

Thank you for posting that source code -- the program confirms my suspicions, that there are a whole bunch of corrupt symlinks that I didn't know about. It also explains why a couple of applications have stopped working for no apparent reason (kinda hard to work when the library symlinks are all snarled up).

Is this a known bug with Apple?

Disregard my "is this a known bug" question -- in reading back through the thread I see that it is. I still have the small drive that I upgraded from, so I can mount it up and see what the symlinks are supposed to be and fix them. It is beyond annoying to have to do this.

The catch is that you should not expect the fixed symlinks to stay fixed. I went round and round this mulberry bush back in December and January. It was a lot like playing a game of wack-a-mole where it felt like every time I fixed a link, another broken one popped up. What was more than a little 'interesting' is that certain symlinks would seem to break on a fairly regular basis. Not sure what the pattern was but it almost seemed like links that were used frequently were at more risk of breaking than others (just to be clear, this was an observation that was not based on all that much evidence).

What eventually worked for me was repartitioning my drive such that no partition was larger than 2TB (I actually drew the line at 1.9TB just to be sure). Unfortunately, others have reported that this did not solve the problem for them so I'm not sure what to actually suggest that you do.

I do not suggest that you plan on fixing them as you appear as that gets old awfully quickly.

On the topic of "does Apple know about this" . . . I made an Applecare call as part of trying to figure out what was going on with my six month old mid-2012 Mac Pro. I wrote them off roughly at the point that they told me to uninstall XCode since that felt an awful lot like just a different form of wack-a-mole. It didn't help that I had installed the XCode command line tools and the tool vendor (i.e. Apple) provides no advice or process to uninstall them. Rather than do yet another reinstall from scratch (third one in a month), I decided to repartition the drive. As I said above, that made the problem go away for me.

I was hoping 10.8.3 would have fixed this, but no dice. One of the Java frameworks got corrupted and CrashPlan stopped working. I am running the checkSymLink program now and finding all kinds of stuff.

It would be very difficult to split this drive in two; maybe I'll temporarily install the system on a spare drive for now.

I have this problem too. I'm running MacOSX 10.8.3 and Server.app 2.2.1 on a MacPro Server. The boot disk was a RAID5 with over 5 TB size.

I made an AppleCare Server Support Call. Apple knows about the problem, but when Apple solves the problem, they don't know.

Apple confirms that the problem exists and that it happened less frequently on a boot disk smaller than 3 TB.

Today I made my boot disk (partition) less than 2 TB.

I hope so, that you all are right saying that the problem doesn't occur on a smaller boot disk.

The easiest way to get Apple's attention on an issue like this is to first prove it. Posts on the internet do not qualify as proof. You need to understand the problem and hopefully be able to reproduce it or tell others how to reproduce it. I suggest using the fileXray software to try to understand it. It can't be any less efficient than a "me too" internet post.

It's simple to proof the defect: run a software that uses some links on disk for some time (say some hours), then put your Mac into sleep state and then activate it again. That is, it is really possible that you have some broken links. As test software you can use Mail, gcc, Xcode just to give you some examples, fileXray (or hfsdebug) is just a plus.

My opinion is that Apple MUST resolve this issue quickly. I have spent much (much!!) money for my workstation and this bug is unacceptable. Obviously, the next time I will need a workstation I will remember that.