Q: symbolic links get corrupted by system process?

dburr wrote:

 

As mentioned before, Homebrew is specifically engineered to run with standard user permissions (I.e. NOT requiring sudo).  Since link corruption has been reported in areas such as /System/Library/Frameworks, areas where mere mortals normally don't have license to muck about in, this sounds highly unlikely that Hiomebrew would be the cause.

There is nothing special about Homebrew. One of the first replies in this thread contained the following:

Launch Daemons:

                 [loaded] homebrew.mxcl.memcached.plist

                 [loaded] homebrew.mxcl.redis.plist

That is Homebrew running with root permissions, doing whatever it wants.

And lsof is part of the standard UNIX tools that Apple distributes as part of the base OS X install (it is located in /usr/sbin), so if that were the culprit then fixing it is part of their purview.

That is just one of the things that hstimer reported as running with root permissions. That is not something one normally does. As I said before, there is nothing known, running as root or not, that will cause this other than hardware failure. If you think it is a software failure, then the burden is on you to prove it.

And there are users who have reported this issue that don't run any of the software mentioned (including myself - I do not use Wireshark, Vmware, etc.)  Not to mention people who have reported this problem as occurring even on "vanilla" rebuilds.

And that brings us right back to reply #5 in this thread I think. A dozen people, with radically different machines, are complaining. Some of whom have startup volumes of 3 TB, and some have startup volumes of 750 GB. Some say Lion causes it. Some say Mountain Lion is required. What is the common thread between them? They all seem to be doing unusual things and experiencing unusual results.

You claim people are reporting this on "vanilla" systems? They might be in there somewhere. What I see is everyone running some very unusual software. There is nothing wrong with running unusual software. I do that myself. But if you run unusual software and have unusual result, it is your responsibility to track it down.

Ergo, there is a problem that is impacting 0.0004875 % of all Mac users. I think it is a hardware problem. Apparently Apple thinks it is a hardware problem. You think it is something else? You have to PROVE IT!

Report the bug to Apple. Complaining that somebody else on the internet claims to have done that and Apple told them to go to the Genius bar is not the same thing as "reporting the bug to Apple". Apple does not read these forums for good reason. This is not a problem that anyone can fix via an internet forum. Posting here is equivalent to doing nothing. The only thing that I, or anyone else, gets out of posting in this thread is more wasted time.

Hmmm, there is no overlap between the kext on your machine and mine.

Some clarification:

1) Launch daemons don't have to run as root

2) Homebrew doesn't install any services directly, but some of the packages you might install do add launchctl files

3) By default, I believe they are installed in ~/Library/LaunchAgents which would have them running as the user. You can move them of course into /Library/LaunchAgents where they will run as root, unless the plist specifies a different user.

I don't run any of those developerish debuggy things. I just run Mountain Lion with normal apps. I had the most problems with iTunes and anything in Java.

syd, could you run the following in the terminal and reply back with the results?

      kextstat | grep -v apple

Thanks!

Alas, after fighting this for several months (and having my bugreporter account not work for almost that long), I finally got really angry, reformated, and partitioned to a much smaller boot partition. Have not seen the problem since. Wish I could help.

For those who aren't seeing the problem, it would be interesting to see what a scan might turn up. If you are a programmer than it should be pretty easy.

Download and build:

      https://github.com/daboulet/MacOSX-broken-symlink-finder

Then from the build directory, run:

      sudo find / -xdev -type l -type l -exec ./checkSymlink \{\} \;

Please report positive or negative results. If negative, then it would be helpful to get results from the following:

    launchctl list | grep -v apple

     kextstat | grep -v apple

etresoft wrote:

This whole thread is very fishy. If anything like this happened to me there is no way on earth I would continue to run and put my data in jeopardy.

 

This is what people are saying here: "Yeah, some kind of strange corruption is happening on my system. Symbolic links keep getting overwritten with garbage. But hey, I just fix the links and keep on truckin'!"

 

Really? I mean, really?

No, not really. I'm surprised myself at some of the people. I immediately froze all work on the system, backed everything up, and after a few experimental reinstalls and destructive disk tests, I took the thing apart.

I experienced the issue after installing a non-Apple-certified 3TB HDD (which was, after all, free from any corruptions, it's been working fine under Windows ever since). Still, being "oddball" hardware, I couldn't really make a case for Apple. I returned the original 1TB drive, and the issue ceased.

Anyway, exactly your point, I'm exiting the Mac scene as a primary platform, I've been really missing FL Studio anyway. I simply cannot trust something that may or may not randomly corrupt symlinks or other data. And to pay above two grand for something, I really really need to trust it.

If it's a hardware issue, then it is clearly complemented by driver or FS issues. (Ever watch those documentaries on NatGeo about plane crashes? Tiny errors adding up to a catastrophe. No plane will crash from a slightly peeled signal cable, but a slightly peeled signal cable PLUS a leaking fuel pipe, PLUS some metal part electrified that shouldn't be...)

Anyway, something to try, but since I no longer have the hardware in its problem state, I cannot do it. Just install Linux on the same Mac, and keep watching if any corruption occurs. If it doesn't...

I ran the link checker on my MacBook Pro last night and it found a garbage link. MyBook Pro has the same software configuration as my MacPro:

10.8.3

vmware, virtualbox

homebrew

If I can pry my wife and daughter's laptops out of their hands, I'll run the link checker there too. My daughter has vmware, but no other non-appstore apps. My wife's computer is pure appstore.

Just keep in mind that my symlink checker is heuristically based - if it finds a link that it doesn't like then that link is probably but not necessarily bogus. Even if it is bogus, it might not be related to the topic of this thread. Finally, I know for a fact that it fails to report some links that are characteristic of this issue. I suppose that I could fix this particular issue but that would just leave the other false negatives that I don't know about.

Filexray (I finally got my copy) lets you dump the link contents, so it is pretty easy to verify if it is garbage data.

A link should never have garbage in it's data fork. This thread should be more accurately called "Soft links with  garbage data".

Wife's computer, MacBook Air with only AppStore apps, came up clean.

kext are clean

launchctl only came up with two non-apple things:

The ssh-agent is distributed by Apple, so there is really only one non-apple process.

Using FileXRay, I've confirmed that the blocks immediately before and after the soft-link block are not corrupted. I was able to verify with 2 separate corrupted links.

The link's blocks are in a section of the volume that seems to be where single block files are put. The four files I inspected where all single block text files.

etresoft wrote:

... As I said before, there is nothing known, running as root or not, that will cause this other than hardware failure. If you think it is a software failure, then the burden is on you to prove it.

Perhaps what you intended to say was "there is nothing known, running as root or not, that will cause this *including* hardware failure." 

I've worked with computers for over 30 years, troubleshooting, repairing, upgrading, and programming.  I have seen more hardware failures, including hard drive failures, than I can even rememer.  And I have *NEVER* seen a hardware failure that targets only certain kinds of files on a hard drive.  Every time I have seen files within a certain class being corrupted, it was always software.

Clearly, nobody knows what causes this, because if they did, they'd have fixed it by now.   I happen to agree with you that the proper course of action is to try to replicate and isolate, and report it to Apple.  But just as I am certain the sun will rise tomorrow, I am certain this is a software problem.   It may not be *Apple's* software, but it's definitely software.

robertk1 wrote:

 

Perhaps what you intended to say was "there is nothing known, running as root or not, that will cause this *including* hardware failure."

I said precisely what I intended to say.

But just as I am certain the sun will rise tomorrow, I am certain this is a software problem.   It may not be *Apple's* software, but it's definitely software.

I never said I was certain. I said there was "nothing known". You are the one claiming certainty. I suggest you spend a few more years learning about operating systems and file systems before making claims like that.

When the installer installs the operating system, it writes all the files and then creates any symbolic links. On a journaled file system like HFS+, those writes are going to occur on previously unused portions of the disk. Therefore, symbolic links are likely to reside physically close to other links on the surface of the disk. And, on big enough drives, they may be on previously unused blocks. Considering the wide range of setups reported in this thread, there is likely to be little in terms of SMART reporting.

All drives have a certain percentage of bad blocks. The bigger the dirve, the more bad blocks. External DIY RAIDs on big, cheap, 1st generation, high-density drives are particularly sensitive to this possibility. Those that have blocks fail on critical portions of the disk are alerted when something breaks. If those bad blocks contain only symbolic links, then a failure might not be noticed and would seem to affect multiple symbolic links, such that those created en masse by an installer.

If it were a hardware failure such as I have described above, then there would likely be a low failure rate that matches the handful of reports in this thread. A software problem would affect all systems having the same configuration. Quite frankly, the rest of the world hasn't noticed anything like that. Ergo, it is probably hardware problems being repeated dozens of times among tens of millions of drives.