Trying to build tree-sitter fails

VikingOSX wrote:

Homebrew uses /usr/local/homebrew on Intel Macs and /opt/homebrew on Apple Silicon. It automatically does this to avoid stomping on existing X86_64 binaries in /usr/local/bin that may have pre-existed to the Apple Silicon hardware replacement. Then it is down to one's PATH statement to reconcile which is used.

I don't think the PATH environment is the problem here. The clang compiler automatically includes /usr/local/include in the search path. Since the OP's computer has files there, they are being included instead of the system files from the SDK path. There is a stdlib.h file there that then includes Availability.h. That file then includes AvailabilityInternalLegacy.h, which doesn't exist.

Unfortunately, the only result from Google for "AvailabilityInternalLegacy.h" is this thread.

But DuckDuckGo has this one: https://github.com/raspberrypi/usbboot/issues/188 which seems to be the same problem.

According to that, the Homebrew prefix is just /usr/local. There's no "homebrew" part. This clearly points to Homebrew as the direct cause. I can't comment on the /opt/homebrew path. It all looks like /usr/local from here. If Homebrew is putting a partial set of include files in that directory, that's just a Homebrew problem. No news here.

VikingOSX wrote:

Even if you cloned this from the tree-sitter Github site and did not use homebrew, you won't find help here for the tool, its dependencies, or how to successfully build the tool.

I don't know about that. I had never heard of this tool. I certainly have never seen the Github site before.

But just for fun, I downloaded it. I was able to successfully build the tool. Here is the command I used and the output in its entirety.

/tmp/tree-sitter-master $ make
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/alloc.o lib/src/alloc.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/get_changed_ranges.o lib/src/get_changed_ranges.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/language.o lib/src/language.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/lexer.o lib/src/lexer.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/node.o lib/src/node.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/parser.o lib/src/parser.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/query.o lib/src/query.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/stack.o lib/src/stack.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/subtree.o lib/src/subtree.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/tree.o lib/src/tree.c
cc -O3 -std=gnu99 -fPIC -fvisibility=hidden -Wall -Wextra -Wshadow  -Ilib/src -Ilib/include   -c -o lib/src/tree_cursor.o lib/src/tree_cursor.c
ar rcs libtree-sitter.a lib/src/alloc.o lib/src/get_changed_ranges.o lib/src/language.o lib/src/lexer.o lib/src/node.o lib/src/parser.o lib/src/query.o lib/src/stack.o lib/src/subtree.o lib/src/tree.o lib/src/tree_cursor.o
cc  -dynamiclib -Wl,-install_name,/usr/local/lib/libtree-sitter.0.dylib lib/src/alloc.o lib/src/get_changed_ranges.o lib/src/language.o lib/src/lexer.o lib/src/node.o lib/src/parser.o lib/src/query.o lib/src/stack.o lib/src/subtree.o lib/src/tree.o lib/src/tree_cursor.o  -o libtree-sitter.0.0.dylib
ln -sf libtree-sitter.0.0.dylib libtree-sitter.dylib
ln -sf libtree-sitter.0.0.dylib libtree-sitter.0.dylib
/tmp/tree-sitter-master $ 

etresoft wrote:

…

According to that, the Homebrew prefix is just /usr/local. There's no "homebrew" part. This clearly points to Homebrew as the direct cause. I

Probably already aware, but Homebrew uses /usr/local on x86-64 Intel, and uses /usr/homebrew in Apple silicon.

Details: https://docs.brew.sh/Installation

Yes; /opt, not /usr.

b63s wrote:

1: apologies, it’s not building actual tree-sitter it’s the language libraries (grammars)

Ah, OK. I looked for how to do that and apparently one has to download some unsigned binary in a gzip. I'll pass. 😄

2: for context, yes, I do run homebrew, however not for this

If you've installed Homebrew, then you use it for everything. You can't opt in or out for certain projects.

Well, in theory I guess you could. But what's the point of using Homebew if you know how to manually configure the environment at that level?

3: it is not related to homebrew, as this a silicon chip, homebrew doesn’t touch /usr/local

Ah, interesting. See https://mac.install.guide/homebrew/index.html

For clarification, Homebrew wasn't criticized for using /usr/local. They were criticized for making permission changes to /usr/local/ that significantly reduced security. But that's a general consensus. I criticize them for different reasons.

Using /opt/homebrew/ is an objectively better path. However, that means that all of the many thousands of open-source projects that were hacked-up and hard-coded for Homebrew in /usr/local/ will now need to be re-hacked.

4: these are header files included from CommandLine Tools which is the toolchain from xcode, so it is in fact apple product line.

why they would be placed in /usr/local beats me though, POSIX standard should be any third party goes in /usr/local - I have no additional tooling deliberately placed there.

By default, /usr/local is empty. If you have files there, someone put them there. I'm guessing it was Homebrew, at one point or another, regardless of what it claims to be doing now.

I am on the run at the moment so on my phone.

when I get home, I’ll post a snippet of some of the headers to show Apple ownership.

I’ll also verify

xcrun --sdk macosx --show-sdk-path

which should point to xcode and not /usr/local

just to be safe, I’ll also verify clang and which compiler is used.

To clarify, are you using Xcode or the Command line tools? And which versions? And which OS versions?

But to re-iterate again, these kinds of problems are always caused by Homebrew.

xcrun --sdk macosx --show-sdk-path

will show the path to the SDK, not the include locations. clang -v will show the include paths. I think /usr/local/include is included, but would have no effect being empty.

I was only suggesting where homebrew stashes package contents per architecture and PATH to control where it retrieves a binary.

It gets a whole lot nastier if one has Apple Silicon, performs a migration from an Intel Mac and uses homebrew to install a newer version of the package. Now, one has dependent include and libraries in different locations and clang may be pulling in outdated build dependencies from the /usr/local tree on an Apple Silicon machine.

b63s wrote:

Any one seen similar problems?

Nope. Because none of those files actually exist on a standard system. You've installed some 3rd party system modifications, haven't you? I'm guessing Homebrew. Amiright?

That big 'ole can-o-worms that is. Pretty much anytime I see people trying to use the Terminal and having some strange problem with something really basic, Homebrew is always the reason. Every. Single. Time.

Even if you cloned this from the tree-sitter Github site and did not use homebrew, you won't find help here for the tool, its dependencies, or how to successfully build the tool. The answer is or is not at the tree-sitter site.

But just to clarify, I didn't test the output in any way. A causal review suggests that it might have problems. The "-install_name,/usr/local/lib/libtree-sitter.0.dylib" is obviously wrong. They are hacking up the output to assume it lives in /usr/local/lib.

These kinds of problems are common in open-source projects. It is probably a hack added without much thought to try to make something work. Because most people use Homebrew, it worked fine. So they never gave thought any more about it. It might still work even in other contexts. But it could also fail at any point now or in the future or on different platforms and they'll be scrambling for weeks, blaming Apple I'm sure, trying to fix it. All they needed to do was not do the hack in the first place.

VikingOSX wrote:

Well, it had to be config && make or just make… 🤓 though I confess, I was not seeking a morning migraine or messing with something I would never use.

Strangely enough, it was just "make".

Autotools seems to be much less popular these days. People are either using CMake or one of those multi-stage Google build monstrosities. I can certainly see the appeal of Homebrew with these things.

Homebrew uses /usr/local/homebrew on Intel Macs and /opt/homebrew on Apple Silicon. It automatically does this to avoid stomping on existing X86_64 binaries in /usr/local/bin that may have pre-existed to the Apple Silicon hardware replacement. Then it is down to one's PATH statement to reconcile which is used.

Well, it had to be config && make or just make… 🤓 though I confess, I was not seeking a morning migraine or messing with something I would never use.

But of course, none of that is going to help the poor OP. For them, it's another trip to Recovery mode for an OS wipe and reinstall.

Did you mean /opt/homebrew instead of /usr/homebrew? Been using homebrew for several years now on both Intel and Apple Silicon Macs and the former uses /usr/local and the latter /opt/homebrew.