You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Clustering Macs for Compute Power

I have a couple of Macs around that have underutilized compute power available. I don't imagine I'm alone with that. I would like to be able to "cluster" them together such that I can be on one Mac and have the computational power combined.


I am vaguely familiar with the (now abandoned) Xgrid; I wonder if there is some way to perform this task on a smaller scale (ie: at home or a small office) so that the resources can be better utilized. I can't imagine I'm alone in wondering this, too.


Thoughts?

MacBook Pro 15", macOS 10.15

Posted on Nov 23, 2019 12:01 PM

Reply
Question marked as Top-ranking reply

Posted on Nov 23, 2019 12:23 PM

Any particular tasks in mind here that need those computes?


There’s no path for client consolidation nor for distributed computing integrated with current macOS, with the existing apps.


I’m well aware of clustering, and macOS just doesn’t have any offerings here.


Xsan and SMB shares only get you so far.


Not short of adding something akin to Apache Spark or Apache Hadoop. Or SETI@home.


None of which will be particularly integrated with any of the typical macOS client apps.


Xgrid worked well for some stuff and could make for faster builds with Xcode, but so too did SSD upgrades. But few other apps used it.


For some of what’s involved, have a look at whatever of the the Xgrid and distcc stuff that’s still around, and at the Apache stuff mentioned earlier.


Compute-intensive work loads such as image rendering will have means for distributing load and that may well include Apache Hadoop, but these efforts don’t usually involve macOS.


You would probably consider lower-end Mac or iOS or iPadOS clients and a more powerful server and running one or more guests in a virtual machine, if you were building this configuration anew.

7 replies
Question marked as Top-ranking reply

Nov 23, 2019 12:23 PM in response to Forrest

Any particular tasks in mind here that need those computes?


There’s no path for client consolidation nor for distributed computing integrated with current macOS, with the existing apps.


I’m well aware of clustering, and macOS just doesn’t have any offerings here.


Xsan and SMB shares only get you so far.


Not short of adding something akin to Apache Spark or Apache Hadoop. Or SETI@home.


None of which will be particularly integrated with any of the typical macOS client apps.


Xgrid worked well for some stuff and could make for faster builds with Xcode, but so too did SSD upgrades. But few other apps used it.


For some of what’s involved, have a look at whatever of the the Xgrid and distcc stuff that’s still around, and at the Apache stuff mentioned earlier.


Compute-intensive work loads such as image rendering will have means for distributing load and that may well include Apache Hadoop, but these efforts don’t usually involve macOS.


You would probably consider lower-end Mac or iOS or iPadOS clients and a more powerful server and running one or more guests in a virtual machine, if you were building this configuration anew.

Nov 23, 2019 12:36 PM in response to MrHoffman

Yeah, perhaps I'm just SOL with this. I thought for example, I could offload graphics rendering, RAM sharing (ie: cache) but that would likely require some fiber optic connections -- certainly not worth it. As you point out, using virtual machines and such are less expensive and more efficient. Even if you have them in the cloud. I see some Mac hosting providers, but I really don't know what their strategy is -- for example I don't think you can launch a MacOS instance in AWS (yet).

Nov 23, 2019 1:06 PM in response to Forrest

Graphics buses and the processor-to-RAM buses are far faster than your network.


Primary cache is co-resident on the processor, and last-level cache is right next to the processor at its furthest distance, and the latency increases as the signal distance from the processor increases.


If you’re interested in caches and bandwidth and latency and coherence, here’s some reading:

https://software.rajivprab.com/2018/04/29/myths-programmers-believe-about-cpu-caches/

https://www.realworldtech.com/coherency/ (multiprocessors and from 2006, but still pretty good)

https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html (very visual)


The Mac hosting providers use Mac system hardware, either dedicated systems or with whatever macOS guests are permissible by license when used on Apple hardware.


Chunking stuff up within an app, that app—Blender rendering, as somebody else here has mentioned—dictates what distributed computing or clustering implementation gets used.

Clustering Macs for Compute Power

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.