Zsh kills Python process with plenty of available VM

gggg87 wrote:

I run the following python script in a the same conda environment to see if conda is at fault here>

...

this gets killed by zsh once about 9GB of VM are occupied. I think this is a pretty strong lead on conda being the problem

Conda is only relevant in terms of complicating your setup. You claimed to be trying to start the script without zsh, yet you are still getting messages via zsh. That suggests that your attempts to change your shell from zsh to something else are being thwarted by Conda.

The specific shell being used is irrelevant. If Conda is causing a problem, it might be because, in addition to running a different shell, Conda might also be running an entirely different version of the script than what you expect. This could simply be a bug that was fixed 4 months ago, but Conda is still running the old buggy version.

It is the operating system kernel that is killing the process. zsh is not involved other than being the messenger of bad news.

gggg87 wrote:

I think this is a pretty strong lead on conda being the problem here (or python itself perhaps?) but not gurobipy.

Neither conda nor python are responsible for the process being killed. It's using too much RAM and the kernel is killing it.

I don't know the details of how the memorystatus architecture works. It's a big issue on iOS because RAM is so limited. And since macOS is just a "flavour" of iOS, it exists there too, albeit with different parameters.

You might want to try asking your question in the Apple Developer forums. There are actual Apple developer support engineers who might know the details of how memorystatus works on macOS, possibly even how you could disable it or work around it. To be honest, they're usually surprisingly accommodating to people asking funky questions like this. I guess it's a nice break from all the scammers who are complaining about getting their apps rejected.

gggg87 wrote:

I agree with you that the issue must lie in the conda environment. And thanks a lot for checking that other languages work better. So I will try with an environment created with venv instead, bypassing conda, just as in my Linux machine. conda makes installing gurobipy very straightforward with certain licenses, it also has a very nice IDE, that is why I was using it, but it is quite clear that the price to pay is not worth it in my case. The theory is that a conda environment should not interfere with how the OS works, and only the IDE curbs memory usage at 6GB, but from terminal it should work as if it is not even there. But it is clear that in practice things are different. Luckily a slightly more convoluted licensing can be used with gurobi and just python and I will try that now.

The more you post about that app, the worse it appears; the closer to a research project or one-off or prototype.

Regarding the shell, somehow I think changing the shell in the terminal settings does not really change the shell as an environment variable. While the terminal settings clearly show in the shell folder /bin/bash, if I echo $SHELL I get zsh, and every time I open a new terminal page I get a warning that the standard shell is zsh and how I can change back to that. So somewhere zsh is still the default and that's perhaps why kernel messages come through zsh still, even though the syntax I use to, say, open GUI applications etc, is the bash one (I hate having to go through the all zsh wrapping with open and -a etc to just do something as simple as "gedit file_name").

This message?

The default interactive shell is now zsh.

To update your account to use zsh, please run chsh -s /bin/zsh.

For more details, please visit Use zsh as the default shell on your Mac - Apple Support.

The message is telling you your login shell is bash, not zsh, and that the bash shell is old and deprecated. The linked article has details in how to change the default, as well as how to suppress the deprecation warning ifmyou want to keep the very old version of bash as the default.

Finally, a question: you say your memory usage went up to 130GB, what was the total still available in the VM volume? Was it little (I would expect the OS leaves some extra space to spare for good functioning of an ssd, but I would expect that to be, say, no more than a quarter of the total capacity only)?

Usual free percentage recommendations is probably most of ten percent free on a smaller SSD, and five percent or so on a larger. Running close to the capacity is not the greatest idea with an SSD, too.