Level 1

1 points

Where does OS X store environment variables?

I'm new to OS X but I've used Linux (and a Unix a while ago) on and off. I'm a bit confused as to how the particular version of Unix Mac OS X uses works.

In Linux, I kept environment variables in .bash and .bash_profile files in my home directory. What is the default file Mac OSX uses to store all this information? So, for instance if I do

$env

I get information about my environment such as:

PATH=/usr/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/ usr/X11/bin
...
USER=josepmfontana
...
HOME=/Volumes/DATA

The problem comes when I try to modify this information. I even did a grep:

Josep-M-Fontanas-iMac:~ josepmfontana$ grep -r USER=josepmfontana ~/.*

But I only found this string in .bash_history

Actually, I found another unexpected problem. grep stalled with the following error message:

grep: warning: /Volumes/DATA/../Macintosh HD/Applications/VirtualBox.app/Contents/Resources/VirtualBoxVM.app/Contents/Res ources: recursive directory loop

I've checked a few manuals, I've Googled for default environment variables Mac OS X but I'm stumped.

Any help I can get from the community will be greatly appreciated.

Josep M.

iMac 27" Core i7, Mac OS X (10.6.4)

Posted on Jun 25, 2010 4:30 AM

26 replies

KJK555

Level 4

2,906 points

Jun 26, 2010 2:51 PM in response to etresoft

.profile, that's where they wrote mine.
I guess if you don't have a .bash_profile file it
writes it to .profile ? or vice versa? or
scooby dooby do? 

-------------------
#test -r /sw/bin/init.sh && . /sw/bin/init.sh

#Setting PATH for MacPython 2.5
#The orginal version is saved in .profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}"
export PATH
#
#Your previous .profile (if any) is saved as .profile.mpsaved
#Setting the path for MacPorts.
#export PATH=/opt/local/bin:/opt/local/sbin:$PATH

#Setting terminal window behavior
export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagacad

##
#DELUXE-USR-LOCAL-BIN-INSERT
#(do not remove this comment)
##
echo $PATH | grep -q -s "/usr/local/bin"
if [ $? -eq 1 ] ; then
PATH=$PATH:/usr/local/bin
export PATH
fi
----------------

etresoft

Level 9

56,656 points

Jun 26, 2010 6:38 PM in response to KJK555

KJK555 wrote:
.profile, that's where they wrote mine.
...
#Setting the path for MacPorts.

That's why I qualified my response with "no good 3rd party software would mess with your dot files" 🙂

jfontana Author

Level 1

1 points

Jun 27, 2010 3:39 AM in response to etresoft

etresoft wrote:

KJK555 wrote:
.profile, that's where they wrote mine.
...
#Setting the path for MacPorts.

That's why I qualified my response with "no good 3rd party software would mess with your dot files" 🙂

Well, I thought I had reached some peace of mind but I guess I haven't.

So basically what you are saying is that a good third party application would install itself without adding any path variable and simply leave it to the user to do that. Given my experience (where a third party python installation had added a path variable that interfered with the default python installation for the Mac and I didn't know where to go to fix the problem) I would tend to agree with you. However, not making necessary changes to the environment so that the given application can function properly out of the box would certainly create other problems for some users as well.

There is not even agreement in this forum (where most people would qualify as advanced users) as to where it is more advisable to store and centralize environment variables so what would the text file with instructions that would have to accompany the app say? Add this path variable to either .bash_profile .profile or .MacOSX/environment.plist, whichever you prefer?

I know we are talking about UNIX and most "normal" users of OS X will never have to mess around with these configuration files but it would be nice to have some "official" documentation clarifying these aspects and establishing some clear and "official" protocol for environment variables. This would be helpful for users and developers of third party apps as well.

JM

jfontana Author

Level 1

1 points

Jun 27, 2010 4:16 AM in response to jfontana

Just a little link I found that shows that the confusion is rather spread:

http://johnnywey.wordpress.com/2008/04/17/fixing-bash-profile-in-os-x/

I guess there are a lot of "bad" third party applications out there 🙂

This time the problem was that an application had added the path variable in .bash_profile (unlike in my case where the path was added to .profile). These users seem to assume that the default location for environment variables should be .profile rather than .bash_profile. The blogger doesn't explicitly say so but from what he says and from what the people commenting say it looks like they advocate preserving .profile as the default environment storage file (they make it sound as if .bash_profile is the unwelcome "intruder".

If .bash_profile is higher in the hierarchy of the bash shell, then it would make more sense to keep that as the default storage file for environment variables and transfer any information that gets added to .profile by third party apps to it. I think that this is what etresoft was suggesting.

My point is, however, that this should be made more clear in the documentation (although I guess it is clear enough in the bash shell official documentation if you know how to read it 🙂 )

JM

jfontana Author

Level 1

1 points

Jun 27, 2010 4:25 AM in response to KJK555

Hi KJ,

as I said in my response to etresoft's post, now my conclusion is that one should keep .bash_profile as the default file for storage of environment variables. So your suggestion of creating a symbolic link to .bash_profile named .profile so that "misguided" applications wouldn't mess things up is a good one. Thanks for the link to symboliclinker. I find this little app very useful.

JM

etresoft

Level 9

56,656 points

Jun 27, 2010 8:43 AM in response to jfontana

jfontana wrote:
So basically what you are saying is that a good third party application would install itself without adding any path variable and simply leave it to the user to do that.

An ideal MacOS X application would come in a .app wrapper that could be dropped into /Applications and run from there - or run from the command line. I think safe rule of thumb would be that no application should modify any existing file that it doesn't own. How would you uninstall such a thing?

I know we are talking about UNIX and most "normal" users of OS X will never have to mess around with these configuration files but it would be nice to have some "official" documentation clarifying these aspects and establishing some clear and "official" protocol for environment variables. This would be helpful for users and developers of third party apps as well.

There is no official documentation because there is no official way to do it. Bash is just one of many shells. There is also csh, tcsh, zsh, fish, etc. A good UNIX tool will provide its own script that starts a new shell (using just sh), sets up the environment correctly, and runs. You shouldn't have to add anything to your path to run it. If you want to run it without having to type the full path, just make an alias to it in your shell. Modifying a user's path has all kinds of side effects and is just no good.

etresoft

Level 9

56,656 points

Jun 27, 2010 8:53 AM in response to jfontana

jfontana wrote:
What is the default file Mac OSX uses to store all this information?

Perhaps it would be good to step back a minute and ask some basic questions. It is common for people to start down a path, get stuck and then ask for help on how to resolve the sticky situation. The correct answer is often that they are on the wrong path to begin with.

What problem are you experiencing that causes you to ask this question?

PATH=/usr/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/ usr/X11/bin

There are certainly some redundancies in there. It should look like:

PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/usr/X11/bi n

HOME=/Volumes/DATA

I don't like the looks of this. How did you accomplish this? It doesn't look right. If you want your home directory to be on a different volume, it would be best to mount that volume at /Users.

The problem comes when I try to modify this information. I even did a grep:

Josep-M-Fontanas-iMac:~ josepmfontana$ grep -r USER=josepmfontana ~/.*

You can't modify USER or HOME this way. Well, perhaps you can, but you shouldn't. Bad things will happen. If you want to change USER, you should create a new user account. If you want to change HOME, you should do it through the Accounts pane in System Preferences. Select your account and right/control click on it to bring up "Advanced Options"

Actually, I found another unexpected problem. grep stalled with the following error message:

grep: warning: /Volumes/DATA/../Macintosh HD/Applications/VirtualBox.app/Contents/Resources/VirtualBoxVM.app/Contents/Res ources: recursive directory loop

That just looks all screwed up.

jfontana Author

Level 1

1 points

Jun 27, 2010 11:03 AM in response to etresoft

Etresoft,

Let me tell you I really appreciate the time you are taking to help me out.

etresoft wrote:

jfontana wrote:

Perhaps it would be good to step back a minute and ask some basic questions. It is common for people to start down a path, get stuck and then ask for help on how to resolve the sticky situation. The correct answer is often that they are on the wrong path to begin with.

Wise words.

HOME=/Volumes/DATA

I don't like the looks of this. How did you accomplish this? It doesn't look right. If you want your home directory to be on a different volume, it would be best to mount that volume at /Users.

Mmm, I'm embarrassed to admit I did something pretty stupid. I simply followed the directions I found here:

http://superuser.com/questions/33935/how-to-make-the-users-directory-a-different -partition-in-snow-leopard

What I initially wanted to do was to separate my data directories and my applications into two different partitions but stupidly I also reassigned my home directory to the new partition where I had moved my data.

Having been a Windows user so many years I learned that separating data and applications into two different partitions was the most efficient and safe way to configure a PC. For one thing you can clone the partition where you have the OS, applications and configuration files and keep it to a small size. If something goes wrong in the hard disk (or, as it is the norm with Windows, everything starts going slow and your computer starts to be unstable) it is not likely to affect the OS and the data at the same time. You then simply copy the image onto the partition where you had the Windows installation and you are good to go. If the data partition is the one that is screwed up, well, that's why you need to back up. In Windows at least many people argue that the computer runs faster if you have your data and your applications in different directories.

Now, I realize that with the OS X arquitecture what I did doesn't make that much sense since many of the configuration files live in the Home directory (which I inexplicably reassigned to the new partition) and fragmentation is not an issue.

That is, because I wasn't using my brain instead of simply moving the directories with my data to the new partition, I made the new partition my home directory following the instructions from the link I pasted above. Don't ask me why. I am myself wondering about that. As I said, my brain must not have been working properly. What is surprising now that I realize what I did is that the system has been working without any apparent problem all this time

What can I do to fix it? Do I go back to system preferences > accounts > advanced options and I assign my home directory to Users/josepmfontana again? Or is there a better, safer way to do this?

You can't modify USER or HOME this way. Well, perhaps you can, but you shouldn't. Bad things will happen. If you want to change USER, you should create a new user account. If you want to change HOME, you should do it through the Accounts pane in System Preferences. Select your account and right/control click on it to bring up "Advanced Options"

As I said, this is what I did but I had no need to change HOME since all I wanted was to have my data files on a different partition. Now the question is, is it to late to reverse this?

Actually, I found another unexpected problem. grep stalled with the following error message:

grep: warning: /Volumes/DATA/../Macintosh HD/Applications/VirtualBox.app/Contents/Resources/VirtualBoxVM.app/Contents/Res ources: recursive directory loop

That just looks all screwed up.

Yes, but do you think this is related to having moved my home directory to a new partition?

Again, thanks for your help and patience.

Josep M.

etresoft

Level 9

56,656 points

Jun 27, 2010 12:18 PM in response to jfontana

jfontana wrote:
Mmm, I'm embarrassed to admit I did something pretty stupid. I simply followed the directions I found here:

http://superuser.com/questions/33935/how-to-make-the-users-directory-a-different -partition-in-snow-leopard

Some of those instructions are good. There is nothing wrong with moving your home directory. There are some hypothetical benefits it could provide. In practice, with large, modern drives, it is going to be more of a hassle than its worth.

Now, I realize that with the OS X arquitecture what I did doesn't make that much sense since many of the configuration files live in the Home directory (which I inexplicably reassigned to the new partition) and fragmentation is not an issue.

The logic behind moving your home directory is still sound. It is just that with Time Machine and Migration Assistant, it isn't that big of an issue.

What is surprising now that I realize what I did is that the system has been working without any apparent problem all this time

What can I do to fix it?

It isn't broken. It isn't standard, but it is a setting that is user configurable.

Do I go back to system preferences > accounts > advanced options and I assign my home directory to Users/josepmfontana again? Or is there a better, safer way to do this?

You could do that if you wanted to. That is the best way. If you copy your home directory to the new location and make sure it has the correct permissions and ownership, you could reassign, logout, and log back in without missing a beat. However, there is some risk involved if you don't get the correct permissions and ownership. It will be fine the way it is.

As I said, this is what I did but I had no need to change HOME since all I wanted was to have my data files on a different partition. Now the question is, is it to late to reverse this?

No. You can change it back anytime you want. That is why you should always use the HOME environment variable instead of hard coding it.

Yes, but do you think this is related to having moved my home directory to a new partition?

Perhaps. Use VirtualBox to create a new virtual machine and use that as a template to fix the broken one.

KJK555

Level 4

2,906 points

Jun 27, 2010 3:26 PM in response to jfontana

Ahah! Now the truth comes out! 🙂 Relax, though, we'll set you on the straight and narrow. 😉

From a performance standpoint, there is little to gain by placing the user directory on a different
partition , *+on the same drive+*. On a Mac Pro, placing the user account and data an a volume on a
*second disk*, can indeed speed things up (unless the second drive is a LOT slower than the boot
drive). The good news is: using 2 partitions is not particularly detrimental to performance and it
sure makes backups quicker and easier. It makes sure the volume holding the OS files never gets
to full (barring a malfunction of course). It simplifies and speeds up Time machine backups too.
I don't use Time Machine to back up my system files, I use another program. I do use TM for
my data and the OS I clone using Carbon Copy Cloner.

Do yourself a favor though, and create a spare admin account on your boot (system) volume, in
case something should happen on your data volume account (such as corruption), you will
still be able to login.

Most regular apps never put anything in the profile files, some developer apps will such as
Mac Ports will. Apps that make profile entries are usually apps that depend on the Unix subsystem
and usually they just link up their own subsystem binary package, so their app package will work.

GUI apps don't know and usually don't care what goes on in the Unix subsystem, unless they are
are "front end" for a actual Unix binary app, such as CronniX, which is an Aqua frontend to the
powerful Unix tool "cron".

If a GUI app is having problems running wherever you put it (including GUI front end apps), then
most likely it's the developer's crappy programming messing up the works, and not your OS
layout.
There are an extremely small number of applications that don't like a dual partition arrangement.
usually there is a workaround to solve the problems. Virtual box may be one that complains.
I played with virtual box a little several months ago. If I remember correctly, it did not like being placed
on my secondary drive. I simply moved it to my boot drive /Applications folder and it was okay with
that.

If you think some screwy application is having a hard time finding your users directory, make yourself a symbolic link to link the boot drive to your user directory,
example:
at the terminal prompt;
sudo link -s /Volumes/data/Users/<your user_name> /Users/<your username>
if your (short) user name is "jf"
eg: sudo link -s /Volumes/data/Users/jf /Users/jf

Now anything looking for your user directory will be forwarded to the proper location on the
data drive.

This will fix some poorly written app that assumes your user directory is in the default location,
rather than asking the system where it is.

jfontana Author

Level 1

1 points

Jul 4, 2010 3:28 AM in response to etresoft

Dear Etresoft and KJK555 (whatever your real names are :-))

My apologies for having left the conversation so suddenly without even having thanked you for your help. All of a sudden things got very busy with my work and I had to leave setting things up on my Mac for later.

etresoft wrote:

jfontana wrote:
Mmm, I'm embarrassed to admit I did something pretty stupid. I simply followed the directions [...]

There is nothing wrong with moving your home directory. There are some hypothetical benefits it could provide.

The reason I was embarrassed was because I moved my HOME to /Volumes/DATA and that was totally unnecessary and potentially problematic. In Linux it makes sense to have your HOME on a different partition because of the way you upgrade your system with newer versions. With a Mac, doing this doesn't make much sense. It iis not that I created a new /Volumes/DATA/Users/myusername directory and move all my HOME files there. I simply declared /Volumes/DATA/ my HOME directory.

All that was necessary was to move my /Documents directory and subdirectories (which is where I keep my data) to the new partition and leave the rest (including the HOME directory) in the original partition. That makes more sense for cloning (things can get complicated if you create a separate image for your OS and applications and a different one for your HOME directory). You see what I mean? Doing what I did was against the philosophy I was pursuing: keep data and OS/Applications separated to be able to create images of the OS and applications separately from data backups. I don't know how cloning works with Macs but I imagine it is not different from cloning with PCs: you can only clone a partition, not a collection of directories and subdirectories.

The logic behind moving your home directory is still sound. It is just that with Time Machine and Migration Assistant, it isn't that big of an issue.

Now, I don't have a lot of experience with Time Machine yet because my external drives are formatted for Windows and Linux and I can't write data to them from the Mac (I should have known this but I didn't). Furthermore, I have a NAS which in principle should work with a Mac but every time I try to backup via TM, I get error messages. All this will change very soon because I've gotten so fed up that I've decided to shell out some cash and buy yet another HD. I've done some research and I'm going to get an Iomega Minimax with 1TB and firewire 800 to take care of all my backup and cloning.

My question is, though, Time Machine is really a cloning application or a backup application? I'm a bit confused about the functionality of TM.

If I'm not mistaken, it does something similar to what Windows does: it takes snapshots of your hard disk so that you can restore your system from the last snapshot in case things go wrong. The problem with this for me is that I keep a huge amount of data on my hard disk and then snapshots get to be huge. For this reason, I like to keep different backup copies for OS/applications/system configuration and for data.

KJK555 says he (sorry if you are a she, but the username is not transparent 🙂 ) uses TM for his data and he clones the OS using Carbon Copy Cloner. From this, I infer that you can tell TM which directories to backup. At any rate, I'm a little weary of TM since I have heard some negative reviews and because of all the problems interacting with my NAS.

Do I go back to system preferences > accounts > advanced options and I assign my home directory to Users/josepmfontana again? Or is there a better, safer way to do this?

You could do that if you wanted to. That is the best way.

[...]

Well, I've done it now and everything seems to work. Well, everything except that, as KJK555 predicted, I had problems with VirtualBox. The VMs that I had installed were not visible because I had installed them in the new partition. I moved them (the .vdi files) to the partition containing now the HOME directory and changed the configuration of VB to tell it that the VM was there. VB works but sometimes it aborts once before starting. This didn't happen before. I don't think this is too serious a problem since I can simply reinstall VB.

Actually, I found another unexpected problem. grep stalled with the following error message:

grep: warning: /Volumes/DATA/../Macintosh HD/Applications/VirtualBox.app/Contents/Resources/VirtualBoxVM.app/Contents/Res ources: recursive directory loop

Perhaps. Use VirtualBox to create a new virtual machine and use that as a template to fix the broken one.

Actually, this doesn't have anything to do with VirtualBox. I tried to do some other greps and I got similar grep: warning messages referring to almost every other application I have installed.

Uff, that was a long message. Sorry.

Josep M.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Where does OS X store environment variables?