l008com

Q: `hdiutil` Option "-notremovable" Doesn't Work

I don't mean to repost my similar question, but what I thought was an OS X Server RAM disk based problem, I have come to find out is a general 10.6 problem with `hdiutil`.

 

So the -notremovable option in hdiutil is a pretty obscure feature. I don't think many people use it which is why you don't get much when you google it. You get a few threads from other people trying to use it in 10.6, for different things than I'm trying to use it for, but with the same result.

 

I'm using it to mount a RAM disk. I've been using this RAM disk script for a few years in 10.5 and it works great. I've also tested it in 10.7 and 10.8, it works great in both of those too. The only problem is 10.6. From what I can tell, -notremovable fails in every version of 10.6, server or not server. If you take out the -notremovable flag from your not-working hdiutil command, it will work fine.

 

The problem for me is that without that option, your RAM disk gets mounted as a "removable" disk, like a mounted disk image. With that option, it gets mounted as a non removable disk, like a hard drive. That is what I need for two reasons. One, it lets me back up the RAM disk using Time Machine, which is very useful. And two, disk images are ejected before shutdown scripts are run, but hard drives are ejected afterwards. So my shutdown script to copy the contents of the RAM disk doesn't work, because the RAM disk is gone before script even runs.

 

The exact command I'm using (that runs as root) is this:

 

/usr/sbin/diskutil erasevolume HFS+ "web-ram" `/usr/bin/hdiutil attach -notremovable -nomount ram://2200000`

 

In 10.5, 10.7 and 10.8, that creates an ~1GB RAM disk, that gets mounted to the desktop as if it was a physical disk.

 

A little more background info. The 10.5 machine I have now is a G5 Xserve. It is being replaced by an early 2008 Xserve running 10.6. Upgrading to 10.7 isn't an option for me, because the primary purpose of this server is web serving, and Apple really gutted the web server functionality of 10.7. I'd have to totally bypass the server admin tools and manually manage apache which 1) I don't want to do and 2) if I'm going to do that, why run OS X Server at all?

My usage is quite simple, I store all of my web folders on the RAM disk (except a few folders of larger rarely accessed documents, which live on disk). It works very well, especially since even with a bunch of sites, we're only talking about approximately 40 MB of files. So keeping it all in RAM is very easy, and provides the ultimate in high speed, low seek time access.

 

So if I have to, I can move my web folder back to the hard disk, and run the server with that more traditional setup. But I'm really hoping there is some way to make this work. Some other way to mount the RAM disk, or some undocumented way hdiutil works in 10.6 that is somehow different. It wouldn't be the first time that has happened.

 

FYI I benchmarked this Xserver's RAM disk and I was able to get about 1 GB/sec read speed and 2 GB/sec write speed. The general idea is I want to remove the slow bottleneck that is hard disk speed from my server's time sensitive functions, so the web sites can flow out of this thing like a flood. I need my server to be able to take in stride, the huge burst of traffic that I'm always hoping I get

Xserve, Mac OS X (10.6.8), 8 x 3.0 GHz, 8 GB RAM, Early 2008

Posted on May 16, 2012 3:26 AM

Close

Q: `hdiutil` Option "-notremovable" Doesn't Work

  • All replies
  • Helpful answers

Page 1 Next
  • by l008com,

    l008com l008com May 20, 2012 8:50 PM in response to l008com
    Level 1 (43 points)
    Notebooks
    May 20, 2012 8:50 PM in response to l008com

    Any thoughts?

    Anyone?

  • by twtwtw,

    twtwtw twtwtw May 21, 2012 8:58 AM in response to l008com
    Level 5 (4,935 points)
    May 21, 2012 8:58 AM in response to l008com

    Thoughts, but no obvious solutions.  it's unlikely that the hdiutil utility changed that much from 10.5 to 10.6 to 10.7. That suggests that there's something peculiar about your 10.6 installation that's producing different behavior.  Best thing to try as a diagnostic is to boot into safe mode on 10.6 and see if you still have the problem.  If not, then something you did to your system is getting in the way.

     

    I'd also suggest (as a general rule) that you avoid this kind of complex syntax.  Mount the disk in one line, then erase it in the next.  Trying to do it all in one line looks cool, but I wouldn't trust it to be consequence-free.

  • by l008com,

    l008com l008com May 21, 2012 12:04 PM in response to twtwtw
    Level 1 (43 points)
    Notebooks
    May 21, 2012 12:04 PM in response to twtwtw

    I have tried this command on other 10.6 machines, and even through machines netbooted off of a 10.6 images. No dice.

  • by twtwtw,

    twtwtw twtwtw May 22, 2012 7:58 AM in response to l008com
    Level 5 (4,935 points)
    May 22, 2012 7:58 AM in response to l008com

    well, I switched into my 10.6 partition and tested.  I see the problem you're having, and it seems to be a function of diskutil, not hdiutil.  when I split the commands up, the hdiutil segment works correctly, but diskutil throws the following errors:

     

    Started erase on disk2

    Unmounting disk

    Erasing

    newfs_hfs: write (sector 0): Cannot allocate memory

    Mounting disk

    Could not mount disk2 with name (null) after erase

    Error: -9944: File system formatter failed

     

    apparently in 10.6 there's some kind of bug in creating the file system in this context. 

     

    I'm not sure why you're using diskutil here anyway.  wouldn't it be easier just to create the file system using hdiutil?

  • by l008com,

    l008com l008com May 22, 2012 9:21 AM in response to twtwtw
    Level 1 (43 points)
    Notebooks
    May 22, 2012 9:21 AM in response to twtwtw

    At first, it does seem like the problem is diskutil. But the more you play with it, the problem does end up being hdiutil. No matter what you do. it doens't work with -notremovable and it does work without it. BUT if you have any specific things you want me to try, I'm more than happy to give it a whirl.

  • by snow-punk,

    snow-punk snow-punk Jun 14, 2012 11:27 PM in response to l008com
    Level 1 (5 points)
    Jun 14, 2012 11:27 PM in response to l008com

    Hi,

     

    I sucessfully created a notremovable ramdisk by adding the option -kernel when using the tool hdid. Tested on Mac OS X 10.6.8.

     

    Cheers,

    snow-punk

  • by l008com,

    l008com l008com Jun 14, 2012 11:36 PM in response to snow-punk
    Level 1 (43 points)
    Notebooks
    Jun 14, 2012 11:36 PM in response to snow-punk

    Can you share the full command you are using?

  • by snow-punk,

    snow-punk snow-punk Jun 15, 2012 12:23 AM in response to l008com
    Level 1 (5 points)
    Jun 15, 2012 12:23 AM in response to l008com

    Attention, the following lines are not ready for a final script, cause they contain the sudo command. I post here just the ready-for-terminal version of my tests.

     

    #creating the disk

    name=ramdisk

    #size in MB

    size=64

    device=`sudo hdid -nomount -kernel -notremovable ram://$((2048*$size))`

    sudo diskutil erasevolume HFS+ $name $device

     

    #testing stuff

    touch /Volumes/$name/test

    ls -l /Volumes/$name

    hdiutil detach $device

    ls- l /Volumes

    ls -l $device

    sudo hdiutil mount $device

    ls -l /Volumes/$name

     

    The testing stuff section shows that I have no clue atm, how to remove the device later. So it will be existing and maybe consuming RAM until next reboot. So I guess when scripting this, one has to persist the $device name on which the ramdisk was created. Checking the existence of $device via

     

    test -e $device

     

    and then deciding, to run the create command with diskutil if the $device has not been created or just mounting the already existing $device.

     

    If you or somebody else finds a way to remove the notremovable , I d appreciate to read a solution here.

     

    //snow-punk

  • by l008com,

    l008com l008com Jun 15, 2012 10:27 AM in response to snow-punk
    Level 1 (43 points)
    Notebooks
    Jun 15, 2012 10:27 AM in response to snow-punk

    I'm trying it your way but alas I am having no luck at all I tried as a single-line and as a multi-line, but still no love. FYI my script is being run as root.

     

     

    One-Liner:

     

    /usr/sbin/diskutil erasevolume HFS+ "webfolder" `/usr/bin/hdid ram://4400000 -notremovable -nomount -kernel`

     

     

    Two-Liner:

     

    ramdisk = `/usr/bin/hdid ram://4400000 -notremovable -nomount -kernel`

    /usr/sbin/diskutil erasevolume HFS+ "webfolder" $ramdisk

     

    Any thoughts?

     

    I also wasn't able to get your script to work either.

  • by snow-punk,Helpful

    snow-punk snow-punk Jun 15, 2012 3:15 PM in response to l008com
    Level 1 (5 points)
    Jun 15, 2012 3:15 PM in response to l008com

    Try this:

     

         sudo /usr/bin/hdid  -notremovable -nomount -kernel ram://128000

     

    Show the ouput here if it is not a device name.

     

     

    Then if the 1st command was successful do:

     

         sudo /usr/sbin/diskutil erasevolume HFS+ "webfolder" <DEVICE_NAME>

     

    <DEVICE_NAME> is the output of the command before.

     

    Show the output here, if you still need a third eye.

     

    Btw your two liner has spaces arround the operator = . This is not correct bash syntax.

     

    //snow-punk

  • by l008com,

    l008com l008com Jun 15, 2012 5:17 PM in response to snow-punk
    Level 1 (43 points)
    Notebooks
    Jun 15, 2012 5:17 PM in response to snow-punk

    So I am able to get some results doing it this way. The disk shows up, but it's not "-notremovable". That is, it's showing up as an external hard drive. Which is a step up from being a mounted disk image, but not as good as showing up as a real HDD. The mounted disk image is not backup-able through time machine, for example.

     

    Also it seems part of my problem is that I'm trying to make disk images that are too big, I think that's why these commands were failing in my script. But any thoughts on the "removability" of these mounted ram disks? I'd love to be able to have Time Machine backup my RAM disk.

  • by l008com,

    l008com l008com Jun 15, 2012 5:24 PM in response to l008com
    Level 1 (43 points)
    Notebooks
    Jun 15, 2012 5:24 PM in response to l008com

    The largest RAM disk I seem to be able to make is a little over 1 GB using this technique. Which I could definitely live with if I could get it to mount... better

  • by snow-punk,

    snow-punk snow-punk Jun 16, 2012 4:07 AM in response to l008com
    Level 1 (5 points)
    Jun 16, 2012 4:07 AM in response to l008com

    Ok I think there are some things you can evaluate:

     

    1. Do a self-developed backup (maybe as a periodical launchdaemon script) which utilizes the power of rsync. Some inspiration can be found here: http://hints.macworld.com/article.php?story=2011010204203424

    That way you can keep a hdd folder in sync with your ramdisk and simply let timemachine backup that hdd folder.

     

    2. I read something about an unsupported option in timemachine to add some include paths to the timemachine:

    http://superuser.com/questions/285225/how-to-make-timemachine-back-up-contents-o f-any-path-or-mounted-volume

    If that works, one problem might be to assure that the ramdisk is always mounted when the backup happens (the discussion about that is included in the link above). Maybe switch off the standard Timemachine scheduler and use a more advanced one: http://www.klieme.com/TimeMachineScheduler.html

    It supports auto-mounting unmounted drives. Ok it just auto-mounts the Backup-drive, but I think there is a chance to modify the daemon that is installed by TimeMachineScheduler and wrap around some script code that also mounts additional devices. But all this is very theoretical, as far as I understand your case, your ramdrive is always mounted. I just wrote that down for other use-cases than yours.

     

    3. On the size limit of 1024MB. Well maybe creating multiple ramdisks and a set of symlinks can help out here. Depends on your application, and what you are doing with the ram space.

     

    One comment to the notremovable: For me this option solves one major problem I had with ramdisks:

     

    Without that option my ramdisk very often vanishes if I closed my macbook's lid or if I just switch from airport network to LAN network.

     

    looking into

     

         ls -l /dev/disk?

     

    showed me that also the device has been removed in that situation.

     

    But now with the notremovable flag, the volume still gets unmounted when the magic system event network-change happens. But, surprise surprise, the device under /dev/ still exists and so a simple

     

         hdiutil mount /dev/disk<some_number>

     

    brings my ramdisk and all its contents back.

    I think I can script this to happen automatically and I am happy now.

     

    So in my opinion the ramdisk is now much more not removable than before.

     

    Good luck,

    snow-punk

  • by snow-punk,

    snow-punk snow-punk Jun 16, 2012 4:17 AM in response to snow-punk
    Level 1 (5 points)
    Jun 16, 2012 4:17 AM in response to snow-punk

    PS: If cosmetics matter: the drive icon can be changed via CMD+i dialog on the volume. The change is stored in some hidden files on the volume then.

Page 1 Next