Level 1

95 points

EXS in L8 versus L7: no more 3.5 GB RAM limit (with screen grabs)

okay

i’ve done some testing of EXS in Logic 7.1.1 and Logic 8. while all of this testing originated in the “is logic 64 bit” thread, i’ve started a new thread to post these results since, honestly, i really have no idea where to post in that old thread anymore. suffice it to say that Orren and Rohan are absolutely correct -- if we use Virtual Memory in EXS in Logic 8, we are no longer limited by Logic's 3.5 GB RAM limit for EXS and we can now use all of the RAM in our computers with EXS and Logic.

a couple of notes:

1. all songs were created and/or opened on a fresh system directly after a reboot. i never reopened a song or switched songs without a reboot in between. and all testing was done on my dual G5 with 8 GB of RAM, OS 10.4.10, Logic 7.1.1 or Logic 8.

2. i never had any programs other than Logic, Activity Monitor, and Grab open.

3. all testing was done with the EXS Virtual Memory option Active and set to “disk drive speed: slow” and “disk recording activity: extensive”. one of the confusing things in reporting this testing has always been that both EXS and Activity Monitor have a thing called “Virtual Memory” and they are not the same thing. i use Activity Monitor VM as my “how much RAM is this program using” reference because it is the one consistent figure i can find: when Logic hits 3.55 GB of Virtual Memory space in Activity Monitor, it crashes. but Activity Monitor’s Virtual Memory figure is a completely separate thing from the EXS Virtual Memory option. so from here on, when i refer to VM, it is the figure as reported by Activity Monitor. please know that the EXS VM option was active for all testing reported here.

so i started by creating a song in Logic 7 and i added EXS instruments (and ONLY EXS instruments) to the song until i was about to hit the 3.5 GB Virtual Memory limit. as you can see in this screengrab...

...once i had loaded 18 EXS Vienna Symphonic Library instruments, Logic’s Memory figures in Activity Monitor were:

Logic Pro 7.1.1

Real Memory: 3.08 GB
Shared Memory: 47.4 MB
Virtual Memory: 3.42 GB

the system was using 4.15 GB of RAM, leaving 3.85 GB of RAM free. when i tried to load a 19th EXS VSL instrument, the program crashed.

so i then rebooted and opened this same song in Logic 8. as you can see in the screengrab, the memory figures as reported for Logic 8 are very different:

Logic Pro 8

Real Memory: 1.94 GB
Shared Memory: 562 MB
Virtual Memory 2.45 GB

Logic’s Real and Virtual memory figures are much smaller, but the Shared memory figure is much larger. also, the system is using almost the exact same amount of RAM: 4.13 GB (leaving 3.87 Free).

i then proceeded to add EXS instruments (and ONLY EXS instruments) to this song (it has a new name because i didn’t want to save over the original song) until the System RAM was almost all used. as you can see in the following screengrab, the song now has 32 EXS VSL instruments loaded and the system is using 7.68 GB of RAM.

what’s interesting is that despite all of the added EXS instruments, Logic’s memory figures have seen little change. they are:

Real Memory: 2.12 GB
Shared Memory: 571 MB
Virtual Memory: 2.62 GB

and while Activity Monitor isn’t showing a process that is responsible for storing the EXS samples, all 14 of the new EXS instruments are playable (as are the original 18).

so clearly what Rohan and Orren reported about EXS in Logic 8 is true -- we can now load/stream more instruments in EXS in Logic 8.

to test further, i removed a couple of instruments from this song so that i wouldn’t be running quite to the edge of my system RAM, saved it, rebooted and reopened the song. as you can see in this screengrab...

there are now 28 EXS instruments open and Logic’s memory figures are reported as:

Real Memory: 2.07 GB
Shared Memory: 562 MB
Virtual Memory: 2.52 GB

The system is using 6.75 GB of RAM, leaving 1.25 GB free.

so then i added 7 Altiverb plug ins to 7 channels in Logic 8 (different settings for each altiverb instance so they were each using their own RAM).

as you can see, not only has the System used more RAM, but Logic’s memory figures have increased as well because Altiverb is loaded into Logic’s RAM. Logic’s RAM figures now are:

Real Memory: 2.63 GB
Shared Memory: 582 MB
Virtual Memory: 3.15 GB

so i then proceeded to add two more EXS instruments to this song and you can see that the System is now using a lot more RAM (around 750 MB more RAM) but Logic’s memory figures have remained almost the same:

Real Memory: 2.65 GB
Shared Memory: 587 MB
Virtual Memory: 3.16 GB

and, of course, all 30 EXS instruments are playable. how well they will perform in real world use remains to be seen. but they all work.

finally, as near as i can tell, Logic 8 doesn’t let EXS have its own memory space until Logic hits around 1.75 GB of Virtual Memory (as reported in Activity Monitor). before that point, it appears to me that Logic keeps EXS “inside” the Logic memory space. the “real” and “virtual” memory figures for Logic continue to rise as EXS instruments are added and the “shared” memory figure for Logic stays at around 40 MB.

but after 1.75 GB of VM, the “shared” figure starts to rise with each instrument until Logic hits around 2.4 GB of VM, at which point, all EXS appear to be loaded into non-Logic RAM because the “used” System RAM increased but all Logic figures remain almost the same. (i’m sorry if describing this is confusing -- i’m doing the best i can)

anyway, i apologize for the long post, but i wanted to clarify as best i could what the state of EXS is on Logic 8 for anyone still curious.

while i certainly make no claim to know how or why this is happening, i can say that on my system, what Rohan and Orren have reported is correct: EXS in Logic 8 is no longer limited to Logic’s 3.5 GB RAM limit so long as you have the EXS Virtual Memory option active (if it isn’t active, the old RAM limit remains).

thanks to all who helped get to the bottom of this. i hope this post is helpful in demonstrating what EXS can now do in Logic 8. if not, well, i tried. 😉

cheers

Posted on Sep 18, 2007 12:49 PM

103 replies

Sep 29, 2007 7:23 PM in response to Orren

Sometimes automatic censoring on these boards is really stupid. I wrote that loading lots of samples takes a h ell of a long time. BFD. With censoring asterisks it looks like I said an F-word of a long time, and I would never do that except for when I do.

Anyway Orren, that doc sort of explains what we already found out and sort of doesn't. The way it's worded sounds like EXS can only access a little less than 4GB of its own, yet it also says that it can access as much as you have installed. It also says that the amount of RAM used by other plug-ins in Logic has an effect on the EXS. What does that mean?

Sep 30, 2007 5:24 AM in response to derivativemusic

It is a little bit vague, sounds to me like if it needs more than 4 gig they could be just opening more exs processes. It's good that they've made this fix, but it seems pretty limited. They need to take it further and make more ram available to other parts of the app including third party plugs. I don't really care if they do that with 64 bit code or by calling all plugs as separate processes, I just want to see them provide a solution.

Orren, thanks for the link.

Sep 30, 2007 1:04 PM in response to Mike Connelly

You all seem to have a somewhat deep understanding of this issue.
All i know is, I am dying to load more samples into my kontakt.
When my activity monitor says logic has used 1.99 gigs, everything's
fine. Then, once I reach 2 gigs, it says i'm using 16,777,216 TB. Yes, terrabytes. It crashes soon after. I've been told that this is a problem exclusive to intels. Ironically, g5's can load DOUBLE the
RAM at the moment.

All this hoopla about 64 bit is very interesting to me, and I eagerly
await a solution to the memory adddressing problems, but I'd be ok
for now if i could just use 4 stinkin' gigs.

Does anyone have a solution for me?

Orren

Level 1

0 points

Oct 1, 2007 11:31 AM in response to derivativemusic

{quote:title=derivativemusic wrote:}
Anyway Orren, that doc sort of explains what we already found out and sort of doesn't. The way it's worded sounds like EXS can only access a little less than 4GB of its own, yet it also says that it can access as much as you have installed. It also says that the amount of RAM used by other plug-ins in Logic has an effect on the EXS. What does that mean?{quote}

This is how I understand it:

In Logic's memory space, there is a certain "RAM heap" that plug-ins draw from. Obviously, the size of that heap isn't fixed, it depends on how much RAM is in your machine, how much the system is using at the time, etc. If you don't have 5GB RAM or more in your computer, or if you don't have any instances of EXS that would push the limit, all your plug-ins will happily grab their little bit of RAM from this heap. When there's no more RAM in the heap, and you keep trying for more, you'll get an error message, or Logic will crash, etc.

Since the EXS can now access its own address space if you have 5GB or more RAM installed, it is no longer limited to whatever RAM inside of Logic's memory space. However, until you surpass the amount of RAM available inside of Logic's "RAM heap" the EXS will still grab its memory from that heap.

Assuming my understanding is correct, this is where the use of 3rd party plug-ins come in. If you're not using any plug-ins that are taking up a lot of RAM, there is more memory inside the heap available for the EXS. If you are using a lot of 3rd party plug-ins that eat up that heap, the EXS will use it's own address space earlier. I am also assuming this is a dynamic process, so if you load an EXS instance that takes up the entire internal heap, but then add some 3rd party plug-ins, the EXS will dynamically switch to using a separate address space, if you remove some 3rd party plug-ins it and you're using a small enough sample instrument, the EXS instance can return to the internal heap, etc.

As for the EXS accessing "a little less than 4GB on it's own," I don't think that's the case. But I'm not sure.

Hope that helps,
Orren

iSchwartz

Level 5

4,321 points

Oct 1, 2007 12:58 PM in response to Orren

Thanks for providing that link Orren. Couple of things...

• Wondering why this kind of info has to filter down from a forum member rather than being officially posted (in the form of a sticky at the top of this forum) for the benefit of all.

• I see from the screenshot at that link that the disk speed option is still there. It makes for a very confusing understanding of what VM is. Is it streaming from disk? According to description, no, it's not. So could someone please explain what the various EXS disk options are for?

Thanks.

Oct 1, 2007 1:49 PM in response to iSchwartz

hi iS,

they are undoubtedly confusing. as orren explained in another post, the 'virtual memory' in the context of the exs is infact disk streaming. but as we know its not just disk streaming, it is streaming from RAM as well (specifically the headers). the rest of the options are hangovers from days of yore whe then feature was first implemented. don't forget the ability to stream from disk has been around longer than the ability or the likelihood of having enormous amounts of physical RAM.

so you were given options allowing you to choose scenarios where you may not have had much RAM but plenty of HD bandwidth, and vice versa. choosing an option that presumes you have a rubbish HD will cause logic to load more of the sample into RAM. if you have lots of RAM choosing this option is probalby going to provide you with the best performance.

i say probably advisedly, because i have often found optimizations seem to prefer certain settings (i am speaking more generally) and you might find that you get better performance with the less intuive setting. eg, i often get better performance with the process buffer setting in the audio window set to medium rather than large, even though having a larger buffer should work better if you have lots going on.

i think you have to try these things out for yourself and see what works best for you.

Oct 1, 2007 1:51 PM in response to iSchwartz

As I understand it, with VM enabled, EXS still streams from disk but with the beginning of each sample loaded as a buffer, same as it always has. Now it just allows that buffered memory to grow beyond 4 gig, allowing more samples to be loaded (or to set the VM settings so that they don't need to hit the hard drive as much). The VM settings are still needed because even with the higher ram limit, it's still streaming from disk after the buffered audio runs out.

I totally agree that apple could do a better job of getting this info out.

iSchwartz

Level 5

4,321 points

Oct 1, 2007 2:14 PM in response to Rohan Stevenson1

Rohan and Mike,

I appreciate your replies. But I'm not sure how the scheme of reading headers out of RAM and streaming tails off of disk is any different from disk streaming in the old-fashioned sense. It's not that I don't believe what you guys are saying, but from the EXS page that Orren linked to:

"In Logic Pro/Express 8, the EXS24 addresses its own memory space. This means that EXS24 instances can use as much RAM as is available on the computer."

Nowhere in that doc is there mention of headers loaded into RAM, tails streamed from disk. Unless I can't see the forest through the trees, a read of that document suggests that samples are loaded into RAM exclusively.

So on paper, it's confusing as ****. And if this "new" feature is nothing more than "old" disk streaming (albeit with potentially larger head count vis a vis more RAM access), that feature would be as useless to me as it is now in L7 (VM results in too much latency/glitching).

Anyway, thanks again.

Orren

Level 1

0 points

Oct 1, 2007 2:37 PM in response to iSchwartz

{quote:title=iSchwartz wrote:}
Nowhere in that doc is there mention of headers loaded into RAM, tails streamed from disk. Unless I can't see the forest through the trees, a read of that document suggests that samples are loaded into RAM exclusively...if this "new" feature is nothing more than "old" disk streaming (albeit with potentially larger head count vis a vis more RAM access), that feature would be as useless to me as it is now in L7 (VM results in too much latency/glitching).{quote}

Again, here is my understanding:

First, EXS tries to load 100% of a sample instrument into RAM.
Second, if the EXS instrument cannot load completely into RAM, then the instrument will load sample headers only into RAM, with the rest of the sample being read from the hard drive as needed.

What Rohan and Mike are suggesting, which I too assume, is that this process has not fundamentally changed.

What has fundamentally changed, however, is that the size that an EXS instrument could be before #2 kicked in has vastly increased.

Let me give some specific examples, based on a system with 7GB RAM installed (assume Logic gets up to 4GB of that, and OSX always takes at least 1GB for itself):

Example A—playing a 200MB Sampler Instrument
Logic 7: fully loads into RAM
Logic 8: fully loads into RAM

Example B—playing a 2GB Sampler Instrument
Logic 7: loads headers into RAM, reads the rest of the sample from hard drive
Logic 8: fully loads into RAM

Example C—playing a 20GB Sampler Instrument(!)
Logic 7: loads headers into RAM, reads the rest of the sample from hard drive
Logic 8: loads headers into RAM, reads the rest of the sample from hard drive

Hopefully, that makes (at least my understanding of) it a bit more clear.

Orren

Oct 1, 2007 2:50 PM in response to iSchwartz

iSchwartz, I think that doc is just very incomplete and takes the streaming stuff for granted.

I think it really is just "larger head count". But that may be good enough if it means you can use the highest settings to avoid glitches. Have you had latency and glitching even with the highest VM settings (slow disk, large)? It would be cool if it did this dynamically like Orren says and just fills up whatever ram is available, but I don't know what would keep performance from degrading as it needs to stream more and more from disk.

One thing that may help situations like yours would be if they would expand the VM settings with more possibilites, and loading even more into ram. Or just being able to load everything in ram while using the new extended memory, it seems kind of goofy that you can only use higher ram when you're streaming from disk. I wonder if apple's fix is by doing something like creating a hidden ramdisk behind the scenes and "streaming" from that?

We know that L8 can load more instruments than before, but is it able to play them all back reliably without glitching?

iSchwartz

Level 5

4,321 points

Oct 1, 2007 2:53 PM in response to Orren

Thank you Orren, your examples helped clarify what Rohan and Mike were saying.

At the risk of slinging mud at this improvement... it seems to me that your example C is most telling, for it completely contradicts what a studied read of that EXS doc explains. This is what I get from reading that doc:

"EXS will load/read whole samples exclusively in to greater quantities of RAM than previously possible."

If (per your post) your understanding is correct, I'd say that the documentation warrants clarification. Otherwise I think it would be fair to characterize Apple's documentation as incomplete at best, misleading at worst.

Orren

Level 1

0 points

Oct 1, 2007 2:56 PM in response to iSchwartz

{quote:title=iSchwartz wrote:}
At the risk of slinging mud at this improvement... it seems to me that your example C is most telling, for it completely contradicts what a studied read of that EXS doc explains.{quote}

iScwartz,

Example C involves is a 20GB sample library. How could a 20GB possibly be loaded entirely into 7GB of RAM, which is the example amount of RAM? The point is that reading from disk happens when RAM is exceeded. Would you rather that Logic 8 simply not be able to use sample libraries beyond physical RAM?

Orren

iSchwartz

Level 5

4,321 points

Oct 1, 2007 3:43 PM in response to Orren

Orren,

Of course your 20GB couldn't be fit into 7GB of RAM 😉 but that's not my point...

Let's say you had 5 GB or RAM and you wanted to load eight slightly-less-than .5 GB instruments. Taking into account that (round numbers) OSX takes 1GB and Logic takes 1GB, you have 3GB left for samples. That'll let you load up six instruments with a little room left over. According to your example, assuming there's 7th instrument is going to load headers into RAM, not whole samples. Of course, how could it? You're out of RAM 😉 So all I'm saying is that the documentation is incomplete when it says:

"...EXS24 instances can use as much RAM as is available on the computer..."

And it probably goes without saying, but I think it would behoove any Logic owner wanting to take advantage of this system to maximize on their RAM because apparently Logic will vary the way it loads samples (and vary in its performance of recording and playback) depending on the number of apps loaded.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

EXS in L8 versus L7: no more 3.5 GB RAM limit (with screen grabs)