Level 1

95 points

EXS in L8 versus L7: no more 3.5 GB RAM limit (with screen grabs)

okay

i’ve done some testing of EXS in Logic 7.1.1 and Logic 8. while all of this testing originated in the “is logic 64 bit” thread, i’ve started a new thread to post these results since, honestly, i really have no idea where to post in that old thread anymore. suffice it to say that Orren and Rohan are absolutely correct -- if we use Virtual Memory in EXS in Logic 8, we are no longer limited by Logic's 3.5 GB RAM limit for EXS and we can now use all of the RAM in our computers with EXS and Logic.

a couple of notes:

1. all songs were created and/or opened on a fresh system directly after a reboot. i never reopened a song or switched songs without a reboot in between. and all testing was done on my dual G5 with 8 GB of RAM, OS 10.4.10, Logic 7.1.1 or Logic 8.

2. i never had any programs other than Logic, Activity Monitor, and Grab open.

3. all testing was done with the EXS Virtual Memory option Active and set to “disk drive speed: slow” and “disk recording activity: extensive”. one of the confusing things in reporting this testing has always been that both EXS and Activity Monitor have a thing called “Virtual Memory” and they are not the same thing. i use Activity Monitor VM as my “how much RAM is this program using” reference because it is the one consistent figure i can find: when Logic hits 3.55 GB of Virtual Memory space in Activity Monitor, it crashes. but Activity Monitor’s Virtual Memory figure is a completely separate thing from the EXS Virtual Memory option. so from here on, when i refer to VM, it is the figure as reported by Activity Monitor. please know that the EXS VM option was active for all testing reported here.

so i started by creating a song in Logic 7 and i added EXS instruments (and ONLY EXS instruments) to the song until i was about to hit the 3.5 GB Virtual Memory limit. as you can see in this screengrab...

...once i had loaded 18 EXS Vienna Symphonic Library instruments, Logic’s Memory figures in Activity Monitor were:

Logic Pro 7.1.1

Real Memory: 3.08 GB
Shared Memory: 47.4 MB
Virtual Memory: 3.42 GB

the system was using 4.15 GB of RAM, leaving 3.85 GB of RAM free. when i tried to load a 19th EXS VSL instrument, the program crashed.

so i then rebooted and opened this same song in Logic 8. as you can see in the screengrab, the memory figures as reported for Logic 8 are very different:

Logic Pro 8

Real Memory: 1.94 GB
Shared Memory: 562 MB
Virtual Memory 2.45 GB

Logic’s Real and Virtual memory figures are much smaller, but the Shared memory figure is much larger. also, the system is using almost the exact same amount of RAM: 4.13 GB (leaving 3.87 Free).

i then proceeded to add EXS instruments (and ONLY EXS instruments) to this song (it has a new name because i didn’t want to save over the original song) until the System RAM was almost all used. as you can see in the following screengrab, the song now has 32 EXS VSL instruments loaded and the system is using 7.68 GB of RAM.

what’s interesting is that despite all of the added EXS instruments, Logic’s memory figures have seen little change. they are:

Real Memory: 2.12 GB
Shared Memory: 571 MB
Virtual Memory: 2.62 GB

and while Activity Monitor isn’t showing a process that is responsible for storing the EXS samples, all 14 of the new EXS instruments are playable (as are the original 18).

so clearly what Rohan and Orren reported about EXS in Logic 8 is true -- we can now load/stream more instruments in EXS in Logic 8.

to test further, i removed a couple of instruments from this song so that i wouldn’t be running quite to the edge of my system RAM, saved it, rebooted and reopened the song. as you can see in this screengrab...

there are now 28 EXS instruments open and Logic’s memory figures are reported as:

Real Memory: 2.07 GB
Shared Memory: 562 MB
Virtual Memory: 2.52 GB

The system is using 6.75 GB of RAM, leaving 1.25 GB free.

so then i added 7 Altiverb plug ins to 7 channels in Logic 8 (different settings for each altiverb instance so they were each using their own RAM).

as you can see, not only has the System used more RAM, but Logic’s memory figures have increased as well because Altiverb is loaded into Logic’s RAM. Logic’s RAM figures now are:

Real Memory: 2.63 GB
Shared Memory: 582 MB
Virtual Memory: 3.15 GB

so i then proceeded to add two more EXS instruments to this song and you can see that the System is now using a lot more RAM (around 750 MB more RAM) but Logic’s memory figures have remained almost the same:

Real Memory: 2.65 GB
Shared Memory: 587 MB
Virtual Memory: 3.16 GB

and, of course, all 30 EXS instruments are playable. how well they will perform in real world use remains to be seen. but they all work.

finally, as near as i can tell, Logic 8 doesn’t let EXS have its own memory space until Logic hits around 1.75 GB of Virtual Memory (as reported in Activity Monitor). before that point, it appears to me that Logic keeps EXS “inside” the Logic memory space. the “real” and “virtual” memory figures for Logic continue to rise as EXS instruments are added and the “shared” memory figure for Logic stays at around 40 MB.

but after 1.75 GB of VM, the “shared” figure starts to rise with each instrument until Logic hits around 2.4 GB of VM, at which point, all EXS appear to be loaded into non-Logic RAM because the “used” System RAM increased but all Logic figures remain almost the same. (i’m sorry if describing this is confusing -- i’m doing the best i can)

anyway, i apologize for the long post, but i wanted to clarify as best i could what the state of EXS is on Logic 8 for anyone still curious.

while i certainly make no claim to know how or why this is happening, i can say that on my system, what Rohan and Orren have reported is correct: EXS in Logic 8 is no longer limited to Logic’s 3.5 GB RAM limit so long as you have the EXS Virtual Memory option active (if it isn’t active, the old RAM limit remains).

thanks to all who helped get to the bottom of this. i hope this post is helpful in demonstrating what EXS can now do in Logic 8. if not, well, i tried. 😉

cheers

Posted on Sep 18, 2007 12:49 PM

103 replies

Sep 25, 2007 9:05 AM in response to derivativemusic

...in other words, it takes a **** of a long time to load up a sample-based template when you have 8GB of RAM (and are running programs outside the DAW). With 16 or more GB it would be intolerable.

xs4is

Level 4

2,800 points

Sep 25, 2007 9:11 AM in response to Mike Connelly

Rohan,

Some of the confusion of this topic might be your wording. neither Logic or the EXS are accessing more than the 32 bit allotment of RAM. The EXS needs multiple instances to "go past the limit." But no one instance of EXS will go past 3.5 gigs of RAM. This implies that the EXS is storing it's memory outside of Logic, even though Activity Monitor doesn't show it.

Mike,

What are you getting at? You keep arguing against 4 well respected members of this forum. Justin C alone is probably the smartest person you've ever communicated with (I know him in real life....he's scary smart). We know you want a 64 bit Logic, but you don't seem to know what benefits/disadvantages that would have (except the RAM allocation). This implies to me that you have only a vague idea of what you are talking about at best. Rather than arguing with these nice folks, you should be asking questions, trying to understand what it is you are asking for.

X

Sep 25, 2007 9:30 AM in response to derivativemusic

Good point about the load times. But that's not really a reason to not implement the functionality.

Sep 25, 2007 9:35 AM in response to xs4is

xs4is, I have explained in detail why I'd like Apple to update Logic to be able to run as a 64 bit host. I am aware of the benefits and disadvantages - for me and others who need the functionality, the benefits outweigh the downsides.

Assuming apple ships it as a fat binary and users have the option of continuing to run it as a 32 bit app on 64 bit hardware and 64 bit OS, what objection do you have to apple creating a 64 bit build?

Sep 25, 2007 9:46 AM in response to xs4is

Rohan,

Some of the confusion of this topic might be your wording. neither Logic or the EXS are accessing more than the 32 bit allotment of RAM. The EXS needs multiple instances to "go past the limit." But no one instance of EXS will go past 3.5 gigs of RAM. This implies that the EXS is storing it's memory outside of Logic, even though Activity Monitor doesn't show it.

sure xs4is, you may well be write about my wording, and what i have quoted you here is exactly as i understand it myself.

i have tried to be as clear i can about what i know to be going on, what i can infer from that knowledge, and what i can speculate about. somehow we have managed to get bogged down on the technicalities - precisely the opposite to what is important here.

the important thing to know is that as far as we are concerned there is no reason that 32-bit apps cannot access memory outside of its usual limits, that it is possible for 3rd party's to do the same following apples lead.

the implication is that we can abandon the hype surrounding whether an application is 64-bit or not and put it to 3rd parties whose libraries we want to use that they should do the same as apple so that we can use large libraries and exploit the ability to use RAM with the attendant benefits.

wringing our hands about the question of 64-bits does nothing to further this.

Sep 25, 2007 9:52 AM in response to Rohan Stevenson1

Rohan, assuming apple ships it as a fat binary and users have the option of continuing to run it as a 32 bit app on 64 bit hardware and 64 bit OS, what objection do you have to apple creating a 64 bit build?

Orren

Level 1

0 points

Sep 25, 2007 11:11 AM in response to Mike Connelly

{quote:title=Mike Connelly wrote:}
Rohan, assuming apple ships it as a fat binary and users have the option of continuing to run it as a 32 bit app on 64 bit hardware and 64 bit OS, what objection do you have to apple creating a 64 bit build?{quote}

Talk about selectivity!

I have mentioned previously:
1) A fully 64-bit application means the GUI runs at half speed (without some fancy footwork on the part of the developers)
2) A fully 64-bit application means that you will only be able to use half as many total tracks, as the audio will need to be blown up in memory to fill a 64-bit address (this was told me to directly by an audio developer, and no I can't go into any more detail.). Note that I don't mean the maximum number of channels. Right now, for example, Logic 8 supports 255 channels, but good luck actually using more than 90 mono tracks of 24-bit/96kHz audio, even on a Mac Pro with a seperate 7200RPM drive for audio. With a fully 64-bit Logic, you'd be lucky to get 45 mono tracks of 24-bit, 96kHz audio.

So since you love to question others, please answer the following questions:
1) How will your work benefit from a Logic that is twice as slow?
2) How does reducing Logic's maximum streamable track count in half make it a better application?

Orren

Sep 25, 2007 12:10 PM in response to Orren

Sonar is already shipping a 64 bit version that can access extended memory. If those two issues are real, shouldn't those be showing up on that app? Shouldn't Sonar users be seeing half the GUI performance and half the track count?

The results I've found online so far from Sonar users have been that it runs the same or better on the 64 bit version. Why am I not seeing the Sonar users screaming out in horror about how 64 bits crippled their system? I should note that they have been screaming about the 64 bit version of Vista, it seems to be performing much worse than XP64.

Here's a set of benchmarks, these show a fairly small dip or about the same results on the xp64 version compared with xp32.

http://www.dawbench.com/blofelds-xp-v-vista.htm

Or do you think for some reason that OSX will have much worse 64 bit performance than on the windows side?

I'll answer your questions even though they seem to be based on wrong assumptions.
1 GUI performance isn't the bottleneck for the sessions I'm working on, RAM is.
2 Streaming audio tracks isn't the bottleneck for the sessions I'm working on, RAM is.

Justin C

Level 6

8,700 points

Sep 25, 2007 12:58 PM in response to Rohan Stevenson1

Rohan Stevenson1 wrote:
there are advantages and disadvantages to logic both going 64 bit or staying 32 bit. clearly they decided that the advatages in remaing 32-bit outweighed the disadvantages.

There are several factors that force the front end to remain in 32 bit while used with Tiger. And yes, there can be a significant performance and compatibility hit (from what I have read) for an app like Logic to switch at this time (if it were a possibility).

parts of the program that require 64-bit processes have them, but where it doesn't matter it remains 32 bit.

Really? Sorry, I haven't read everything surrounding this topic (and I don't mean to sound skeptic). Where did this info come from? I'd like to read more. My surprise originates from the idea that a secondary processes would be spawned (a la VSL).

J

Sep 25, 2007 1:02 PM in response to Rohan Stevenson1

{quote:title=Rohan Stevenson1 wrote:}
surely, the next stage is to be able to determine what is played back from RAM and what is streamed from disk? for example, you might want to optimize your library in such a way that multivoice instruments wiht long tails such as piano and harp are loaded into RAM where you can get much more value from faster streaming, as opposed to perhaps certain single line instruments which could be streamed from disk without taxing it too much.{quote}

I think this is a fine way to go, it's already done with databases in the corporate world where heavily accessed data is "forced" into RAM on large servers in order to move load from the disk.

I don't see that there is a great deal of difference between data sets and audio samples (although I know far more about the former than the latter). But I think you're right, this is a really smart way to go, as the composer/producer would have a pretty good idea where the resource would best be utilized.

Keep your fingers crossed.

T.

Sep 25, 2007 1:09 PM in response to Justin C

Justin, I assume he's referring to this bit from the logic tech specs on the apple website.

"Internal audio resolution: 32-bit floating point; 64-bit precision where required"

I assume this means the audio data is stored internally at 32 and 64 bits and has nothing to do with the code.

Tony, I agree that it would be useful to have more control over ram use versus streaming, it would be great to see that in addition to making more ram available.

Justin C

Level 6

8,700 points

Sep 25, 2007 1:10 PM in response to Rohan Stevenson1

10.4 is 64-bit as well. leopard comes with other optimisations which we should benefit from such as faster GUI and some other fancy stuff i didn't take the trouble of understanding.

Here's the simple explanation:
The (underlying) OS is capable, but many of the higher OS X libraries are not. Basically, what is available in Tiger is very low level and restricted to certain programming languages. If you peek at:

/System/Library/Frameworks/

you'll see a very large collection of frameworks (or Libraries of executable code and resources which programmers use... sort of like a sampler instrument library + samples). Most of these frameworks do not support 64 bit in Tiger. So... what has to happen is the frameworks need to be updated to work properly using 64 bit addressing. This is where the saying "If it has a UI, it will not run in 64 bit on Tiger" comes from. Without these frameworks, developers would have to reinvent too much (basically an OS). Plus, it would require significant workarounds to be a good Tiger citizen... not really an ideal solution.

J

Orren

Level 1

0 points

Sep 25, 2007 1:32 PM in response to Mike Connelly

{quote:title=Mike Connelly wrote:}
Or do you think for some reason that OSX will have much worse 64 bit performance than on the windows side?{quote}

Short answer: yes. But that doesn't tell the story.

The full story: You really like black and white terminology, don't you? 🙂 OSX doesn't have "worse" GUI performance than Windows...but if you stand someone in front of an iMac running OSX Tiger, and a Dell running WinXP, the Windows machine will seem to have a more responsive interface—even if the Mac is faster.

Windows simply doesn't have the deeply complex imaging and graphics engine that OSX does. Even Vista, although it looks more 3D, doesn't have nearly the imaging technology that OSX does. So this extra imaging power takes cycles, resulting in a greater CPU hit. And more CPU means more time. And more time means a slower interface.

Now, OSX Cocoa technologies such as CoreImage 2D, Quartz Extreme, and so on, purport to move elements of the application GUI onto the Graphics chip. This theoretically reduces the CPU overhead, as well as runs the GUI of the application far quicker. I say "purport" because even though the marketing text implies that these things exist in Tiger, they are not activated for general purpose apps, although they are in Leopard. So the theory is, a Cocoa GUI that takes advantage of these technologies would gain some benefits on systems with powerful enough graphics cards.

Now, the fastest Tiger interfaces are ones that are "homegrown" and use their own ultra-fast graphics engines. Like, for example, Cubase. On the other hand, those homegrown graphics engines load onto the CPU and will not gain benefits from being a Cocoa UI. Applications such as Logic 8, which are Cocoa, use the native OSX engine, which is by definition a general purpose, not-as-super-optimized-for-a-DAW graphics engine. But, it will be able to take advantage of all the cool Leopard speedups in GUI. Theoretically. 😉

So to sum up that incredibly long answer: A 64-bit OSX will be doing more than a 64-bit Vista, and therefore will not be as responsive. Therefore, applications running at 64-bit in Leopard will run slower than applications running in 64-bit in Vista. There are technologies in Leopard that can help, theoretically, and Logic 8 should be able to take advantage of them "out of the box." But we'll have to see if that is truly the case.

As for the audio file streaming issue, well, the fact is that Logic has always had extremely poor track count maximums in comparison to other applications. I think Cubase can run something like 300 tracks at 24/44.1, and Logic 7 can't do more than 128 or so. It has improved, but a 64-bit Logic, in which all tracks needed to fill 64-bit addressing space (just as now, all 24-bit tracks have to fill a 32-bit addressing space) would take more time, and stream slower. If SONAR gets the same track count, I can only assume that SONAR was always ahead of Logic in terms of track count, and will remain so.

{quote:title=Mike Connelly wrote:}

I'll answer your questions even though they seem to be based on wrong assumptions.
1 GUI performance isn't the bottleneck for the sessions I'm working on, RAM is.
2 Streaming audio tracks isn't the bottleneck for the sessions I'm working on, RAM is.{quote}

Fair enough. But again, if performance were truly 50% of what it is now (and who knows what a theoretical 64-bit Logic would perform like), it just might become the bottleneck, even if your RAM problems were lifted.

In the end, I see there to be two things of great importance:
1) The fastest, highest performance Logic that is possible
2) A Logic that is conducive to using sample players and libraries that require lots of RAM

I don't care if this is done by making it a 64-bit application, making half the application 64-bit and running it as a separate process, making none of it 64-bit and tying hamsters or wheels inside every Mac, or whatever. What's important is the end result.

Orren

Justin C

Level 6

8,700 points

Sep 25, 2007 1:35 PM in response to Mike Connelly

"Internal audio resolution: 32-bit floating point; 64-bit precision where required"

I assume this means the audio data is stored internally at 32 and 64 bits and has nothing to do with the code.

Here's how I read the quote from the tech specs page:
Up to 64 bit (in application) storage for audio processing (i.e. samples passing through the mixer) - which will affect calculations in many programs. At this time, I would guess Logic is still mostly 32 bit, but some programs will use 64 bit internal sample storage through their calculations. This is along the lines of internal resolution and has nothing to do with memory allocation (for things like samplers).

This isn't necessarily the source of Rohan's information though... i.e. I haven't read much of the new manuals/documentation to date. I just wasn't able to locate (and had not encountered) this bit of info.

J

Sep 26, 2007 12:28 AM in response to Justin C

This isn't necessarily the source of Rohan's information though... i.e. I haven't read much of the new manuals/documentation to date. I just wasn't able to locate (and had not encountered) this bit of info.

sorry, justin i don't have time to go into detail, i have already spent far too much time on this issue. my info comes from a developer who was explaining the 64-bit myth and how it relates to logic. not being a programmer i didn't understand the technicalities but the thrust of what he was saying is that logic does not need to be a 64-bit program to access more memory. it then came to light that the exs was able to load past the usual amount which was subsequnetly substantiated here.

one of the things he was saying that logic 7 was already using 64-bit processes (not resolution - i don't think, but i couldn't hand on heart swear it) where it was necessary*, and that it wasn't necessary to code the program entirely as 32-bit to gain the benefits of 64-bit technology. zibba and orren have pretty much paraphrased the conversation i had and are far better qualified (actually as are you) to explain the technicalities.

+*incidentally, i suspect the tech specs give the impression that it is audio resolution, but i think, and i stress, i think that that is down to marketing not fully understanding the technicalities, and therefore it is misleading. this not 'wild speculation' on my part, but it could be mis-interpretation of information i received. i could have sworn that my understanding was in the context of processes.+

it was also made clear that 3rd party developers have 'all the information' they need to do the same thing as the exs, which may have meant they had been informed by apple of the technique or it may have meant that apple assumed they ought to be able to figure out how to do it with knowledge already existing. since my informant is non-native english speaking, it came across as though he had been informed directly and i am afraid i may have given that impression also (particualrly to mike) but either way, it is clear that it IS possible now within the current context of logic and tiger.

and that is all that really matters. i was trying to report that; a) the it is incorrect to assume that 32-bit applications cannot access 64-bit memory space and not to engage in the 'hype' surrounding logic going 64-bit b) the exs can extend beyond the 32-bit memory limit for practical use in this way, c) that therefore 3rd party samplers can perform the same trick, and that they should be encouraged to.

this is the information as i have it, and i am not qualified to argue the ins and outs of it, although others have (including yourself), thank god.

i say again, there has been a lot of hype about logic being made a 64-bit application as if that was going to solve everyones problems, and i was attempting report that it is not. the disadvantages outweigh the advantages.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

EXS in L8 versus L7: no more 3.5 GB RAM limit (with screen grabs)