I found this happeneing too since upgrading from 10.3.x to 10.4 and can't beleive its still there in 10.5.
Mine gets to between 1.2 and 1.6Gb before I get the 'Can't save library' error popup. If it happens late at night then the next day iTunes may be completely unresponsive - can't even bring it in to focus, so have to kill the process in Task Manager.
This iTunes instance has been running for 15min short of one day of continuous playback! and based on it's current memory leakage, it's about to fail.
For what it's worth, you can downgrade to 10.3 (you should be able to find sites archiving previous versions - Apple used to provide a link on their iTunes page, but I can't find it now). I was running or over a week with no decernable memory leakage, but as I wanted to upgrade to iOS5 on my phone, I had to go to 10.5. I've got a MacBook Pro too, so may use that for iPhone updating and see if iOS5 will still sync music on 10.3 ...via direct USB connect of course!
But, you'll also have to find the various guides for downgrading.
Loosly put, you have to locate the backed up .itl files in your profile. It will have a date stamp in the filename to indicate when the upgrades took place.
i.e iTunes Library 2010-11-22.itl
It's a bit of trial and error unless you know how many upgrades you've done since 10.3!
On Vista/Server 2008, the path to the backups is
Users\<your user ID>\Music\iTunes\Previous iTunes Libraries
Make a copy of your current .itl (in the parent iTunes folder) and copy and rename one from the backups. You'll need to rename the file back to the same as the current one (iTunes Library.itl)
Hope this helps and hope that Apple pull the finger out of their cloud!