With MacOS Lion most applications build larger caches than with previous versions of MacOS. For me this problem stopped after upgrading my RAM to 8GB.
Also it helps to to make sure that no other applications with huge demands on RAM are running at the same time, e.g. Safari or iPhoto.
To see if you have a problem with RAM launch the Activity Monitor (from Applications -> Utilities) and look at the "Real Mem" column. Clicking at the Column header will sort the processes according to the memory usage.
(Hm, I thought I had opted for not publishing the question.)
Thanks Léonie. I tested all kinds of stuff, used Activity Monitor, and more or less came to the conclusion that it is a problem of having not enough RAM. And the question why the problem would persist thorugh a restart, might have been answered as well: a restart does not clear all cache, while a shutdown does.
My dealer says that my type of iMac has a RAM max of 4 GB (officially). Alas.
By the way: the problem even occurs if GarageBand is the only application!