I have a 600 GB Aperture library with 40,000 images and startup is quite fast.
Aperture can hold a lot more than that without slowing down; the key is to not have projects that are HUGE (thousands of images). Lots of smaller projects makes more sense; I typically have a project per shoot or a project by day and for multi-day events or trips I just create a folder with the projects within it.
Repair and/or rebuild your Library.
I run some Libraries with 3/4 million Images off of external drives. While any operation on _a lot_ of files (50K? 100K?) takes time, in general Aperture performance when using huge Libraries is acceptable on currently spec'd machines. 41 GB is, for Aperture, well within the "normal" range. You should not experience any problems that are _Library-specific_. If your hardware is not up to the task, or your Library is corrupted, you will encounter problems, but they are not due to the size of you Library.
When you first convert a Library (or create a Library) you must let Aperture run for many hours to finish its 'basement' bookkeeping.
Migrating to a new computer should never present any problems, _as long as there are no problems before you migrate_. Again, repair you Library before any migration.
I don't see a lot of difference in performance between Libraries with predominantly Images whose Originals are managed and Libraries with predominantly Images whose Originals are referenced. The choice of referenced or managed should be made based on storage, not performance.
What is your "organization schema"?
Once you import a file into an Aperture Library, Aperture refers to it as an Original. Aperture has a relatively full suite of tools for managing your Originals. The one thing to keep in mind is that you must use Aperture, and only Aperture, to move, rename, or otherwise administer your Originals.
You will likely find this page in the User Manual informative.
Note that Libraries are not in themselves either "referenced" or "managed". Every Image has an Original. Each Original can be referenced or managed. Changing an Original from referenced to managed, or managed to referenced, is easy using Aperture.
So what happens if I use a referenced library and then have to move the originals to a larger drive? How does Aperture keep track of the moved images and maintain the path info? Is there anything I need to do when I transfer the files so that Aperture can keep track of them?
Aperture has a built-in command "relocate original file(s)". If you need to move originals, let Aperture do it - then will Aperture will know where they are. And if your drive crashes and needs replacing and you have to restore the library from a backup, Aperture has the built in command "Locate Referenced files" to reconnect originals and version.
Aperture can export partial libraries and merge libraries - everything you needed IPLM for with iPhoto is directly built in Aperture. It supports referenced files and working with multiple library files very well.
What about the question of having Aperture organize the photos internally versus using a referenced setup? Does that effect program speed? Is Aperture better than iPhoto at handling migrating across drives and such?
As long as your Aperture library fits on one single drive, "managed" is saver, as William Lloyd said. "Referenced" is the only possibility to distribute a gigantic library across several drives. If you need to do that, keep the library itself on the fastest drive and the referenced originals on other drives.