Wrong place to put them.
Performance tip: Keep the Desktop clutter-free (empty, if possible)
Mac OS X's Desktop is the de facto location for downloaded files, and for many users, in-progress works that will either be organized later or deleted altogether. The desktop can also be gluttonous, however, becoming a catch-all for files that linger indefinitely.
Unfortunately - aside from the effect of disarray it creates - keeping dozens or hundreds of files on the Desktop can significantly degrade performance. Not necessarily because the system is sluggish with regard to rendering the icons on the desktop and storing them in memory persistently (which may be true in some cases), but more likely because keeping an excessive number of items on the Desktop can cause the windowserver process to generate reams of logfiles, which obviously draws resources away from other system tasks. Each of your icons on your desktop is stored as a window in the window server, not as an alias. The more you have stored, the more strain it puts on the window server. Check your desktop for unnecessary icons and clear them out.
Keeping as few items as possible on the Desktop can prove a surprisingly effective performance boon. Even creating a single folder on your Desktop and placing all current and future clutter inside, then logging out and back in can provide an immediately noticeable speed boost, particularly for the Finder.