Do a Apple menu > About this Mac.
You need a Intel processor Mac + about 1-2GB of RAM or so to run 10.6, if it says PowerPc processor then you can't run 10.6. or if you don't have enough RAM it also might not work.
It's possible you got a bad 10.6 disk, and if you want to upgrade 10.6 over 10.5 keeping your programs and files, you have to run the installer that shows in the disk window.
Another method is to hold option/alt key down and boot directly off the 10.6 disk, there it gives a option to install OS X, however I don't know if it will upgrade over 10.5, I suspect it should you can try it.
There is certainly the option to use Disk Utility to erase the drive and then install 10.6, but you lose your 10.5 programs and users files if you erase.
Backup your files off the machine regardless and if the disk doesn't work, call Apple for a new one.
Read on here for more advice before upgrading