You don't connect via FW to create a dual display. I think what you are attempting to do is use the large display from your your 24" iMac to display images that are running on your MacBook Pro. First please confirm if this is correct.
If I am correct you are attempting to use a feature called Target Display Mode, however it will not work on your configuration. I would recommend you carefully read Target Display Mode 2011 Macs. In short the ONLY way to make this work is to replace the 2008 iMac with a 2011 or 2012 iMac or buy an external Display and connect it to the MBP. The 2008 iMac does not have Thunderbolt and cannot be upgraded to support it so your only alternative is either to buy another external Display to to the MBP or replace the current iMac.