The ! turns up when iPhoto loses the connection between the thumbnail in the iPhoto Window and the file it represents.
What version of iPhoto?
Are you running a Managed or a Referenced Library?
A Managed Library, is the default setting, and iPhoto copies files into the iPhoto Library when Importing. The files are then stored in the Library package
A Referenced Library is when iPhoto is NOT copying the files into the iPhoto Library when importing because you made a change at iPhoto -> Preferences -> Advanced. (You unchecked the option to copy files into the Library on import) The files are then stored where ever you put them and not in the Library package. In this scenario you are responsible for the File Management.