Check if your "Instruments Library" is properly installed and has the correct permissions codes.
The "Instruments Library" should be in the system Library in the folder
Your System Volume -> Library -> ApplicationSupport -> GarageBand.
If the Library is there, and the permissions are correct, but GarageBand cannot find it, then probably your preferences are corrupt.
Go to your user library and then into your preferences folder. Look for "com.apple.garageband.plist" and throw it in the trash. After you empty the trash, open up garageband. A new preferences file will be created, and the program should open up as long the library exists.