I've had a similar experience, and what worked for me was to:
* Do a search for the album on Google or CD Universe, and save the album artwork as a jpg in a seperate folder (I created a new "artwork" folder in the iTunes folder on the hard drive;
* In iTunes, select all the songs on the album, right click and select Get Info;
* Using the check boxes, select the attributes you would like displayed (I choose Album, Artist, Genre) and also check the box next to Artwok, then double click the empty artwork box;
* Navigate to the artwork folder, and select the corresponding jpg;
* Click ok and let iTunes update.
This has worked flawlessly for me on my iPod Touch. As mentioned by other users, you need to be careful about "generic" album names such as Greatest Hits or Best Of, as well as making sure sure you don't have two names for the same album... eg my spelling for the album Age of Winters was different, which resulted in the songs being spilt between two albums (one of each spelling).
Sorry for rambling on, I hope the steps I've outlined above help the issue for other users.
Cheers,
Ben