Ugh. This happened to me tonight and it took quite awhile to figure out a fix.
BACKGROUND -- I have an iPod Touch 3G, 64gb. I am on iOS 5.1.1. I sync VBR MP3 files and I do no conversions upon syncing. Nor do I sync by wi-fi. With this setup I have been syncing without any major issues for years. I have about 6,000 songs on my Touch. Suddenly today right after a sync I noticed dozens of albums on my Touch had the wrong art. The covers were all correct in iTunes, but somehow on the Touch many were all mixed up. So I deleted all the music on my Touch and re-synced all music back to the device. This took hours.
PROBLEM -- While the covers were no longer mixed up, a new problem arose -- Now about 60% of the albums on my Touch had no art at all. Of course, as before, all the art was fine in iTunes. I tried syncing again a few times but it didn't fix things. Finally I figured out that anything which will cause an album to be imported again to the device will fix the missing art. So what's the most convenient way to do that? One way to do it is to remove the album from your library completely, then sync, put the album back into the library and then sync again. But this is a major pain. And simply adding back the cover art to the album won't make it re-import into the device.
SOLUTION -- So the simplest method I found was to write to an empty tag field, which will cause the song to be copied over again to your device. I chose the "Sort Artist" field. First select all the songs in an album, then select "Get Info" and go to the "Sorting" tab. In the "Sort Artist" field just type in the name of the artist ... or copy and paste in the name of the artist from the "Info" tab. If there was already an artist listed under "Sort Artist" what I did was delete it, then hit OK to save, then go back into the tracks and add the name of the artist back again. Now when you sync, all these tracks will be copied over to your device again, and the missing art should show up. Using this method, you can do all albums by a single artist at once. For example I have 13 Beatles albums in my library. So I sorted by artist, selected The Beatles, then selected all the tracks for the 13 albums at once and made the change.
This worked for me and hopefully it might help someone else. It's still a chore to fix it all, but at least doing it by artist makes it go a little faster.
This problem cropped up shortly after I installed the new iTunes 11 so I'm wondering if that caused the issue in my case.