First of all, do not connect the iPod to iTunes at this time. You're going to make some changes in your iTunes Library. Once that's done, then connect the iPod.
In your iTunes Library, select the album (or, to be more precise, all the songs on the album) and on your keyboard, use the CTRL+I ("eye") keys to open the Multiple Item Information. You will see this image shown below (after clicking on "yes"):

As you can see, the artist name is already highlighted. Simply overtype this with the correct artist. The field below that, Album Artist may have "Various Artists" in it. Delete "Various Artists" for a single artist abum, or type in the name of the main artist if some tracks have guest artists listed. Before you click the OK button, have a quick check on the Options tab (just above the Artist field):

make sure the Part of a compilation box states "No". Now you can click OK.
Hopefully, you use Sync to manage your iPod. If so, all you need to do now is connect the iPod to your computer, iTunes should see it and perform the Sync. The revised album information will now be on the iPod too.
If you do not use Sync, but instead you use Manually Manage, you will need to remove the songs from the iPod, then add back the new, revised, songs. Now would be the time to consider using Sync to manage your iPod, because if your iPod ever needs a Restore, or when you replace the iPod with a newer one, Sync will make it much easier to put your music back onto the old, or new iPod. (A Restore will remove everything from an iPod, and sometimes that has to be done.)
Message was edited, for clarity, by: the fiend