If you click on the first song on the album, hold down the Shift key on your keyboard, then click the last song, you can "Get Info" on multiple items. You can then change the art for the whole album at once.
Even easier, you can also work in Grid View:
Then you just click the album Click get info. Then change the art on all items.