Found workaround, though lengthy, no thanks to Apple.
So I have this same problem and I guess I should be careful what I say since my first post, Apple did not post. Lost ALL album artwork in the album view in Music. When I play the song, I can see the artwork on the top bar that shows the song playing. However nothing in album view or when going into the Info for either the track or the album.
SO this all started for me again when I "upgraded" to Big Sur so I could sync my new iPad, when my phone synced just fine. I run my Music off an external drive because the amount is so large (150+GB). when I started up Music (2 months after my upgrade, to add a new album), the pathway and last library was completely missing! So I had to choose one from 2019. Yeah, thats how many libraries were wiped out by Big Sur. (this is by using the process to hold the "option" key when opening Music. Initially I just opened Music as before since establishing what library to pull from.)
Tried the "get album artwork" and it works for only about half the albums if that. This is seriously ridiculous and not customer friendly (unless you want to scrap all your music and just 'subscribe' to apple music and be beholden to whatever they want to make available. might as well just listed to corporate radio...but I digress). So many of the albums are main stream albums that are common (ABBA - Gold) and yet no artwork avail. So its hit and miss.
Mind you, I have over 150 gigs of music. Many of the albums are local music, not sold on iTunes or on Apple music.
So I had to do the following. Mind you, there was no option to remove the album from the library without sending the files on my external drive to the trash. So, I remove the album from my library. Then go into the Trash and pull them back over into the external drive, re-creating that folder to move the files into. Then, double click each track to re-add into Music. There is the artwork again.
-- Now I tried a few other options and when I delete from Library, there is the option to Keep Files; CHOOSE THAT! (hey, I'm doing this by trial and discovery :) ). Then go into that file folder for that track (wherever that is) and double click; it will add it back to Music and boom, there is the artwork for the Album view. HOWEVER, if you want the album art to show up in Info for each track, you have to do the removal and double click process for each track.
I am a semi-serious audiophile with all genres of music so I am going through this process for all my music. Will likely take me days, if not weeks. Will be searching for a different and more secure music organizing app, but seems Apple has a bit of monopoly on this.
Apple, its seriously time to start putting more effort into protecting the Music app from your upgrades. This is NOT the first issue I and others have had with Music/iTunes during upgrades. Not everyone wants to subscribe to Apple Music, as cool as it is, and have their entire music beholden to what you offer.