Hi Jeremy,
Answers below in bold...
You mentioned you don't see the cloud icon next to these songs. Have these songs been downloaded directly to your computer? If so, it would be normal for the songs to appear without the cloud icon: About iCloud Music Library icons and status
These songs have been added to my library, but are not downloaded. In the past, the highlighted section would either show the cloud icon (in my library, not downloaded) or a plus sign (not in my library). It was an easy way to see which songs are already in my library. When they are downloaded, a checkmark appears in that space.
Do you see the same behavior with all albums/songs in your library?
The library & store tabs seem to be working correctly. I'm only noticing the visual issues (discolor/blur clicked items, missing graphics) on For You & Browse.
What happens when you try playing one of the songs?
No playback issues.
Also, I notice you mention you recently reinstalled Windows; are there any updates for your Windows software remaining, or is it fully updated?
I believe it's fully updated. It's been about a week & I've downloaded every system update that has popped up since then. I tried deleting & re-installing iTunes but I'm still having the same issue.