Q: Do I need to buy storage capacity in iCloud to cover my music, or does the $24.99 cover the storage?

Any music you purchase can be downloaded again. Any other music can be stored on your computer in iTunes. So not sure why you really need iCloud storage but it is another option for backup.

iCloud storage ony keeps the backups of your devices. So you need whatever amount of data is contained within your devices. Usually free 5Gb of storage you get is enough for most uses.

Music, and other Media from iTunes is not stored in iCloud. Once you purchase an item you can just download it again from the iTunes servers without paying, but it does not occupy iCloud space.

Do you mean iTunes Match?

If yes, it include the storage for the music.

iCloud: iTunes Match

iTunes Store: Subscribing to iTunes Match