A) Power. Driving the speaker takes much more power then using low energy Bluetooth. A typical audio conversation does not last for hours, but playing Music can and does go on-and-on-and-on...
B) Sound quality. Maybe not important to you, but there are reviewers that would harangue Apple for the lousy music sounds coming from the Apple Watch speakers. As long as 'A' is still important, there is no reason for Apple to get bad press because it is a tiny speaker and would generate low quality music.
When they install a fusion reactor in the Apple Watch, and can have all the power they need, then I guess they would have to re-evaluate whether to allow the speaker to play music. Until then, use a Bluetooth device for music listening.