There's no reason you can't turn your TV off when playing back music, or use another input, but if the sound is going through HDMI you're likely to lose that.
There's no way to just turn the video display black, though I can understand why you might not want distracting graphics if listening to music through the TV.
An alternative if using HDMI if your TV has audio jack inputs might be to use both HDMI and audio out from the AppleTV to the TV, and then switch to another video input with no signal - the audio in via the jacks might still sound if the TV thinks you've connected analogue audio for that source but not a video source. It may or may not work. If you use component video and stereo audio jacks then just pulling the component leads leaving audio connected would achieve the same but is hardly practical.
A workaround I suggest is this:
Create several 'dummy' photos which are pure black (or any other desired colour), sync them and set them as the screensaver with a short delay - not perfect but would still allow visual control when you need it via the remote.
AC