watching Apple TV on my non-Apple computer, how do I turn on subtitles?

Is that closed captioning or subtitles?

Closed captioning is in the same menu as subtitles. Closed captioning will have ‘CC’ after the language name there. Subtitles will just transcribe spoken dialog, while closed captioning will additionally also mention other sounds and sometimes music.

• For the web access through tv.apple.com, while content is playing or paused, find the Subtitles menu in the lower-right corner of the player window, when the mouse pointer is hovering over that window. Click that icon for a pop-up listing of options.

and how do I turn it on permanently?

The web access through tv.apple.com will remember (by cookies) the last used setting for subtitles/closed captioning.