Technically, neither option-0 or option-K are the degree symbol (option-o is technically 'masculine ordinal indicator' an option-k is 'ring above, as in å).
The correct character for the degree symbol is option-shift-8. However, neither 'keystroke "8" using {option down, shift down}', nor 'keystroke "*" using option down' has the desired result.
The only solution I've found is to use the key code command rather than the keystroke, and this seems to work:
tell application "TextEdit" to activate
delay 1
tell application "System Events"
tell process "TextEdit"
keystroke "8"
key code 28 using {option down, shift down}
end tell
end tell
I can't explain why keystroke "8" using {option down, shift down} doesn't work, though.