How to unplug Toslink optical cable?

I am moving my old, first generation Apple TV, to a new TV we just bought. I cannot figure out how to unplug the Toslink optical out cable -- it seems "stuck." I don't want to break it. Is there some kind of release or trick to getting it out?