I also have this issue with ATV 4 and Harmony 700 universal remote. The latter was previously programmed to work with previous gen ATV.
Upon attempting to fast forward and rewind in a podcast using the Harmony, the playback speed got stuck at a slower speed. This was the normal operation on the prev gen ATV and seems to have caused this issue with the ATV4.
Like others here, no amount of ff or rew button presses would fix the issue. A simple restart didn't work. Force closing the app didn't work.
After seeing this thread I tried the Reset All Settings option and it worked.
Now I have to download and set up all the non-Apple apps all over again. Great...:(
This was a pretty awful user experience! And whe can we see speed control properly implemente, Apple? This is a very useful feature.