Bluetooth Speakers - Audio skips when I listen to audiobooks, but NOT when I listen to music?

I went ahead and "forgot" some devices in the hopes it would help the connection between my phone and my device (a heyday portable bluetooth speaker). But it doesn't help. I just want to listen to audiobooks (on the Libby app) while I clean but it skips every other word and makes it impossible to listen to - but when I play music off spotify, it's perfect? What should I do?