Q: Unlock with Apple Watch - Works Occasionally

I have unlock with Apple Watch on macOS Sierra. It works sometimes on my Mac but quite frequently it says 'Unlocking with Apple Watch' and then just goes back to requiring a password.

I always have my Mac in a fixed position, always close to WIFI and I'm sitting right next to it when this happens, so I don't understand why it is only working some of the time?

I have an iMac (Retina 5K, 27-inch, Late 2014)