It could be the Magic Mouse or the Mighty Mouse as pictured here:
In either case it sounds like the latch isn't connecting to hold the cover on. Looking at the bottom of the mouse with it held vertically, the top of the cover slides into a slot to hold it stationary while the bottom of the cover you press (hard) to cause the tab to come through the cover and click, locking it into place. If either of the tabs is broken or the brand of battery you use is to "fat" then the cover won't stay on.
It's the Magic Mouse. It's just like you say; the latch doesn't want to catch. I h ave to push hard and it usually catches but not always. I didn't realize different brands of batteries might be different diameters but it makes sense. Quality control rarely goes beyond 1 decimal place. Interestingly, this usually happens after a battery change. But then gets better and resolves without changing the batteries. I suspect sooner or later I close it "just right" and it's fine until next battery change. Irritating.
Went for first sail of season today. Cold and 20k wind. What could go wrong did go wrong; typical first sail. Enjoyed!