This is easily the most annoying of a long list of bugs that are in Monterey.
I do experience lots of behaviours that may have the same origin – It seems as though the programs refresh randomly or maybe get indexed all the time. If I have the folder open for a while I can scroll back and it takes quite a while longer for it to jump to the highest selected level.
Another issue I have with contacts – I cannot enter a new contact as it keeps refreshing an my new contact window is gone before I can finish it.
In calendar when I add new calendars it will refresh and be untitled before I can name it. (Although I cannot reproduce that right now, but observed it a couple of times)
In mail (which I have on a second screen) I see the inbox being reloaded from time to time out of the blue (not refreshing to get new mails just everything is gone for a split second before it comes back up). It's annoying there, too, when you are reading in the depth of a conversation and suddenly you are kicked out again.
Somehow I feel this is all related as it all is a reload/rebuild kind of behaviour. And it really spoiled the excitement of the new MBP...