You are right in some way. I could list other entry points like "keyboard navigation", "selection between textfield", all topics that use the tab key on the keyboard when the mouse is not available. That topics is about holding down the tab key that moves continuously the focus from place to place until you stop on the desired button/textfield (move back with ****-tab).
I learned that some system preferences limit this movement to specific fields but it cannot be disabled completely, a nightmare. For me now, the 'x' disappears within a second, as soon as the focus move somewhere else. It was a sticky 'x' few days ago and then I could click on it easily.
Some settings have changed in between and I don't know which ones. I tried to investigate the contents of some preference files without success (complex SQL databases replace most the plist now).
That could happen to anyone like me using intensively all the macOS capabilities (well over the iOS capabilities). this is just a question of time for any one else. The unsupported and undocumented implementation of the iOS feature inside the macOS is not under control and more a test for Apple to carry its "eco-system" to the next commercial level.
Hope this clarify my investigation but without any valuable progress.
