Caps Lock key has delay even when set as Control modifier.

For years I've set my Caps Lock key to be a control modifier in keyboard settings, but recently (after updating to 11.5.1) I've noticed that I am getting a delay from when I hit the Caps Lock key until it takes effect. I think it's a well-known effect for the caps lock key when acting as a caps lock key, but this behavior doesn't seem helpful when trying to use it as something else.

Anyone else had issues with this?