tap on trackpad works inconsistently (m1)

I am having this frustrating issue with the trackpad. I use tapping for the trackpad. Unfortunately it doesn't work consistently at all. I'm not sure if this has been happening since Big Sur, it's been a frustrating experience and it took me a while to realise there was something wrong with the Trackpad, and not with any particular application (initially I thought it was related to Firefox only).

The problem is that on regular intervals the tap is simply ignored. I have to press a second or a third time for it to work.

In Firefox, for example, there are moments where I need to press exactly 3 times for a new tab to open. Then it starts working normally.

In other contexts it is rather random, but often enough that it's becoming really hard to use my Trackpad properly. I haven't seen any issues when pressing ("normally") on the Trackpad, but I also haven't used it often enough.

A clear context where it consistently works badly is System Preferences - Trackpad.

When I check/uncheck an option such as 'Look up & data detectors', some taps are ignored. If I tap once again or twice, it works. If I press all the way down (not using force touch), it works correctly.

I'm using a Macbook Pro m1 (2020) on Mac OS Monterey, but this behaviour manifested itself in Big Sur too, so I would rule out that upgrading is the issue.

I've tested the exactly same thing (Trackpad in System Preferences) on a Macbook Pro (intel i7, I think also 2020; the last before m1) with Big Sur and it works exactly as expected.

Any ideas where the issue might be?

I've come across a somewhat similar topic on dev.apple (can't remember the link), but it was quite different. People had issues with big sur, somehow regardless of the laptop model, and they couldn't use it at all - it had nothing to do with tapping.