M1 Air ambient sensor too sensitive?

Recently bought a brand new M1 Air and I've noticed that the ambient sensor is unreasonably sensitive when auto-brightness is turned on, to the point where a single hand put in front of it from a 20 cm distance makes the display noticeably dimmer. It also dims the display after I close the lid, which results in a slow brightness increase when I open it every single time, which doesn't seem to be intentional. Any suggestions? I understand that I can simply turn off auto-brightness, but that isn't the point here.