You are most welcome, Jure! What ios are you currently using? Make sure you update to the latest one supported by your phone., preferably ios 14.5.
Also try this;
Turn on “Hey Siri” for the device you connect your AirPods to. Return your AirPods to their case and hold the setup button to reset them. Reset All Settings on your device, turn on “Hey Siri,” and pair your AirPods again.
also, try to go to the phone settings and disable hey siri from there, then restart your phone. If it works then great, if not you open bluetooth in the settings and go in the airpods and select right ear to siri. The double tap option, if that works it means your airpods are ok, which you said works, i think there maybe some software or app causing this prob. Let me know if this works. If not you will have to reset your phone and set it up again, but hopefully after the unpairing and reset of your airpods, it should be ok.
Just try to pair and forget the devices a few times, also signing out of i cloud on your device and restart it, then sign back in and i am sure it should work. Good luck!
Regards