Although the related article sways, that the alarm should go off on the phone and the headphones, that does not work on my iPhone 12 either, latest iOS 16.0.3 installed.
If the AirPods are connected, the alarm will be heard over the headphones, if they are disconnected, the external speaker of the iPhone will play the alarm sound.
...
- If your alarm only vibrates, make sure that your alarm sound isn't set to None. Open the Clock app, tap the Alarm tab, then tap Edit. Tap the alarm, then tap Sound and choose a Sound.
- If you connect headphones to your iPhone, the alarm plays at a set volume through the built-in speakers on your iPhone as well as wired and wireless headphones.
...
copied from: How to set and change alarms on your iPhone - Apple Support