How can I stop an alarm on a locked iPhone?

A long shot, but if Siri was turned on and set to work when the iPhone is locked, try saying "Hey Siri, cancel the alarm". That might at least shut off that alarm.

As to forcing the iPhone to shut down completely, I'm not aware of any way to do that when the screen isn't working; the designated procedure does indeed just restart it. You'll probably need to take the phone to an Apple Store or authorized Apple service center for assistance.

Regards.

Hello,

1. Press and quickly release the volume up button.
2. Press and quickly release the volume down button.
3. Press and hold the side button until you see the Apple logo (could be as long as 30 seconds) 

The timing is a little critical, you may have to try it several times. See this for more detail:

If your iPhone, iPad, or iPod touch won‘t turn on or is frozen

Good luck.

Put the phone on silent with the side toggle that should at least silence it, you’ll still see the screen showing an alarm but at least you’ll get some peace from the noise of it.

Alternatively try turning the phone completely off, give it 5 mins and turn it back on.

In theory It should “reset” the alarm and trick the phone into thinking it’s not needed until the time set the next day… hopefully.