How much does it cost to upgrade Apple studio monitor to a vesa mount

At the time you buy it, nothing.

You have a choice between a non-height-adjustable stand (+$0), a VESA mount (+$0), or a height-adjustable stand (+$400). As far as I know, neither of the stands are VESA-compliant. If you order the display with a VESA mount, you do not get a stand (even a basic one) at all, so you've got to run out and get something to hold the display right away. If you order a stand, there's no VESA mount to use with a third-party VESA stand, arm, or wall mount later.

With the Pro Display XDR, it looks like you can buy a VESA Mount Adapter later, for $199 – but I see no indication of that adapter being compatible with the Studio Display.

A little searching found this:

Set up and use Apple Studio Display - Apple Support

"f you prefer a different option after purchase, your Studio Display’s stand or VESA mount adapter can be changed at an Apple Store, Apple Authorized Service Provider, or Independent Repair Provider, subject to a parts and service fee."

You'd need to ask Apple or an Apple Authorized Service Provider what that "parts and service fee" would be.