How am I supposed to get refunds on stuff I bought with Apple Pay

Apple Pay does not sell anything. You can't actually buy anything from Apple Pay.

You can use Apple Pay to buy things from people and stores and as such the refund would be obtained by requesting them from the people or stores you bought them from directly. Take your receipt to the store or person and request a refund like you would normally. Apple Pay is merely a conduit for payment it is not in and of itself a merchant to request refunds from.

See here: link-> Get a refund for purchases made with credit or debit cards using Apple Pay - Apple Support

Are you referring to Apple Pay Cash?

If so, you ask the seller to give you your money back... it's just like using cash.