Can I replace an iPhone 6s battery myself

Legitimately...no. The iPhone is not intended to be serviced by the consumer.

Not with an actual OEM new battery. Those are not sold at retail. Only Apple or an Apple authorized service provider has access to OEM new batteries.

NO. And don't try. Apple doesn't sell parts anyway.