How do I change the battery in my Apple Pencil?

Your title says change, but your post says charge?

To charge: Take the cap off and plug it into the iPad‘s Lightening port. Or, use the small adapter in the box and attach that to the pencil, then plug in the lightening cable from your iphone or iPad power brick.

Use Apple Pencil with iPad Pro - Apple Support

The Pencil itself is not user serviceable at all.

Edited:

You can charge it be plugging it into your iPad.