Should your Apple Pencil fail to charge correctly when connected to the Lightning port of your iPad, this may indicate a failing Pencil Battery.
The first generation Apple Pencil will charge at it optimum/fastest rate when connected directly to the Lightning port of your iPad. If connecting the Pencil to your Power Adapter (charger), using the charging adapter that was supplied with your Pencil, the charging rate will be much slower; in general, charging from an external charger is intended to provide a “tickle” maintenance charge when unused.
Charging time for the first generation Apple Pencil, from flat, is approximately 10-20 minutes when connected to iPad - but you should always try to keep your Pencil charged. Permitting the Pencil to remain in a low charge state risks irreversible damage to the tiny internal Li-ion battery - and as the battery cannot be singularly replaced, the Pencil itself must be replaced.
The tiny internal Li-ion battery is susceptible to permanent/irreversible damage through being left discharged for long periods. Even some “new” pencils have been known to exhibit signs of failure out-of-the-box if they are “old stock”.
To maintain the health of the tiny internal battery, it is essential that you keep it charged - whether used or not - so as to protect the battery from deep-discharge. Do not allow a pencil to remain in low-charge state for any period of time - as the internal battery will fail, rendering the Pencil useless.
If the Pencil Battery has failed, the only remedy is to replace the Pencil. If purchased directly from Apple, if the battery fails and is within its one-year warranty, Apple will generally replace the Pencil.
I hope this information proves to be helpful and resolves any queries that you may have.