Most models of iPad require a Power Adapter rated at 12W or greater to reliably charge. Some older models of iPad were supplied with a 10W Power Adapter. 5W Power Adapters intended for older models of iPhone are inadequate to charge an iPad; if the iPad charges at all, charging will be very slow.
When iPads are operating at maximum screen brightness - and/or running CPU/GPU intensive tasks - lower wattage Power Adapters may not be able to provide adequate power to both charge the iPad battery and sustain the display and CPU load. This may account for your observed behaviour.
While you can use higher output Power Adapters, many iPads with a Lightning connector are limited to 12W (i.e., 2.4A @5V). Newer models - and those with a USB-C connector - support USB PD (Power Delivery) at up 20W.
Be aware that many computer USB ports cannot source sufficient power to charge an iPad. Unless explicitly designed for charging connected devices, computer USB ports are typically limited to 0.5-1A @5V (i.e., 2.5-5.0W).