is it okay to use power supply even if the macbook air is fully charged

Yes. The device will control the amount of power as needed

and it cannot 'automatically overcharge' from power adapter.

Good luck & happy trails!🌻🐝

another YES!

since 2010, my MacBook air always was connected as much as possible, which ran 4 hours and had 24% left yesterday