Do the Apps I no longer use but are in the cloud take up storage space?

Apps themselves do not take up your iCloud storage space.

Apps can store data in iCloud if you allow them to do so.

You can go to Settings > [your name] > iCloud and see which apps are using iCloud.

You can also tap Manage Storage to see how much space apps are using

If you have OFFLOAD UNUSED APPS activated then those apps are offloaded and remain for you to download again from the App Store when necessary. The actual apps are not in iCloud but are in the App Store. Your app data remains on your iPhone and in iCloud (if you use iCloud).

Axel F.