Should all app caches be emptied to speed up an old Mac?

No, that will actually slow down your computer as those cache files will have to be recreated. There is no speed benefit in deleting a cache file. The only way that would happen is when your storage is full and your computer is not able to efficiently manage Virtual Memory. If that is the case, you would have much more work to do in freeing up storage space than what is being used by cache files that will only be re-created.

The purpose of the cache file is to speed up your computer by not needing to access the network or load previous data for an app from scratch.

Forget about application caches for the reasons already explained.

You may gain some performance after booting your Mac into Safe Boot mode, then rebooting normally. That will clear certain System caches and rebuild the font and launch services database, among other things.

Secondly, boot your Mac into Recovery mode (Intel) and use Disk Utility First Aid on the main Macintosh HD drive entry. This will sift through the drive and perform repairs if any are needed and should end with a green status. Then reboot normally. A drive with repairable issues may also be a reason for a slow Mac. However, if you purchased that older iMac with an Apple Fusion drive, they are well known as performance degradation.

The opposite. Caches speed up a computer because they don't have to fetch data repeatedly. Their downside is they take up storage space. I see you are using an older system but generally the system knows how to keep a balance between not filling up a drive with cache files, and keeping enough to keep things moving.