There are no answers in this reply. Just questions and suggestions. For better or for worse, performance-related issues can have many triggers, and usually require viewing and potentially actively collecting data from various soures, and can require digging around in the log files.
Are these AFP storage accesses local (as would be typical) or are these remote accesses across an ISP network link? Local access and local hardware is obviously far more under your control, where remote access can involve links that aren't/
In general, resolving this involves checking the local network for wiring errors and swap any cables that look questionable, checking for (for instance) links running more slowly than expected, and check the server for performance issues, and run hardware diagnostics on the server. In more details...
These days, the wired network should be gigabit gear all the way through, and if there's WiFi involved anywhere in these links remove that and run wires at least for performance testing — WiFi can be a performance disaster, as interference can arise from many sources including cordless phones and new WiFi networks set up by a neighbor.
Checking network links is somewhat easier with managed switches as you can access the switch via the management path and view its settings and what's been negotiated with the clients. The unmanaged switches usually have some sort of speed indicator for the connected ports. I'd expect to see gigabit Ethernet full-duplex connections with most any recent hardware.
View the Console.app logs for repeating errors and crashes and related; for any untoward behavior. I'd check the general (all messages) log, as well as the AFP-related file services log.
Confirm your local server DNS services with the following (harmless, diagnostic) Terminal.app (Applications > Utilities) command issued on the server:
sudo changeip -checkhostname
This command will tell you about most of the common DNS or network configuration errors, or it'll tell you that no changes are required to the server network configuration.
Verify your disks aren't being operated at near capacity as near-full disks really slow things down.
Use the Activity Monitor tool (Applications > Utilities) to see if your systems are just busy with CPU or I/O.
