Hey again!
And for sure the source of the files are the ExFAT formatted drive(s)? I know the format is ExFAT, however can you please confirm what the partition scheme is? (Master Boot Record) (Guid), etc.? For each drive?
Open Finder > Applications > Utilities > Disk Utility, press View > Show All Devices, this way you'll see all the disks and volumes/partitions.
Can you also open the .Trash folder, press Command and i to get the info, expand all the sections to see the info? Same as on the external drive you're using?
I'm guessing deleting many files from the Mac Trash has no issues? Same as the HFS+ formatted drives as you mentions?
Can you try the same external drives, in other ports, with other cables? (I know you said multiple drives, but doesn't rule out a faulty cable or port).
Knowing if it's specific to certain file types may help as well, although I'm sure not everyone has those to delete at their disposal.
As it works using sudo, perhaps there is a disconnect between APFS formatted Ventura and ExFAT, I personally haven't seen this issue as well, so wondering what brand of external drive(s) you're using. And any hubs or adapters, etc. and more importantly, the model of your Mac......that's helpful information.
If the permissions are messed up, I'd still try the two oldest Mac troubleshooting tests, Safe Mode and a new Admin user, (If you haven't already):
Start up your Mac in safe mode - Apple SupportOfficial Apple Support › guide › mac-help › mac
Add a user or group on Mac - Apple SupportOfficial Apple Support › guide › mac-help › mac
This may reveal a lot.
Hope you figure it out, glad to assist if I'm able to.