Don't do it!!!!
Really, do not try to un-encrypt the drive or you will likely lose all chances at recovering any data from it.
The encryption is very unlikely the reason you cannot access the data on the drive. The reason is either file system corruption or more likely a failing drive. If it is file system corruption, then see if Disk Utility First Aid can repair the hidden APFS Container. Within Disk Utility click "View" and select "Show All Devices" so that the hidden Container appears on the left pane of Disk Utility. Even if First Aid says everything is "Ok", click "Show Details" and scroll back through the report to see if any errors are listed. If there are errors, then run First Aid again until the errors are gone. If after several attempts the errors remain, then the file system cannot be repaired.
If the external drive has a hardware failure, then any attempts to access the data on the failing drive will most likely make the drive failure even worse even to the point where a professional data recovery service will be unable to recover any data.
The first thing I would do is check the health of the external drive by using DriveDx (free trial period). Post the complete DriveDx text here using the "Additional Text" icon which looks like a piece of paper on the forum editing toolbar so that we can examine it. You will need to install a special USB driver in order to attempt to access the health information of an external drive.
d Most consumer data recovery apps are not able to handle the errors produced by a failing drive....even macOS itself is unable to handle those errors.
The fact that your data recovery software is not working is already proof enough that you should stop using this external drive immediately & contact a professional data recovery service. Data recovery software doesn't care if the drive is encrypted or not. As long as the encrypted drive is unlocked, then data recovery software should have no problems working.
Data recovery software and file repair utilities are unable to work on a drive that is midway through encrypting or decrypting. And if the file system is corrupt, or especially if the drive is failing, then the encryption or decryption process may stall out and never complete.....meaning no special utilities will be able to help you fix the problem.
If the external drive is an SSD, then once the data has been deleted, then that data is gone for good due to how SSDs work unless you have a backup of that deleted data.
You may only have one chance at recovering your data so choose wisely......FYI, the wise choice is to contact a professional data recovery service.
