macOS has a hard time releasing its hold on mounted volumes. Sometimes if you click unmount/eject a second time it will let go (best to wait at least 15-20 seconds, perhaps more).
You can also try using the command line to unmount all the volumes on that drive as it may tell you why it could not be unmounted or ejected.
Get the device identifier for the physical external drive from Disk Utility. You may need to click "View" and select "Show All Devices" before the physical drives appear on the left pane of Disk Utility. The device identifier will appear in the format "diskX" where "X" is a number....for example "disk4".
Open the Terminal app and use the following command template making sure to replace "diskX" with the proper device identifier:
diskutil unmountDisk diskX
In my example:
diskutil unmountDisk disk4
Keep in mind the device identifier can change every time you connect the drive, so you must always check for the current device identifier.