I would recommend that you start by performing a "factory default" reset on the AirPort Express base station. Once reset, and by default, the base station will broadcast an unsecured wireless network with a network name (aka SSID) of something like: Apple Network NNNNNN. It will also default to be on the 10.0.1.n subnet. Unless your iMac is on another subnet the new base station should be discovered by your Mac under the AirPort icon on the OS X menu bar.
Select the new base station and the AirPort Utility should start up automatically. You should then be able to configure it as required.
I am having the same problem with Mavericks, repeatedly getting the "An unexpected error occurred. Please try later" or something similar. And I kept trying again and again and again and again with the same result.
So I booted up my iMac (Processor 2.7 GHz Intel Core i5) with my old Mountain Lion disk and, using the Airport Utility 6.2, I was able to set up the Airport Express on the first try. No unexpected errors.
From a troubleshooting standpoint, I have to posit that something is awry either with Mavericks itself or the new Airport Utility 6.3.2. regarding the setup of the Airport Express. The Airport Extreme doesn't seem to encounter the same problems, but then again I have made only a few minor changes. Nothing involving setup. (And I'm not going to)
If you can boot up under an older OSX, I'd suggest you try that first, and then make sure your Airport utility is in the 6.2 range, not 6.3.
I think this is strictly a software problem.