It depends on your level of paranoia.
When the drive is mounted i.e. after you have entered the password then it is theoretically less secure than when not mounted. That is theoretically a skilled hacker could either try accessing it over the network and yes I saw your comment saying that file sharing etc. are not turned on or if they gain physical access to your Mac they could try breaking in to it by some method of unlocking the screen saver or by plugging in a hypothetical special device to attack your computer.
In reality for most people these will not be of genuine concern. It should be noted that the internal drive which is presumably also encrypted also remains equally vulnerable and obviously you cannot unmount it without shutting down the computer entirely. So it is considered good practice to fully shutdown your computer before transporting it out of your office/home rather than leaving it 'sleeping'.