Ok, tried changing perms and got same result.
I id the ls on the directory and it had the same thing so I changed it as well. So it is updated as well with:
bash-3.2# ls -aFhlTe /Library/Server/Mail/data/scanner/
total 0
drwxr-xr-x 7 root wheel 238B May 16 21:50:52 2014 ./
drwxr-xr-x 13 root wheel 442B May 16 21:47:59 2014 ../
drwxr-x--- 13 _amavisd _amavisd 442B May 19 14:45:21 2014 amavis/
drwxr-xr-x 4 _clamav _clamav 136B May 19 14:45:09 2014 clamav/
drwxr-x--- 1985 _amavisd mail 66K May 19 16:14:10 2014 quarantine/
drwxr-xr-x 3 root wheel 102B Jun 8 15:22:15 2013 spamassassin/
drwxr-x---+ 2 _amavisd _amavisd 68B May 19 16:14:10 2014 virusmails/
0: group:admin allow list,add_file,search,add_subdirectory,delete_child,readattr,writeattr,readextat tr,writeextattr,readsecurity
bash-3.2# sudo chown -R _amavisd:_amavisd /Library/Server/Mail/data/scanner/quarantine
bash-3.2# ls -aFhlTe /Library/Server/Mail/data/scanner/
total 0
drwxr-xr-x 7 root wheel 238B May 16 21:50:52 2014 ./
drwxr-xr-x 13 root wheel 442B May 16 21:47:59 2014 ../
drwxr-x--- 13 _amavisd _amavisd 442B May 19 14:45:21 2014 amavis/
drwxr-xr-x 4 _clamav _clamav 136B May 19 14:45:09 2014 clamav/
drwxr-x--- 1985 _amavisd _amavisd 66K May 19 16:14:10 2014 quarantine/
drwxr-xr-x 3 root wheel 102B Jun 8 15:22:15 2013 spamassassin/
drwxr-x---+ 2 _amavisd _amavisd 68B May 19 16:14:10 2014 virusmails/
0: group:admin allow list,add_file,search,add_subdirectory,delete_child,readattr,writeattr,readextat tr,writeextattr,readsecurity
bash-3.2# sudo /Applications/Server.app/Contents/ServerRoot/usr/bin/amavisd-release /Library/Server/Mail/data/scanner/quarantine/banned-fX3ORr9nexcG
Can't connect to UNIX socket /var/amavis/amavisd.sock: No such file or directory at /Applications/Server.app/Contents/ServerRoot/usr/bin/amavisd-release line 260.