_amavisd/perl5.18 eating up disk space

I might have found a solution, now time will tell if it is a final solution:

I stopped the Mail service and deleted all files from:

sudo rm /Library/Server/Mail/Data/scanner/amavis/.spamassassin/*

Then I started mail again and ran the following command:

sudo sa-learn --force-expire

read the explanation with:

man sa-learn

after letting it run for an hour:

-rw------- 1 _amavisd _amavisd 171966464 Oct 28 12:32 bayes_seen

-rw------- 1 root _amavisd 88080384 Oct 28 12:32 bayes_toks

Which seems hopefull, as --force-expire also includes a --sync (from the manual) this behaviour might be caused by a missing step in the update process of Server.App to Version 5.4 (17S1207).

Again I am not sure yet if this is the whole story, the feeling on this side is that if it is not the solution, it is close!

I had this problem, it got so bad that it ended up causing the server to corrupt the OpenDirectory database!

The way I solved it was.

1. Go to Server and uncheck spam filtering, (no need to stop mail).

2. Delete spam databases sudo rm /Library/Server/Mail/Data/scanner/amavis/.spamassassin/*

3. sudo sa-learn --force-expire

4. Set permissions to allow group readwrite access for _amavisd to /Library/Server/Mail/Data/scanner and all files below.

5 Turn spam filtering back on in server.

I'm not sure if all the steps are necessary but this is why I did each step.

Turning spam filtering off and back on again means that the server app has to write to the config files, it is just possible that disabling and re-enabling causes a setting missing from the file from the previous version to be written to the file when re-enabling.

Deleting spam databases is obviously in case the database has become corrupted in some way or the format has changed between versions and the old format is not being read properly.

The default permissions were to allow only amavisd to read and write files. I'm not sure if there are any other system processes that perhaps need to access the files such as a rotation script, so allowing group access ensures any other process that is a member of the _amavisd group can access the files.

It has been working for me now without error for 3 days.

My next move was to back-up and restore:

First become _amavisd user:

sudo bash

su _amavisd

Backup:

sa-learn -D --backup > bayes.bck

Restore:

sa-learn -D --restore bayes.bck

I got lots of messages:

Oct 28 15:39:05.399 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.412 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.424 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.437 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.452 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.540 [37985] dbg: bayes: unknown seen flag (5fa9e4a2d60d608e1aea94f1171448b5a4830880@sa_generated_) for line: s 5fa9e4a2d60d608e1aea94f1171448b5a4830880@sa_generated_, skipping

Oct 28 15:39:05.617 [37985] dbg: bayes: unknown seen flag (72f221a511e76354ad4c0b94d32a88312634bb4e@sa_generated_) for line: s 72f221a511e76354ad4c0b94d32a88312634bb4e@sa_generated_, skipping

Oct 28 15:39:05.694 [37985] dbg: bayes: unknown seen flag (20b1dc46e7e87fd013eac87e8813bdfab20abb32@sa_generated_) for line: s 20b1dc46e7e87fd013eac87e8813bdfab20abb32@sa_generated_, skipping

Oct 28 15:39:05.771 [37985] dbg: bayes: unknown seen flag (e0081db86fbe49cb18fc790a22dd7c4f21cf76d0@sa_generated_) for line: s e0081db86fbe49cb18fc790a22dd7c4f21cf76d0@sa_generated_, skipping

lets see if this helps..

My next move was to back-up and restore:

First become _amavisd user:

sudo bash

su _amavisd

Backup:

sa-learn -D --backup > bayes.bck

Restore:

sa-learn -D --restore bayes.bck

I got lots of messages:

Oct 28 15:39:05.399 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.412 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.424 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.437 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.452 [37985] dbg: bayes: token has zero spam and ham count, skipping

Oct 28 15:39:05.540 [37985] dbg: bayes: unknown seen flag (5fa9e4a2d60d608e1aea94f1171448b5a4830880@sa_generated_) for line: s 5fa9e4a2d60d608e1aea94f1171448b5a4830880@sa_generated_, skipping

Oct 28 15:39:05.617 [37985] dbg: bayes: unknown seen flag (72f221a511e76354ad4c0b94d32a88312634bb4e@sa_generated_) for line: s 72f221a511e76354ad4c0b94d32a88312634bb4e@sa_generated_, skipping

Oct 28 15:39:05.694 [37985] dbg: bayes: unknown seen flag (20b1dc46e7e87fd013eac87e8813bdfab20abb32@sa_generated_) for line: s 20b1dc46e7e87fd013eac87e8813bdfab20abb32@sa_generated_, skipping

Oct 28 15:39:05.771 [37985] dbg: bayes: unknown seen flag (e0081db86fbe49cb18fc790a22dd7c4f21cf76d0@sa_generated_) for line: s e0081db86fbe49cb18fc790a22dd7c4f21cf76d0@sa_generated_, skipping

lets see if this helps..

Here is a final solution https://www.pingle.org/2011/02/10/rapidly-growing-bayes_toks

You have to modify the permissions of the .spamassassin to allow sudo to read and write.

Here is a final solution https://www.pingle.org/2011/02/10/rapidly-growing-bayes_toks

You have to modify the permissions of the .spamassassin to allow sudo to read and write.

I left it for a while and can confirm the above commands does the trick, no need for setting permissions if you are running these commands as the _amavisd user. (I mean the backup and restore commands in my previous post.

the)

Thanks for posting this... i had the same problem. 40 mins to fill a drive! Wonder what causes it to happen.

Sorry after a day of testing it goes back to be inflated... Ill have to crawl through the code to see what has changed between the 2 releases.

Confirmed NO solution, @ 15:31 beyes_toks is again at 2,5GB !!

2.5G is in my view very small - but I guess it depends on how long it takes to get to that size.

I have this problem too and have investigated this a bit.

You can do the following:

open Activity Monitor.app and go to the disk tab. Sort on the "bytes written" so that the large numbers are on top by blocking on the column header. there will be 1 (or 2 in my case) perl5.18 proccesses running as the _amavis user column. You might have to resize columns or scroll right a bit. Leave Activity monitor open and check the "Bytes Written" column from time to time. The sizes shown there seem to be the "lost" disk space in that time.

I have no clue in why amavis is growing my bayes_toks with ~ 60GB a day, but is really annoying to stop mail, delete the bases_toks file an start the Mail service again.

In the Process I have cleared all data/apps possible in a continuous struggle with a Disk (SSD) Filling op to 100% ever so quick as I could make space for a functioning computer. I should have probably used the safe mode to search for big files. I must admit that the first problem was a lot of large "expire" files I deleted.

I will post there.

For me this started after upgrading a server which started (2011 mini, 16GB RAM and 2,5" SSD) with "Lion + Server" and has been upgraded ever since without any reinstall, I have had my fair deal of "upgrade issues", never ever caused any by the changes i made in a lot of config files, which migrated perfectly over all those versions. Apple does a great job with Server.App but they always seem to hit me in the face with one annoying issue as a special gift on every upgrade 😟. To High Sierra and lates Server.App which comes with it.

I am capable of researching this, although I would need to dig into why amavis is acting up, I have other demanding jobs at the moment. Just for now I keep deleting bayes_toks. A search for "bayes_toks huge" shows that this might be a amavis originated bug. Maybe dig into that will be my next move.

One last thing is I have now 3 Perl5.18 by user _amavisd processes.