Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

What is in the PostgreSQL_Server_Services.log File and How do I Shrink it?

I know OS X Server uses PostgreSQL now instead of MySQL which is fine by me. I am trying to figure out what is in the PostgreSQL_Server_Services.log file in Library/Logs/PostgreSQL/ . It is about 188 GB right now and ideally instead of relocating it along with other services to another drive from the nice 256GB SSD that is in the system I would like to understand what is in the database and how to manage it's size better.


Any pointers for best practice on managing the database or what the heck is in it?


Thanks!

Posted on Sep 23, 2013 8:04 AM

Reply
Question marked as Best reply

Posted on Sep 23, 2013 9:39 AM

Any pointers for best practice on managing the database or what the heck is in it?


Well, for one, it's not a database - it's a log file.


The natural answer to your question would be: take a look!


Postgres might be generating lots of worthless log data, or the log rotation might be broken (so the file grows ad infinitum) or, just maybe, it's trying to tell you something.


Since it's a log file (albeit a large one), any text editor should read it. Better yet, use /Applications/Utilities/Console.app. Look at the messages and try to understand what it's telling you. Also look at the timestamps since that can provide insight, too.

9 replies
Question marked as Best reply

Sep 23, 2013 9:39 AM in response to keithfromvirginia beach

Any pointers for best practice on managing the database or what the heck is in it?


Well, for one, it's not a database - it's a log file.


The natural answer to your question would be: take a look!


Postgres might be generating lots of worthless log data, or the log rotation might be broken (so the file grows ad infinitum) or, just maybe, it's trying to tell you something.


Since it's a log file (albeit a large one), any text editor should read it. Better yet, use /Applications/Utilities/Console.app. Look at the messages and try to understand what it's telling you. Also look at the timestamps since that can provide insight, too.

Sep 23, 2013 10:20 AM in response to Camelot

Thanks for the response. I mistyped in the question as I understood it was a log file. The DB itself is only about 1-2GB. As for reviewing I am fine with modifying the permissions and examining the file in the console or other app but I am most interested in how to manage it safely. Can I just clear the log contents? If so what is the safe way to do so? I know with many DB's the log file is critical to it's function and if things happen to the log file it can render the DB unusable. In addition what is the best way to modify the rotation routine it uses and set the logging level? (I have pasted the .plist contents for Postgresql for Server Services below)


I have also downloaded and installed pgAdmin but have not gone through the steps of connecting it to the DB and log file (users setup etc.)


<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>ProgramArguments</key>

<array>

<string>-D</string>

<string>/Library/Server/PostgreSQL For Server Services/Data</string>

<string>-c</string>

<string>unix_socket_directory=/Library/Server/PostgreSQL For Server Services/Socket</string>

<string>-c</string>

<string>logging_collector=on</string>

<string>-c</string>

<string>log_connections=on</string>

<string>-c</string>

<string>log_lock_waits=on</string>

<string>-c</string>

<string>log_statement=ddl</string>

<string>-c</string>

<string>log_line_prefix=%t </string>

<string>-c</string>

<string>listen_addresses=</string>

<string>-c</string>

<string>log_directory=/Library/Logs/PostgreSQL</string>

<string>-c</string>

<string>log_filename=PostgreSQL_Server_Services.log</string>

<string>-c</string>

<string>unix_socket_group=_postgres</string>

<string>-c</string>

<string>unix_socket_permissions=0770</string>

</array>

</dict>

</plist>

Sep 23, 2013 10:32 AM in response to keithfromvirginia beach

Oh here are some of the errors it has in the log file:

2013-09-23 13:18:23 EDT ERROR: role "collab" already exists

2013-09-23 13:18:23 EDT STATEMENT: CREATE ROLE collab SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN;


2013-09-23 13:18:23 EDT ERROR: role "webauth" already exists

2013-09-23 13:18:23 EDT STATEMENT: CREATE ROLE webauth SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN;


2013-09-23 13:18:23 EDT ERROR: database "collab" already exists

2013-09-23 13:18:23 EDT STATEMENT: CREATE DATABASE collab;



I also saw this further back in the logs... many times in a row

LOG: could not open log file "/Library/Logs/PostgreSQL/PostgreSQL_Server_Services.log": Too many open files

Sep 23, 2013 10:58 AM in response to keithfromvirginia beach

Clearly some process is trying to (re)create existing database infrastructure. Are these recurring? or one-off?


Unfortunately there's not enough data there to clearly indicate who/what is trying to do this. There are numerous apps that might call for a 'collab' or 'webauth' (including some of Apple's own bundled apps).


I think you need to drill down a little deeper to see which processes are connecting to Postgres and trying to execute these statements.

Sep 23, 2013 10:59 AM in response to keithfromvirginia beach

OK a little more info... I found the postgresql.conf file and some of the settings that I would think should be in effect are not and they are commented out instead. Does anybody recommend uncommenting the log rotation etc from below?


# These are only used if logging_collector is on:

#log_directory = 'pg_log' # directory where log files are written,

# can be absolute or relative to PGDATA

#log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file name pattern,

# can include strftime() escapes

#log_file_mode = 0600 # creation mode for log files,

# begin with 0 to use octal notation

#log_truncate_on_rotation = off # If on, an existing log file with the

# same name as the new log file will be

# truncated rather than appended to.

# But such truncation only occurs on

# time-driven rotation, not on restarts

# or size-driven rotation. Default is

# off, meaning append to existing files

# in all cases.

#log_rotation_age = 1d # Automatic rotation of logfiles will

# happen after that time. 0 disables.

#log_rotation_size = 10MB # Automatic rotation of logfiles will

# happen after that much log output.

# 0 disables.

Sep 24, 2013 5:41 AM in response to keithfromvirginia beach

Well to add to this I have gone ahead and copied then deleted the old 188GB log. As soon as I moved it to the trash it created a new one. I rebooted and will monitor it's growth. I guess my only outstatnding issue is how to manage this in the future. Seems the OS should be doing some kind of truncation or rotation automatically as it does not offer direct maintenance out of the box to the Postgresql DB settings.


If anyone can tell me if modifying the conf file settings for the log would work (they are commented out now so I am not sure why it is logging anyways). My hunch is there is a way to modify the plist file to control things since as I noted the log settings in the conf file are commented out.


Thanks!

Oct 17, 2013 3:06 PM in response to keithfromvirginia beach

Did you ever figure out what the issue was?


I noticed some substantial CPU usage by the _postgres user the other day, but didn't think too much about it. Today I was doing a routine backup and noticed that a considerable amount of time was being spent on PostgreSQL_Server_Services.log. A quick check yeilded the following:


$ ls -lh /Library/Logs/PostgreSQL/PostgreSQL_Server_Services.log

-rw------- 1 _postgres _postgres 374G Oct 17 14:51 /Library/Logs/PostgreSQL/PostgreSQL_Server_Services.log



Something is clearly wrong, but I won't have time for a few more days to start digging.

Oct 21, 2013 8:37 AM in response to keithfromvirginia beach

For the future, there is a built-in command to 'rotate log files' automatically in OS X. When a log file reaches a specfied size it will rename it and you can specify how many log files to keep. This is how Apple handle their own log files but it is easy to extend to any additional log files you want.


The command is newsyslog, you can read details by typing man newsyslog in the Terminal. Example config files are in /etc/newsyslog.d/ you can duplicate one and edit it to cover your PostgresSQL_Server_Services.log


Apart from specifying the filename/path the other two main criteria will be the size and count (number to keep).

Jan 29, 2014 5:32 PM in response to John Lockwood

Even easier: edit /Library/Server/PostgreSQL For Server Services/Config/com.apple.postgres.plist, and add the log rotation parameters that you found in the postgres.conf file, like so:


<string>-c</string>
<string>log_rotation_size=100MB</string>
<string>-c</string>
<string>log_filename='PostgreSQL_Server_Services-%d%M.log'</string>


The latter log_filename setting replaces the existing one, so that you will keep at most 31 log files (one for each day of the month), and then they will go back and start overwriting previous ones, and if you rotate based upon size before you roll over to the next day of the month, postgres will select a new name for the file so that it doesn't get stomped. Please refer to http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html for more details.


Because Server doesn't offer control over this postgres instance, the cleanest way to put these changes into effect is to reboot.


Note that any directive that goes in postgres.conf can be added to the command line in a similar fashion.

What is in the PostgreSQL_Server_Services.log File and How do I Shrink it?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.