Rusty Ross

Q: Server 3.2.1 (10.9.5) – Postgres won't start

After upgrading from Server 3.1.2 (Mavericks 10.9.3) to Server 3.2.1 (Mavericks 10.9.5), I cannot start Postgres:

 

# serveradmin start postgres

postgres:error = "CANNOT_START_SERVICE_TIMEOUT_ERR"

 

Here is what I see in system.log:

 

Sep 18 18:36:20 server com.apple.serverd[78]: ERROR: SMJobSubmit: The operation couldn’t be completed. (kSMErrorDomainLaunchd error 9 - The job dictionary specifies that it is disabled.)

Sep 18 18:36:20 server serveradmin[2249]: servermgr_postgres: waiting for postgres to respond

Sep 18 18:36:24 server serveradmin[2249]: servermgr_postgres: Timed out trying to start postgres.

Sep 18 18:36:24 server serveradmin[2249]: posting dist not

 

Not sure what to do from here. Postgres was running fine under 10.9.3.

 

 

Rusty

Posted on Sep 18, 2014 3:38 PM

Close

Q: Server 3.2.1 (10.9.5) – Postgres won't start

  • All replies
  • Helpful answers

first Previous Page 3 of 7 last Next
  • by Rusty Ross,

    Rusty Ross Rusty Ross Sep 20, 2014 7:20 PM in response to Rusty Ross
    Level 2 (175 points)
    Sep 20, 2014 7:20 PM in response to Rusty Ross

    This is untested (USE AT YOUR OWN RISK) but this shell script should probably do it. It would want to be run as root.

     

    #!/bin/bash
    
    
    # Disable postgres
    serveradmin stop postgres
    
    # Swap "unix_socket_directory" to "unix_socket_directories"
    perl -p -i -e "s/unix_socket_directory/unix_socket_directories/g" /Library/Server/PostgreSQL/Config/org.postgresql.postgres.plist
    
    # Move old database
    if [ ! -d "/Library/Server/PostgreSQL/DataOld" ]; then
      mv /Library/Server/PostgreSQL/Data /Library/Server/PostgreSQL/DataOld  
    fi
    mkdir -p /Library/Server/PostgreSQL/Data
    chown -R _postgres:_postgres /Library/Server/PostgreSQL/Data
    chmod 700 /Library/Server/PostgreSQL/Data
    
    # Upgrade database
    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/initdb /Library/Server/PostgreSQL/Data -E utf8  
    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_upgrade -b /Applications/Server.app/Contents/ServerRoot/usr/libexec/postgresql9.2 -B /Applications/Server.app/Contents/ServerRoot/usr/bin -d /Library/Server/PostgreSQL/DataOld -D /Library/Server/PostgreSQL/Data  
    
    # Start postgres
    serveradmin start postgres
    
    # Okay, we're done!
    
  • by John Caradimas,

    John Caradimas John Caradimas Sep 21, 2014 3:38 AM in response to Rusty Ross
    Level 1 (19 points)
    Servers Enterprise
    Sep 21, 2014 3:38 AM in response to Rusty Ross

    OK., last command i the script produces these results.

     

    sh-3.2# sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_upgrade -b /Applications/Server.app/Contents/ServerRoot/usr/libexec/postgresql9.2 -B /Applications/Server.app/Contents/ServerRoot/usr/bin -d /Library/Server/PostgreSQL/DataOld -D /Library/Server/PostgreSQL/Data

    Performing Consistency Checks

    -----------------------------

    Checking cluster versions                                   ok

    Checking database user is a superuser                       ok

    Checking for prepared transactions                          ok

    Checking for reg* system OID user data types                ok

    Checking for contrib/isn with bigint-passing mismatch       ok

    Creating dump of global objects                             ok

    Creating dump of database schemas

                                                                ok

     

    lc_collate cluster values do not match:  old "C", new "en_US.UTF-8"

    Failure, exiting

     

     

    Help pleaaaaaaseeeee!!

  • by John Caradimas,

    John Caradimas John Caradimas Sep 21, 2014 3:55 AM in response to John Caradimas
    Level 1 (19 points)
    Servers Enterprise
    Sep 21, 2014 3:55 AM in response to John Caradimas

    OK, after hitting the wall with the above, I run the following commands:

     

    sh-3.2# sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/initdb /Library/Server/PostgreSQL/Data -E utf8 --lc-collate=C --lc-ctype=C

    The files belonging to this database system will be owned by user "_postgres".

    This user must also own the server process.

     

    The database cluster will be initialized with locales

      COLLATE:  C

      CTYPE:    C

      MESSAGES: en_US.UTF-8

      MONETARY: en_US.UTF-8

      NUMERIC:  en_US.UTF-8

      TIME:     en_US.UTF-8

    The default text search configuration will be set to "english".

     

    Data page checksums are disabled.

     

    fixing permissions on existing directory /Library/Server/PostgreSQL/Data ... ok

    creating subdirectories ... ok

    selecting default max_connections ... 100

    selecting default shared_buffers ... 128MB

    creating configuration files ... ok

    creating template1 database in /Library/Server/PostgreSQL/Data/base/1 ... ok

    initializing pg_authid ... ok

    initializing dependencies ... ok

    creating system views ... ok

    loading system objects' descriptions ... ok

    creating collations ... ok

    creating conversions ... ok

    creating dictionaries ... ok

    setting privileges on built-in objects ... ok

    creating information schema ... ok

    loading PL/pgSQL server-side language ... ok

    vacuuming database template1 ... ok

    copying template1 to template0 ... ok

    copying template1 to postgres ... ok

    syncing data to disk ... ok

     

    WARNING: enabling "trust" authentication for local connections

    You can change this by editing pg_hba.conf or using the option -A, or

    --auth-local and --auth-host, the next time you run initdb.

     

    Success. Since you appear to be using the default database,

    the system-wide instance of postgres will be launched on-demand by

    the various services which use it.

     

    If necessary, you can now start the database server using:

     

        serveradmin start postgres

     

    See webappctl(8) to learn how to launch postgres on-demand for

    your web application.

    sh-3.2# sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_upgrade -b /Applications/Server.app/Contents/ServerRoot/usr/libexec/postgresql9.2 -B /Applications/Server.app/Contents/ServerRoot/usr/bin -d /Library/Server/PostgreSQL/DataOld -D /Library/Server/PostgreSQL/Data

    Performing Consistency Checks

    -----------------------------

    Checking cluster versions                                   ok

    Checking database user is a superuser                       ok

    Checking for prepared transactions                          ok

    Checking for reg* system OID user data types                ok

    Checking for contrib/isn with bigint-passing mismatch       ok

    Creating dump of global objects                             ok

    Creating dump of database schemas

                                                                ok

    Checking for presence of required libraries                 ok

    Checking database user is a superuser                       ok

    Checking for prepared transactions                          ok

     

    If pg_upgrade fails after this point, you must re-initdb the

    new cluster before continuing.

     

    Performing Upgrade

    ------------------

    Analyzing all rows in the new cluster                       ok

    Freezing all rows on the new cluster                        ok

    Deleting files from new pg_clog                             ok

    Copying old pg_clog to new server                           ok

    Setting next transaction ID for new cluster                 ok

    Setting oldest multixact ID on new cluster                  ok

    Resetting WAL archives                                      ok

    Setting frozenxid counters in new cluster                   ok

    Restoring global objects in the new cluster                 ok

    Adding support functions to new cluster                     ok

    Restoring database schemas in the new cluster

                                                                ok

    Removing support functions from new cluster                 ok

    Copying user relation files

                                                                ok

    Setting next OID for new cluster                            ok

    Sync data directory to disk                                 ok

    Creating script to analyze new cluster                      ok

    Creating script to delete old cluster                       ok

     

    Upgrade Complete

    ----------------

    Optimizer statistics are not transferred by pg_upgrade so,

    once you start the new server, consider running:

        analyze_new_cluster.sh

     

    Running this script will delete the old cluster's data files:

        delete_old_cluster.sh

    sh-3.2# serveradmin start postgres

    postgres:error = "CANNOT_START_SERVICE_TIMEOUT_ERR"

     

    Everything appears to be OK, but postgres doesn't start, giving that last line error message.

     

    Any help will be appreciated.

  • by Erik Hagemeijer,

    Erik Hagemeijer Erik Hagemeijer Sep 21, 2014 6:32 AM in response to John Caradimas
    Level 1 (0 points)
    Sep 21, 2014 6:32 AM in response to John Caradimas

    I have the same problem: I can start Postgresql from the Data folder manualy with

    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/postgres --apple-configuration /Library/Server/PostgreSQL/Config/org.postgresql.postgres.plist

     

    The serveradmin status is running but unresponsive, but I am perfectly able to connect to the databases.

  • by Rusty Ross,

    Rusty Ross Rusty Ross Sep 21, 2014 8:49 AM in response to Erik Hagemeijer
    Level 2 (175 points)
    Sep 21, 2014 8:49 AM in response to Erik Hagemeijer

    Maybe try unloading and reloading the LaunchDaemon?

     

    launchctl unload /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/org.postgresql.postgres.plist
    launchctl load -w /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/org.postgresql.postgres.plist
    
  • by Victor Pang,

    Victor Pang Victor Pang Sep 21, 2014 9:41 AM in response to Rusty Ross
    Level 1 (110 points)
    Sep 21, 2014 9:41 AM in response to Rusty Ross

    Rusty, your script can omit the following lines:

    1. mkdir -p /Library/Server/PostgreSQL/Data 
    2. chown -R _postgres:_postgres /Library/Server/PostgreSQL/Data 
    3. chmod 700 /Library/Server/PostgreSQL/Data

    as the initdb command running as the _postgres user will create the new directory with the correct ownership and permission.

     

    Also, there must be a way to check the locale LC_CTYPE of the old database, so that the initdb can create the new database structure with the old database LC_CTYPE (by the way "--no-locale" is the same as "--lc-ctype=C"). For me, I did not touch "/Library/Server/PostgreSQL" but created the same structure as Apple but named them as "/Library/Server/PostgreSQL2" and "/Library/Server/PostgreSQL3" including the PostgreSQL .plist and Launch Daemon .plist files to prevent any future clash.

  • by GSOY,

    GSOY GSOY Sep 21, 2014 12:13 PM in response to Erik Hagemeijer
    Level 1 (15 points)
    Sep 21, 2014 12:13 PM in response to Erik Hagemeijer

    Erik Hagemeijer wrote:

     

    I have the same problem: I can start Postgresql from the Data folder manualy with

    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/postgres --apple-configuration /Library/Server/PostgreSQL/Config/org.postgresql.postgres.plist

     

    The serveradmin status is running but unresponsive, but I am perfectly able to connect to the databases.

    Same here. I tried to reload the LaunchDaemons, but without success. Then I tried loading the Launch Daemon with the -w override option (while being root) – and it suddenly worked. Still trying to figure out why.

  • by Dfundy,

    Dfundy Dfundy Sep 21, 2014 1:57 PM in response to GSOY
    Level 1 (19 points)
    Sep 21, 2014 1:57 PM in response to GSOY

    Thank you Rusty Ross and Erik Hagemeijer for your valuable help.

     

    @Rusty Ross

    sudo -u _postgres ...  from root shell did not work for me. Assigning a shell and NFSHomeDirectory for user _postgres via dscl command enabled me to log in as _postgres and issue the database initialization and migration commands from there.

     

    Daniel

  • by Johan Ylinenjarvi,

    Johan Ylinenjarvi Johan Ylinenjarvi Sep 22, 2014 5:43 AM in response to Rusty Ross
    Level 1 (11 points)
    Mac OS X
    Sep 22, 2014 5:43 AM in response to Rusty Ross
    
    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_upgrade -b /Applications/Server.app/Contents/ServerRoot/usr/libexec/postgresql9.2 -B /Applications/Server.app/Contents/ServerRoot/usr/bin -d /Library/Server/PostgreSQL/DataOld -D /Library/Server/PostgreSQL/Data 
      
    
    

    Generates the following error:

    Performing Consistency Checks

    -----------------------------

    Checking cluster versions                                   ok

    Checking database user is a superuser                       ok

    Checking for prepared transactions                          ok

    Checking for reg* system OID user data types                ok

    Checking for contrib/isn with bigint-passing mismatch       ok

    Creating dump of global objects                             ok

    Creating dump of database schemas

                                                                ok

     

    lc_ctype cluster values do not match:  old "C", new "UTF-8"

    Failure, exiting



  • by Johan Ylinenjarvi,

    Johan Ylinenjarvi Johan Ylinenjarvi Sep 22, 2014 5:58 AM in response to Rusty Ross
    Level 1 (11 points)
    Mac OS X
    Sep 22, 2014 5:58 AM in response to Rusty Ross

    replacing line 19 with: sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/initdb /Library/Server/PostgreSQL/Data -E utf8 --lc-collate=C --lc-ctype=C

     

    did the trick, thanks alot!

  • by Rusty Ross,

    Rusty Ross Rusty Ross Sep 22, 2014 7:40 AM in response to Johan Ylinenjarvi
    Level 2 (175 points)
    Sep 22, 2014 7:40 AM in response to Johan Ylinenjarvi

    I am posting an improved version of a fix script (which, as always, should be run as root). And, as always, USE AT YOUR OWN RISK.

     

    #!/bin/bash
    
    # Disable postgres
    serveradmin stop postgres
    
    # Swap "unix_socket_directory" to "unix_socket_directories"
    perl -p -i -e "s/unix_socket_directory/unix_socket_directories/g" /Library/Server/PostgreSQL/Config/org.postgresql.postgres.plist
    
    # Move old database
    if [ ! -d "/Library/Server/PostgreSQL/DataOld" ]; then
      mv /Library/Server/PostgreSQL/Data /Library/Server/PostgreSQL/DataOld
    fi
    
    # Upgrade database
    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/initdb /Library/Server/PostgreSQL/Data -E utf8 --lc-collate=C --lc-ctype=C
    sudo -u _postgres /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_upgrade -b /Applications/Server.app/Contents/ServerRoot/usr/libexec/postgresql9.2 -B /Applications/Server.app/Contents/ServerRoot/usr/bin -d /Library/Server/PostgreSQL/DataOld -D /Library/Server/PostgreSQL/Data
    
    # Start postgres
    launchctl load -w /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/org.postgresql.postgres.plist
    serveradmin start postgres
    
    # Okay, we're done!
    
  • by rob Copeland,

    rob Copeland rob Copeland Sep 22, 2014 8:48 AM in response to Rusty Ross
    Level 1 (10 points)
    Servers Enterprise
    Sep 22, 2014 8:48 AM in response to Rusty Ross

    Thanks so much for posting this. I plan to try this later today. Do you have any idea how this might affected by any subsequent fixes from Apple?

  • by unclphester,

    unclphester unclphester Sep 22, 2014 9:57 AM in response to Rusty Ross
    Level 1 (0 points)
    Sep 22, 2014 9:57 AM in response to Rusty Ross

    Thank you!

    I did not run as a script, but all the individual steps worked.

    Roundcube is working again.

  • by René de Vries,

    René de Vries René de Vries Sep 23, 2014 4:01 PM in response to unclphester
    Level 1 (0 points)
    Sep 23, 2014 4:01 PM in response to unclphester

    I ran the script (after doing chown on the PostgreSQL directory) but I'm still getting the error messages:

     

    sh-3.2# serveradmin start postgres

    postgres:error = "CANNOT_START_SERVICE_TIMEOUT_ERR"

     

    NEW: Now got it to work, somehow the socket dir had the wrong permissions.

  • by Robb Allan,

    Robb Allan Robb Allan Sep 26, 2014 1:24 PM in response to Rusty Ross
    Level 2 (354 points)
    Servers Enterprise
    Sep 26, 2014 1:24 PM in response to Rusty Ross

    Nicely done, everyone, particularly Rusty Ross for diagnosing and putting a solution together. Works fine.

first Previous Page 3 of 7 last Next