Local LDAP node authentication credentials could not be found

Question

Level 1

22 points

Local LDAP node authentication credentials could not be found

Hi all,

I spent the last few weeks combing through logs and resolving several pernicious errors on my 10.8.5 server. For the last week my server has been running error-free, user experience has been fabulous, zero system hangs, beautiful clean logs, etc. Today, out of the blue, one of my assistants said she could no longer add users via server.app, and I find the following in the system log:

Aug 14 22:44:11 myserver.fqdn.net servermgrd[8729]: -[AccountsRequestHandler(AccountsOpenDirectoryHelpers) authLocalLDAP]: Local LDAP node authentication credentials could not be found

Typically, the sever is bound using authentication to 127.0.0.1 and there is an appropriate application password item in the system keychain that references the correct machine and node.

Something new I noticed today, though, when I view the LDAPv3 service in Directory the Bind appears to be broken (not authenticated and I don't see the DN or Password listed in the Security Pane (whearas the way I set it up, it was authenticated as Diradmin).

If I re-bind and authenticate, then go to the Directory Editor tab of Directory Utility, the server can no longer connect to the node /LDAPv3/127.0.0.1/ and OD breaks.

For now, I've opted to leave the server unbound. OD appears to be operating fine, but we've had to re-bind all of our clients to get them to recognize the server again (using the exact same credentials), even though all of their machine records appear intact. Users are now able to log in again, but I'm still getting the error above whenever I start Server.app, and I cannot add or delete new users via Server.app (I can, however, add them through Workgroup Manager).

I am very, very confused. Clarity anyone?

Thanks!

-Paul

MAC MINI SERVER (LATE 2012), OS X Server, 10.8.5

Posted on Aug 14, 2014 11:05 PM

Reply

Answer 1

Picoscope Author

Level 1

22 points

Aug 14, 2014 11:48 PM in response to Picoscope

I also note that we have no keychain entry for com.apple.opendirectory, and wondering if we should? If so, how shall I create it?

Reply

Answer 2

Aug 15, 2014 9:54 AM in response to Picoscope

When you create your initial DirAdmin (or equivalent) the password is Not stored with the others. In addition, its Workgroup Manager entry says its Home Directory is in:

/private/var/root

Reply

Answer 3

Picoscope Author

Level 1

22 points

Aug 15, 2014 10:55 AM in response to Grant Bennet-Alder

Thanks, Grant. the diradmin password appears to be working fine (I can authenticate to /LDAP/127.0.0.1 using the diradmin credentials - UNLESS I rebind the server using those same credentials, then I can't connect to the node.). In my WGM the home directory for diradmin is listed as /var/empty/ which does exist.

NB: This morning I used the keychain access utility to access the system keychain and created an application password entry for com.apple.opendirectory, using myservercomputername$ and the diradmin password. Still no dice.

I've also rebuilt the kerberos db using:

sudo touch /var/db/openldap/migration/.rekerberize

sudo killall PasswordService

And FYI, changeip -checkhostname returns correct results (correct hostname and IP address - nothing to change)

And nslookup and host queries from client machines all give correct domain and host records for my server. Reverse lookup also resolves correctly.

Thoughts?

Reply

Answer 4

Linc Davis

Level 10

209,547 points

Aug 15, 2014 1:04 PM in response to Picoscope

Search the System keychain on the server for application passwords with the name "/LDAPv3/127.0.0.1" and delete any you find. Sign out of the Server app and sign back in using the FQDN of the server, not "localhost" or "127.0.0.1".

Credit for this observation to ASC member Peter Jurg2. See also this discussion.

Reply

Answer 5

Picoscope Author

Level 1

22 points

Aug 15, 2014 4:04 PM in response to Linc Davis

Sorry, Linc: No Joy. Followed your steps precisely, still no access to adding/removing users, same LDAP error.

-Paul

Reply

Answer 6

Linc Davis

Level 10

209,547 points

Aug 15, 2014 4:09 PM in response to Picoscope

Please describe how you signed in to the Server app. Anonymize any private information.

Reply

Answer 7

Picoscope Author

Level 1

22 points

Aug 15, 2014 5:05 PM in response to Linc Davis

Hi, Linc,

-Screen shared to my Mac Mini Server using ARD and logged in as root.

-On the server, clicked on Server icon in the dock.

-The server app opened with my default server connection (local server), which I closed.

-From the Manage icon, chose "Connect to Server"

-From the available list, chose "Other Mac"

-Entered myserver.fqdn.net for the server and used the diradmin username and password for authentication

Also just tried connecting from a client machine using the server app loaded on that machine. Connected to my server by clicking on it in the list (it appeared with the proper FQDN). Have tried using diradmin credentials and other admin user credentials. Same result. Tried again choosing other and explicitly entered the FQDN plus diradmin credentials. Same result.

Reply

Answer 8

Linc Davis

Level 10

209,547 points

Aug 15, 2014 6:16 PM in response to Picoscope

You should never log in as root. When you sign in to the server from an admin account, are any errors logged?

Reply

Answer 9

Picoscope Author

Level 1

22 points

Aug 15, 2014 7:24 PM in response to Linc Davis

Really? I wonder why on earth not? We've done this for years - actually ever since I first set up the server (10.2 or .3... can't remember now) on an imac all those years ago. Never has caused any issues that I'm aware of and sure saves us a lot of tedious re-authenticating for every darned thing we want to accomplish.

Anyway, I logged out root, logged in as an admin user and repeated the connection to the server via server app as described above (explicitly entered the FQDN) and this time used the same admin users credentials as I used to log into the server. Same result. Same LDAP error. Still can't add or remove users in Server app.

I hope you're finding this interesting...

paul

Reply

Answer 10

Linc Davis

Level 10

209,547 points

Aug 15, 2014 7:28 PM in response to Picoscope

Logs?

Reply

Answer 11

Picoscope Author

Level 1

22 points

Aug 15, 2014 7:39 PM in response to Linc Davis

I'll try to be brief... here's a representative sample from the system log.

Aug 15 19:32:12 fqdn.myserver.net com.apple.SecurityServer[23]: in od_record_attribute_create_cfstring(): returned 2 attributes for dsAttrTypeStandard:AuthenticationAuthority

Aug 15 19:32:12 fqdn.myserver.net com.apple.SecurityServer[23]: checkpw() succeeded, creating credential for user diradmin

Aug 15 19:32:12 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.services.directory.configure' by client '/usr/libexec/opendirectoryd' [28] for authorization created by '/usr/libexec/opendirectoryd' [28] (12,0)

Aug 15 19:32:14 fqdn.myserver.net servermgrd[11988]: servermgr_accounts: Couldn't find keychain entry for local computer record

Aug 15 19:32:14 fqdn.myserver.net servermgrd[11988]: -[AccountsRequestHandler(AccountsOpenDirectoryHelpers) authLocalLDAP]: Local LDAP node authentication credentials could not be found

Aug 15 19:32:14 fqdn.myserver.net servermgrd[11988]: servermgr_accounts: Couldn't find keychain entry for local computer record

Aug 15 19:32:14 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Aug 15 19:32:14 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [74] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (100000,0)

Aug 15 19:32:14 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Aug 15 19:32:14 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [74] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (100000,0)

Aug 15 19:32:15 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Aug 15 19:32:15 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [74] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (100000,0)

Aug 15 19:32:15 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Aug 15 19:32:15 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [74] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (100000,0)

Aug 15 19:32:15 fqdn.myserver.net servermgrd[11988]: --Module servermgr_devicemgr's response has retain count of 3.

Aug 15 19:32:15 fqdn.myserver.net servermgrd[11988]: --request was {

command = getState;

}

Aug 15 19:32:15 fqdn.myserver.net servermgrd[11988]: --response was {

state = STOPPED;

}

Aug 15 19:32:16 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Aug 15 19:32:16 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [74] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (100000,0)

Aug 15 19:32:16 fqdn.myserver.net com.apple.SecurityServer[23]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] for authorization created by '/Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/Serve rManagerDaemon.bundle' [11988] (2,0)

Reply

Answer 12

Picoscope Author

Level 1

22 points

Aug 15, 2014 7:42 PM in response to Picoscope

Note - I may need to go offline for an hour or so while I put my kids to bed - but back soon.

p

Reply

Answer 13

Picoscope Author

Level 1

22 points

Aug 15, 2014 9:37 PM in response to Picoscope

I'm back. Hey- -just glancing at the logs I just posted - I don't recall seeing this one before you and I started working:

Aug 15 19:32:14 fqdn.myserver.net servermgrd[11988]: servermgr_accounts: Couldn't find keychain entry for local computer record

Not sure if this was caused by removal of the /LDAPv3/127.0.0.1 keychain entry, or if I just missed it before, but I'm going to go verify that machine record.

And that makes me wonder if this has something to do with why I had to rebind all my clients... if their machine records got somehow pooched as well.... - though I was able to see all the records in WGM and in directory utility under Computers....

Reply

Answer 14

Linc Davis

Level 10

209,547 points

Aug 15, 2014 9:38 PM in response to Picoscope

Many Open Directory problems can be resolved by taking the following steps. Test after each one, and back up all data before making any changes.

1. The OD master must have a static IP address on the local network, not a dynamic address.

2. You must have a working DNS service, and the server's hostname must match its fully-qualified domain name. To confirm, select the server by name in the sidebar of the Server application window, then select the Overview tab. Click the Edit button on the Host Name line. On the Accessing your Server sheet, Domain Name should be selected. Change the Host Name, if necessary. The server must have at least a three-level name (e.g. "server.yourdomain.com"), and the name must not be in the ".local" top-level domain, which is reserved for Bonjour.

3. The primary DNS server used by the server must be 127.0.0.1 (that is, itself) unless you're using another server for internal DNS. The only DNS server set on the clients should be the internal one, which they should get from DHCP if applicable.

4. Follow these instructions to rebuild the Kerberos configuration on the master.

5. If you use authenticated binding, check the validity of the master's certificate. The common name must match the hostname and domain name. Deselecting and then reselecting the certificate in Server.app has been reported to have an effect in some cases.

6. Unbind and then rebind the clients in the Users & Groups preference pane. Use the fully-qualified domain name of the master.

7. Reboot the master and the clients.

8. Don't log in to the server with a network user's account.

9. As a last resort, export all OD users. In the Open Directory pane of Server, delete the OD server. Then recreate it and import the users. Ensure that the UID's are in the 1001+ range.

Reply

Answer 15

Picoscope Author

Level 1

22 points

Aug 15, 2014 9:47 PM in response to Picoscope

Just looked through the records and found my servers computer record right where I left it:

cn=myserver$,cn=computers,dc=myserver,dc=mydomain,dc=net

NB that the "Machines" directory is empty and as far as I remember, always has been.

Reply