Screen Sharing / vnc with OS X 10.5.6 fails

Question

Level 1

0 points

Screen Sharing / vnc with OS X 10.5.6 fails

I can not get the Screen Sharing that comes with OS X 10.5.6 to work with any vnc client. The server is my laptop, an MBP 2.4 GHz Intel running OS X 10.5.6.

*I enable Screen Sharing by doing the following:*

1) +System Preferences > Sharing > Screen Sharing+ is checked

2) +System Preferences > Sharing > Screen Sharing > Computer Settings > Anyone may request permission to control screen+ is checked

3) +System Preferences > Sharing > Screen Sharing > Computer Settings > VNC viewers may control screen with password+ is checked and I've entered a password

4) I also have Remote Login enabled for SSH access, but no other services are enabled.

*I see two different behaviors depending on one of two clients that I try using:*

1) +Chicken of the VNC v2.0b4+ running on same machine- Always says "Connection Terminated / Authentication Failed" without ever asking for a password. Logs the following in system.log (dates, hostnames and pids removed):

Chicken of the VNC: Server reports Version RFB 003.889
Chicken of the VNC: Bogus RFB Protocol Version Number from AppleRemoteDesktop, switching to protocol 003.007

I believe that the information in system.log is OK, and is just warning me that Chicken of the VNC is working around Apple's strange protocol version.

2) +TightVNC 1.3.9 on Windows XP sp2+- Says "Security type requested", then asks for a password. I enter it, click OK and tightvnc simply waits endlessly.

*Other Notes:*

1) I have tried configuring and starting the server using kickstart from the command line as suggested at http://support.apple.com/kb/HT2370 (+sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/k ickstart -activate -configure -access -on -restart -agent -privs -all+)

2) I have tried unchecking +VNC viewers may control screen with password+, but in this case Chicken of the VNC says "Please configure Apple Remote Desktop to allow VNC Viewers to control the screen. Unknown authType 30,31,32" and TightVNC says "Server did not offer supported security type!"

3) If I disable the Screen Sharing feature that comes out of the box with OS X, I CAN install and successfully run the OSXVnc server available at http://sourceforge.net/projects/osxvnc/. In this case, both Chicken of the VNC and TightVNC are able to connect.

*So Apple's Screen Sharing does not work. Does anyone know what the problem is? Are there any log files I can look at, or is there another way I can debug this? I'm not impressed. :)*

MBP 2.4 GHz, Mac OS X (10.5.6), Apple TV, iPod nano

Posted on Feb 19, 2009 3:35 PM

Reply

Answer 1

BobHarris

Level 9

53,009 points

Feb 19, 2009 4:43 PM in response to phinaesgage

Do you have the Firewall enabled? Try disabling it.
System Preferences -> Security -> Firewall?
Allow all incoming connections.

If that works, you can play with the firewall settings until you find a combination that works for you.

I've been using Chicken of the VNC for years.

I've also been using JollysFastVNC more recently, and it has some improvements over CotVNC.

The only problem I've had with the build-in Mac OS X Leopard VNC server is if I tell CotVNC to use less colors (like 8 bit), then I generally only get a black screen. If I let CotVNC use full color, it is not a problem.

And if using the Leopard Screen Sharing client, I've found that performance has been very good.
Finder -> Go -> Connect to Server ... vnc://address.of.remote.Mac

Finally, you can use the Vine Server (aka OSXvnc) if you do not like the built-in VNC server.
<http://www.versiontracker.com/dyn/moreinfo/macosx/16699>
which has worked well for years.

Reply

Answer 2

phinaesgage Author

Level 1

0 points

Feb 20, 2009 12:07 AM in response to BobHarris

The firewall is set to "Allow all incoming connections," so it's disabled. I am receiving a password prompt in TightVNC, so the connection is being made. If the firewall were enabled then the connection would be refused at the tcp level.

I tried JollysFastVNC, and there are two different behaviors depending on which authentication mode I choose:

1) When I set VNC authentication mode in JollysFastVNC, (which is what TightVNC and CotVNC use), if I enter the password it just keeps asking me for it again. Here's what is written to system.log:

JollysFastVNC: (0015dde0.0208)-[NetworkConnection connectWithSettings:] trying:localhost
JollysFastVNC: (0015dde0.0602)-[NetworkConnection connectToAddress:securityType:] Not using a socks proxy
JollysFastVNC: (0081d600.0325)-[VNCProtocol readSecurityResult] Caught JNXAuthenticationException: VNCClient start: 3.8 security type connection failed.
JollysFastVNC: (0081d600.1407)-[VNCProtocol setupProtocol] Caught JNXAuthenticationException: VNCClient start: security type connection failed

2) When I set Mac authentication mode in JollysFastVNC, I enter my Mac username and password and it works.

So I can get Mac authentication mode working, but VNC mode authentication is failing no matter what client I use. This has to be the server that's the problem. Again, if I use the OSXVnc server, it works, but I would like to use Apple's VNC server that ships out of the box with VNC authentication mode.

Reply

Answer 3

BobHarris

Level 9

53,009 points

Feb 20, 2009 2:18 PM in response to phinaesgage

The JollysFastVNC does narrow down the issues to just an authentication failure, since the server does work using Mac authentication.

I think the VNC password is stored in
/Library/Preferences/com.apple.VNCSettings.txt

There are several other files that get changed when you mess with your Remote Management settings:
/Library/Preferences/com.apple.RemoteManagement.plist
/Library/Preferences/com.apple.RemoteDesktop.plist
/Library/Preferences/com.apple.ARDAgent.plist

You might try trashing some of these files. Of course that will mean reconfiguring your remote desktop server.

Reply

Answer 4

phinaesgage Author

Level 1

0 points

Feb 24, 2009 3:36 AM in response to BobHarris

Thanks for the suggestion. It was possible that one of these files had become corrupted, so I tried moving each of these files to another directory but unfortunately I'm still seeing the same behavior.

Here's what happens with each of these files individually after they're moved:

1) When I set the VNC password in the system prefs, com.apple.VNCSettings.txt gets re-created (with the same contents as the original unless I change the password, so that just looks like a hash of the password).

2) com.apple.RemoteManagement.plist also gets recreated when I change the system prefs. If I use "sudo plutil -convert xml1 com.apple.RemoteManagement.plist" I get the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>ARD_AllLocalUsers</key>
<true/>
<key>ARD_AllLocalUsersPrivs</key>
<integer>2</integer>
<key>ScreenSharingReqPermEnabled</key>
<true/>
<key>VNCLegacyConnectionsEnabled</key>
<true/>
</dict>
</plist>

3) For me, com.apple.RemoteDesktop.plist does not exist.

4) Interestingly, com.apple.ARDAgent.plist does not get created until I run "sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/k ickstart -activate -configure -access -on -restart -agent -privs -all". Its XML contents are as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>AdminConnectInfoList</key>
<dict/>
<key>AgentLogLevel</key>
<integer>3</integer>
<key>ServerConnectInfoList</key>
<dict/>
<key>Version</key>
<real>3</real>
</dict>
</plist>

Looking at this, I tried setting ScreenSharingReqPermEnabled to false, but that didn't help. Then I increased AgentLogLevel to 100 and got a lot of output in system.log (12:28:45 is when I restart the server and 12:29:34 is when I try to connect):

-----

Feb 24 12:28:45 ARDAgent [749]: ******ARDAgent Launched ******
Feb 24 12:28:45 ARDAgent [749]: got a sessionDict, onConsoleRef is 0xa04cd400
Feb 24 12:28:45 ARDAgent [749]: grUserOnConsole is 1
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 122
Feb 24 12:28:45 ARDAgent [749]: LoadMenuExtra: Attempting to unload menu extra
Feb 24 12:28:45 ARDAgent [749]: PostNotificationForced: Going to send notifation value 9
Feb 24 12:28:45 ARDAgent [749]: UpdatePrefs: versionFlt : 0.000000 kCurrentPrefsVersion : 3.000000
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 44
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 22
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 100
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 94
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 96
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 18
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 106
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 74
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 130
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 60
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 45
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 65
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 66
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 105
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 115
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 38
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 12
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 61
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 20
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 62
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 39
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 30
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 101
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 53
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 103
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 107
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 109
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 135
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 41
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 110
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 111
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 28
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 36
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 48
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 59
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 57
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 116
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 58
Feb 24 12:28:45 ARDAgent [749]: InitAppUsageHandlers: Called gTrackingDays is set to 0
Feb 24 12:28:45 ARDAgent [749]: CheckRFBServerPIDFile: return 679
Feb 24 12:28:45 ARDAgent [749]: RemoteCommandListenerThread init communications
Feb 24 12:28:45 ARDAgent [749]: RFBServerStart - did not kill 679
Feb 24 12:28:45 ARDAgent [749]: DOCStartDOC: No serial number. Task Server not started.
Feb 24 12:28:45 ARDAgent [749]: DT_InitLocalProcessing: Loading existing tasks from disk.
Feb 24 12:28:45 ARDAgent [749]: DT_InitLocalProcessing: Found 0 tasks
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 158
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 117
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 118
Feb 24 12:28:45 ARDAgent [749]: AddHandler: Added handler for cmdCode 162
Feb 24 12:28:45 ARDAgent [749]: ARD Agent: RFB Server exited quickly after starting - probable failure.
Feb 24 12:28:45 ARDAgent [749]: PostNotificationForced: Going to send notifation value 6
Feb 24 12:28:45 ARDAgent [749]: PostNotificationForced: Setting lastNotification to value 6
Feb 24 12:28:45 ARDAgent [749]: ******ARDAgent Ready ******
Feb 24 12:29:34 Chicken of the VNC[693]: Server reports Version RFB 003.889
Feb 24 12:29:34 Chicken of the VNC[693]: Bogus RFB Protocol Version Number from AppleRemoteDesktop, switching to protocol 003.007
Feb 24 12:29:36 ARDAgent [749]: PostNotificationForced: Going to send notifation value 1
Feb 24 12:29:36 ARDAgent [749]: PostNotificationForced: Got request for kCurrentStateNotification. Sending value 6

-----

Interesting that it says "ARD Agent: RFB Server exited quickly after starting - probable failure." That doesn't sound too good.

I will try to watch fs_usage more closely to see what else gets modified and accessed, but I'm increasingly suspicious that there's simply a bug which is showing up under whatever specific conditions I have on my machine. Perhaps the included log info will trigger an idea for someone.

Reply

Answer 5

phinaesgage Author

Level 1

0 points

Feb 27, 2009 10:24 AM in response to phinaesgage

I finally had to solve this with an archive reinstall of OS X. I was experiencing a barrage of problems related to security and the login process, including this problem with Screen Sharing, problems with printing and a login loop problem. See these related posts:

Regarding what I was seeing with printing:

http://discussions.apple.com/thread.jspa?threadID=1920520&tstart=0

Regarding what I was seeing with the login loop:

http://discussions.apple.com/thread.jspa?threadID=1920551&tstart=0

I suspect that there was something related to security or login which had become corrupted in the OS, and that these problems may have been related. I was seeing CUPS messages telling me that authentication info hadn't been provided, and was seeing that login loop once I added another user.

It was all solved by the magic dust of the archive reinstall.

Reply

Answer 6

gmcouto

Level 1

0 points

Mar 18, 2009 10:01 AM in response to phinaesgage

Does it have other solution? I am experiencing exactly the same problem, but dont want to reinstall the OS.

Reply