Some Windows 7 computers cannot log in to Yosemite Server regardless of user

I've been banging away at it all day, and I did turn up posts here and there from few people having what sounded like the same problem, usually with old Windows servers (or having a mismatched timestamp). Unfortunately, none of the suggested fixes worked (though the confirmation that that particular error message can, in fact, have absolutely nothing to do with what the actual problem is was useful). Rebooting the server also did not help.

However, just a few minutes ago, I succeeded in getting one of the problem machines to connect after both fiddling with a registry security setting (increasing, rather than decreasing, the level, forcing a more stringent protocol) and disabling Windows Firewall. I do not actually think that the fix was the registry fiddling--pretty sure it still didn't work with just that changed--so my current working theory is that there's something wonky with the firewall on the problematic boxes.

I took a break, however, so haven't dug in farther to see if a firewall reset will fix it, or if something more drastic is required, and if the fix will, in fact, be permanent (and work on the other problem system).

Will post once I have done more testing.

Okay, figured it out!

I was wrong, it was the obscure registry security level thing, not the firewall.

The clients that did not work had the registry key LmCompatibilityLevel in HKLM\System\CurrentControlSet\Control\lsa set to "1".

The clients that worked properly were missing that key entirely (turns out, it defaults to "3" in that case).

Changing the key from "1" to "3" fixed the problem immediately. Didn't even require a reboot.

As for the why, looking at the relevant TechNet article, 1 is "Clients use LM and NTLM authentication, and they use NTLMv2 session security if the server supports it." while 3 is "Clients use only NTLMv2 authentication, and they use NTLMv2 session security if the server supports it." Based on that, one would assume that Yosemite Server is doing something such that Win7 thinks it is asking for NTLM but it won't acctually accept it, while if NTLMv2 is forced it works fine. According to this Microsoft post, the default for Win7 is 3, which explains why that value (which I had initially chosen somewhat arbitrarily) works.

Obviously, deleting the key entirely would also work, since it would effectively force a 3.

Glad to hear that helped someone else.

What I can't figure out is how that key got set in the first place on my systems, since it's supposed to default to 3 on Windows 7 anyway. What lead me there to begin with was some people with what sounded like the same problem but the opposite solution--they couldn't connect to old Windows servers with Windows 7 clients unless they set it to 1, since the old servers don't support NTLMv2.

Regardless, unless there are old NAS shares or very old Windows servers you need to connect to, your IT department shouldn't have any problem removing that key or raising the value to 3, since that is only going to increase security and it isn't even set in most installs by default.

frozen999 wrote:

Strange this is that my Win7 Home x64 had no such login problems to the Yosemite box. Only my Windows7 Pro x64 did I have to edit the registry settings.

Actually it's the Pro one that's strange; according to Microsoft, Win7 doesn't have this set by default (the registry entry doesn't exist), and it will default to "3". This matches up with most of the installs at work--the ones that didn't have it overridden by IT settings didn't have the setting at all, and worked fine. I don't know what would have caused your Win7 Pro box to have that registry setting both exist and be set to a non-default value, but maybe you (or another user?) did it at some point to address sharing issues with an old Windows server? That's usually why that value gets overridden--that's why our IT higher-ups did it.

tjflyin wrote:

Marc, thanks lots for the fix. Perfect. How did you find this in registry? Must know way too much about Windows.

So on this same theme, I have a Win 10 preview machine running by way of VirtualBox, and it's file sharing has stop with update to Yosemite and Server 4.0 from Snow Leopard server. It worked without problem under Snow Leopard server.

I looked in regedit for the same thing, but there is no "LmCompatibilityLevel" listed in the Win10 regedit.

Any ideas?

It was mostly a hunch based on similarity to a problem connecting with old Windows servers caused by the opposite (security too low on that value) after a lot of forum post hunting and trial-and-error.

As for your Win10 issue: In Win7, LmCompatibilityLevel does not exist in the registry by default--the OS just defaults the value to 3 if it isn't there. So unless they've changed this, I wouldn't expect it to exist in a stock Win10 install, either. The question, though, is whether that's what's causing the problem; I don't know whether Win10 defaults to a different value (maybe a higher level that is incompatible?), whether that property no longer exists in Win10, or whether there is something completely unrelated causing the problem.

In any case, it's easy enough to just add the key and set a value. If Win10 does still use it, it will then override whatever the default is, and if that's what's causing the problem, it should fix it.