Q: Server File Sharing unstable on mac and windows network on Mavericks and OSX Server 3.0

Arrrggghh!  Just after posting, it happened to two files I had open.   So frustrating.  I guess I'll try the second part of your suggestion now...

Based on my initial testing with 1 Windows machine having OpLocks On and the other with OpLocks Off and all shring under the SMB Mac share:

If the same file is opened by 2 diferent Windows users the file will be locked by the first user either way.

If a file is opend by 1 Windows user and another by a mac user, the Windows user will lock the file.

If the file is first opened with 1 Mac user first then 1 other Windows user Windows will still be able to edit the file.

I opend Word, Excel, and Text files for this test. The Mac's acted the strangest in may case with the SMB share. If you are just using Mac's then shut off SMB sharing until Apple fully supprots SMB2. Man I wish I could go back to Snow Leopard. It worked the best in my opinion.

Below is a list of commands, settings and machines that I have tried to use  OS X server (SMB2) shares by Windows clients. This list might help someone else to get it working.

Testing machines

I used Oracle Virtual Box and a  Windows evaluation version for testing (it’s free and easy to set up). For Mavericks I used a ‘clean installation’ on a separate  partition. I also installed OS X in Virtual Box (OS X host, no hackintosh), that’s even more flexible. See also http://www.tekrevue.com/tip/create-usb-installer-os-x-mavericks/.

Windows

• Check which SMB dialect your Windows 8 shares are using:
  > Powershell (run as administrator): get-smbconnection.
• Disabling/Enabling SMB2: http://support.microsoft.com/kb/2696547
• Disabling opportunistic locking: Oplocks cannot be turned off for SMB2 (http://www.dataaccess.com/whitepapers/opportunlockingreadcaching.html; search for “Disabling Oplocks on SMB2” ).
• Session timeout: I successfully use SessTimeout = 3600 to prevent an MS-access database from losing its connection and video’s from pausing (files are shared on a Mountain Lion SMB1 server). Details: http://blogs.msdn.com/b/openspecification/archive/2013/03/19/cifs-and-smb-timeou ts-in-windows.aspx. 
  
• Capture network traffic ( http://www.wireshark.org/, also for OS X).

OS X server

• Force SMB1: switching the SMB stack on the OSX server to use SMB1 as a default (http://hints.macworld.com/article.php?story=20131122083837447) did not solve the problem. I guess it only works on the client side.
• Enable SMB ACL: To prevent other users in a group to lose access to shared files, you can turn on SMB Access control lists (ACLs) on the server (see http://support.apple.com/kb/ts4149).
  Additionally:  ‘sudo defaults read /Library/Preferences/SystemConfiguration/com.appl.smb.server’ to only list the smb settings. Or the alternative: ‘sudo serveradmin settings smb’
• Set ACL permissions: You can use the server app to set the ACL permissions (http://www.dummies.com/how-to/content/how-to-set-acl-permissions-in-os-x-lions-s erver-ap.html). Don’t forget to propagate the permissions!

Thanks for the post. I will have to dig deeper into the ACL permissions. Until 10.9.2 I was abble to just disable OpLocks on the Windows client and all was well. I know this is not supposed to work with SMB2 but it did make a difference.

I was also using Keepconn and SessTimesout with a time of 65535 but it didn't seam to work as well as I would have liked.

I think this SMB thing with Apple is going to be a moving target with each new update. After 10.9.2 I'm back to loosing connection with the added complication of locked files. This is with SMB2 off and OpLocks off on the Windows client. The Mac side is not much better now either after 10.9.2. I now connect to my Mac shares with AFP which has fixed the Mac client issues.

Grrrrrrr!

Just going to add a "me too" to the thread. I'm only running AFP (100% Mac environment) and READ SPEEDS from Mavericks Server are...well...pathetic. I'd started this thread detailing my testing and results (before finding this thread):

https://discussions.apple.com/message/25166110#25166110

I'm about to give up and install Mountain Lion Server.

I also wanted to update that, since I'm running all my tests in a testing environment, I was able to quickly install Mountain Lion Server on the same Xserve, set up exactly the same as Mavericks Server was set up, and the File Sharing issues I was seeing in Mavericks Server do not exist in Mount Lion Server. With Mountain Lion Server, I'm getting consistent, sustained speeds of 100 MB/s read/write (where as, in Mavericks, I was only getting 45 MB/s reads).

Am I able to install Mountain Lion Server on Mavericks 10.9.2 ? Or are they not compatible?

Do you have a copy of the old Mountain Lion Server ? Only 3 is abilable on App store.

No, not compatible. Mountain Lion Server is 10.8.x based. I'm going to take a new Macmini with 10.9.x server and see if it's capable of running 10.8.x Server (depends on model-ID and kernel of 10.8.x if this will work).

I'm also looking at some type of Linux distribution that is capable of giving the same functionality as OS X Server for filesharing - FTP- and perhaps mail (Kerio). It's a mistery to me what Apple's roadmap of OS X server and hardware-servers is at this time (I miss the old Xserve and 10.6.8 server

Check with Apple if you can get the old software via the Store (think you should call, I've heart that it is possible to get older versions by calling).

3.1 has just dropped - anyone know if the locking issue is resolved?!?!?!

I installed the 3.1 update as soon as it was released and tested my AFP speed issues and they still persist into the 3.1 release (i.e., Mavericks Server still only serves AFP read times of less than 50 MB/s). Booting back into Mountain Lion Server, speeds return to the full 100 MB/s reads.

I have a bug report open with Apple on this issue and am currently running tests for them.

k.

Server 3.1.1 now Available... Did anyone test the Update?

Updated to 3.1.1 this morning. Symptoms remain.

3/25/14 10:18:51.308 AM smbd[41719]: File system does not support 0X40000, file attrs

3/25/14 10:18:51.308 AM smbd[41719]: File system does not support 0X0 time attrs

3/25/14 10:18:51.308 AM smbd[41719]: File system does not support 0X0, size attrs

3/25/14 10:18:51.310 AM smbd[41719]: File system does not support 0X40000, file attrs

3/25/14 10:18:51.310 AM smbd[41719]: File system does not support 0X0 time attrs

3/25/14 10:18:51.311 AM smbd[41719]: File system does not support 0X0, size attrs

3/25/14 10:18:53.449 AM smbd[41719]: File system does not support 0X40000, file attrs

3/25/14 10:18:53.449 AM smbd[41719]: File system does not support 0X0 time attrs

3/25/14 10:18:53.449 AM smbd[41719]: File system does not support 0X0, size attrs

3/25/14 10:18:53.450 AM smbd[41719]: File system does not support 0X40000, file attrs

3/25/14 10:18:53.450 AM smbd[41719]: File system does not support 0X0 time attrs

3/25/14 10:18:53.450 AM smbd[41719]: File system does not support 0X0, size attrs

10.9.3 will fix the problem. Currently still in beta testing, but i can confirm that is is fixed in the latetest beta we got.

Hope 10.9.3 will come out soon.

With all due respect, the same was reported with the 10.9.2 beta, but the issues remained.

Server 3.1.1 didn't fix the AFP issues. Can't comment on the 10.9.3 beta (NDA), but again, "Server 3.1.1" didn't fix the AFP issues.