Newsroom Update

Beginning in May, a special Today at Apple series titled “Made for Business” will offer small business owners and entrepreneurs free opportunities to learn how Apple products and services can support their growth and success. Learn more >

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

copying data from mac to windows server

Hi all,


Apologies in advance if this has ended up in the wrong section of the forum, hoping someone could point me in the right direction.

I work for a company that currently stores its files on an OS X Journaled NAS device, connected to a Mac Mini via lightning cable, and the mac connects into the network via ethernet.

We are in the process of planning a large data migration (~18TB) from the mac formatted NAS device, onto a Windows Server (using NetApp as the storage solution / VM datastore)

I'm wondering what the best application to manage the transfer of data might be? In Windows environments I've used Robocopy or FTP and the likes but not really familiar on the Mac side of things when it comes to data migrations.

We have a paid version of ChronoSync that we use to run our nightly backups to other external NAS devices - I see this has come up in a few Google searches.


2 others products that are popping up in my research are arRsync and SuperDuper - can anyone comment or recommend these products?


This is what I'm looking for in an app to manage the transfer:

- support copying file attributes from MAC-to-Windows world

- support resuming if the transfer fails / pauses

- be able to provide a meaningful summary for piece of mind when the transfer is completed (kind of like ChronoSyncs log)

- GUI based and have a nice interface 🙂

- Be reliable for transferring large date - currently around ~18TB


Looking forward to hearing your feedback, definitely let me know of any "gotchas" or things I need to take into consideration during the planning stage.

Matt


Mac mini, OS X Yosemite (10.10.5)

Posted on Feb 9, 2016 9:28 PM

Reply
Question marked as Best reply

Posted on Feb 18, 2016 4:46 PM

So I will bite and offer some advice. Been through this a lot (sigh).


First, some of the things to watch for:

1: Naming. Mac users on Mac storage will can use any character they want other than : and /. For example my.file!!@##$>that is so important##$123. . can be a file name. So can *********REALY IMPORTANT FILE!!!!!! Yes there are spaces in the from and end. Yes this will make Window panic. You must do a review of file and folder names BEFORE trying to migrate. Look for files that start and end with the space character. Beware the Windows reserved characters. Clean up your naming before migration. For details, see here https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

2: Maximum path length. Yep. In this day of unicode many parts of Windows can only handle a max file system path of 256 characters. See link above. If you are coming from a system that supported longer paths (like OS X and *nix operating systems) than you will want to review the length of your file paths. Test, test, and test some more with all your tools, including your backup software. Make sure it can both see paths that exceed the API limit and also that it is capable of restoring data in paths that exceed the length.

3: SMB is still a nightmare. From rampant file ID numeration, to slow directory reading, to hung Finder. Your Mac OS version will have varying degrees of success. 10.8 was a mess with DFS. 10.9 was a train wreck with Windows Clustered servers. 10.10 has huge issues with file ID enumeration and periodic deadlock issues. Once again, test in your environment. Get cozy with the nsmb.conf file.

4: Don't let your Macs sleep. AFP supports idle reconnect. Connect to a server, let the mac sleep, on wake it can renegotiate the connection to the server volume (usually). SMB does not do this. If your Macs fall asleep with documents open from the SMB share, you are in a world of hurt.

5: Be prepared to not be able to find anything. Searching on Windows shares has been a frustrating situation. You will not have Spotlight support from the server so you can either use a directory trawling search tool like Find Any File or be prepared to maintain Spotlight indexes on each workstation and hope for the best.


As for the methods to get the data from one place to the other, you need to realize there is a time constraint. Using a 4 GB/min rule of GigE transfer, assuming no issues and no disruptions, the 18 TB of data will take 75 hours to copy. Now, since this is a NAS going to a Windows system, you likely will not get 4 GB/min so increase that number by 20%. If the Windows server has on write virus scanning, add another 10%. If possible the best advice I can give you is to do this in logical chunks. Now, I don't know your data set so this may not be possible. However, if you have multiple shares, move one share per weekend to ensure you have ample time to perform the copy and also to correct any issues that may arise.


Regarding the tools, I've always used rsync because it allows for both detailed logging and also in line incremental runs. Should it get disrupted for any reason, you can pick up where you left off. Sadly, this is not GUI based. Also, the rsync 2.6.3 included with OS X is not adequate in supporting all file system features. I prefer to build a copy of the latest 3 branch. I will also include some of the patches like file flags and hfs-compression. If this is just data, the patches may not be needed. If you need a GUI tool, CronoSync is adequate.


And finally, if you find that SMB on Windows is just too frustrating, there is Acronis Access Connect. This is an add on to Windows server to provide native AFP and Spotlight. In the past few years I have been plagued with SMB issues across numerous versions of OS X. Granted, many of the issues are in corporate environments where I do not have any visibility to the Windows server config. I have no idea what they are doing on those machines, but I do know that OS X integration has been rough.

Hope this helps. Good luck on your project. Test and test some more before putting before your end users.


Reid

Apple Consultants Network

Author - "El Capitan Server – Foundation Services"

Author - "El Capitan Server – Control & Collaboration"

Author - "El Capitan Server – Advanced Services"

:: Exclusively available in Apple's iBooks Store

3 replies
Question marked as Best reply

Feb 18, 2016 4:46 PM in response to matt@hsco

So I will bite and offer some advice. Been through this a lot (sigh).


First, some of the things to watch for:

1: Naming. Mac users on Mac storage will can use any character they want other than : and /. For example my.file!!@##$>that is so important##$123. . can be a file name. So can *********REALY IMPORTANT FILE!!!!!! Yes there are spaces in the from and end. Yes this will make Window panic. You must do a review of file and folder names BEFORE trying to migrate. Look for files that start and end with the space character. Beware the Windows reserved characters. Clean up your naming before migration. For details, see here https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

2: Maximum path length. Yep. In this day of unicode many parts of Windows can only handle a max file system path of 256 characters. See link above. If you are coming from a system that supported longer paths (like OS X and *nix operating systems) than you will want to review the length of your file paths. Test, test, and test some more with all your tools, including your backup software. Make sure it can both see paths that exceed the API limit and also that it is capable of restoring data in paths that exceed the length.

3: SMB is still a nightmare. From rampant file ID numeration, to slow directory reading, to hung Finder. Your Mac OS version will have varying degrees of success. 10.8 was a mess with DFS. 10.9 was a train wreck with Windows Clustered servers. 10.10 has huge issues with file ID enumeration and periodic deadlock issues. Once again, test in your environment. Get cozy with the nsmb.conf file.

4: Don't let your Macs sleep. AFP supports idle reconnect. Connect to a server, let the mac sleep, on wake it can renegotiate the connection to the server volume (usually). SMB does not do this. If your Macs fall asleep with documents open from the SMB share, you are in a world of hurt.

5: Be prepared to not be able to find anything. Searching on Windows shares has been a frustrating situation. You will not have Spotlight support from the server so you can either use a directory trawling search tool like Find Any File or be prepared to maintain Spotlight indexes on each workstation and hope for the best.


As for the methods to get the data from one place to the other, you need to realize there is a time constraint. Using a 4 GB/min rule of GigE transfer, assuming no issues and no disruptions, the 18 TB of data will take 75 hours to copy. Now, since this is a NAS going to a Windows system, you likely will not get 4 GB/min so increase that number by 20%. If the Windows server has on write virus scanning, add another 10%. If possible the best advice I can give you is to do this in logical chunks. Now, I don't know your data set so this may not be possible. However, if you have multiple shares, move one share per weekend to ensure you have ample time to perform the copy and also to correct any issues that may arise.


Regarding the tools, I've always used rsync because it allows for both detailed logging and also in line incremental runs. Should it get disrupted for any reason, you can pick up where you left off. Sadly, this is not GUI based. Also, the rsync 2.6.3 included with OS X is not adequate in supporting all file system features. I prefer to build a copy of the latest 3 branch. I will also include some of the patches like file flags and hfs-compression. If this is just data, the patches may not be needed. If you need a GUI tool, CronoSync is adequate.


And finally, if you find that SMB on Windows is just too frustrating, there is Acronis Access Connect. This is an add on to Windows server to provide native AFP and Spotlight. In the past few years I have been plagued with SMB issues across numerous versions of OS X. Granted, many of the issues are in corporate environments where I do not have any visibility to the Windows server config. I have no idea what they are doing on those machines, but I do know that OS X integration has been rough.

Hope this helps. Good luck on your project. Test and test some more before putting before your end users.


Reid

Apple Consultants Network

Author - "El Capitan Server – Foundation Services"

Author - "El Capitan Server – Control & Collaboration"

Author - "El Capitan Server – Advanced Services"

:: Exclusively available in Apple's iBooks Store

Feb 14, 2016 9:03 PM in response to Strontium90

Nice one, thanks for the response.


At the moment I think we'll try and proceed with using ChronoSync to try and move the data across.


Still in the early stages, just trying to get all departments to do house cleaning & manual archiving so I can see if I can move some archive data at my own pace and then figure out how much "current" data is remaining to move so I can make a more detailed plan.

Feb 18, 2016 4:52 PM in response to Strontium90

Just a heads up in case you were curious

I haven't checked any of the file attributes / extended attributes yet but just for a ballpark idea on speed I did a test copy of 450GB using ChronoSync scheduler during the busiest part of the day and the results are below:


[2016-02-18 11:50:01 -0800] Synchronization start for: TEST-DataMigration.

[2016-02-18 14:09:05 -0800] Scanned: 16664 Processed: 16662 Data copied: 445.47 GB Skipped: 0 Errors: 0

[2016-02-18 14:09:06 -0800] Total Duration: 02:19:05

[2016-02-18 14:09:06 -0800] Operation completed

[2016-02-18 14:09:06 -0800]


So going by that roughly it'll take ~4.5hr/TB


16.23 TB total on file server now....but I should be able to migrate a fair bit of "reference data" over the next couple of weeks to minimize the total size, making the actual cut over data size as small as possible.

copying data from mac to windows server

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.