MacOS 10.14.6 NTFS bug leading to data loss

In the actual case I had a directory containing about 13 files that were created under Linux. They were a few levels down from the root of the NTFS drive in a directory that unfortunately had a name ending with a ?

The copying was done not on the Linux machine but on a Mac as I needed to free up some space on the NTFS drive and I had plenty of space on a APFS drive. So I dragged and dropped the top level directory from the MacOS finder from the NTFS drive to the APFS drive. Some time later the copy was complete and I moved the NTFS drive back to Linux and removed the files I thought I'd copied.

I don't have APFS drivers for Linux so I can't look at that drive other than on a Mac.

Sometime later I wanted one of those 13 files, went to the APFS drive and couldn't find them. They were not on the NTFS drive either because I had erased them assuming the copy had worked.

My expectation is that when an OS claims support for a disk format it should actually work. Being able to create files under Linux but not access some under MacOS is a problem.

I now have Paragon NTFS on the Mac which gives read/write NTFS access. This makes the problem worse because I can copy/move folders on the NTFS drive and loose problem file/directories again without warning. I contacted Paragon and they software is only used when writing to NTFS. They rely on the native MacOS functionality to read the drive.

When raising problems with companies I find it useful to give a simple set of steps to reproduce an issue so they can quickly test and evaluate what may be going on.

Apple unfortunately do not seem to have a mechanism to just report an issue hence my post here. Hopefully at least others will now be aware that there is an issue an take steps to ensure they don't suffer any data loss as a result.

A complete worked example is in the text attachment since I've hit a character limit.

In conclusion, when copying on the Mac neither directory? nor its contained file directory?/file2.txt were copied and no warning given.

A bit of an update. I took the drive into work where I use a Windows 10 (build 1903).

• under Windows it identified the directory ending with a ? but indicated that there was a problem and prevented accessing it.
• under MacOS there is no indication that the directory even exists or that there is an issue.

So with Windows I am warned there is an issue. Thus prompting taking the drive back to the Linux machine to rename the problem directory before trying again. It would be nice if MacOS did the same thing. It would be even better if it allowed access to the directory.

In conclusion. Linux allows the creating of file/directory names on an NTFS volume that are not portable. Windows detects this and warns the user, MacOS doesn't.

Using WSL under Windows 10 seems to have fewer restrictions on file/directory names. A ? is the possible but it is replaced with a different character in the GUI. The 1st directory "directory?" is the one created under Linux.

Back on the Mac (note win.png below is the above snapshot):

It seems that under Windows10 an alternative character is used for the ? which the Mac displays as the question mark in a box.

ExFat has some similar issues hence my move to NTFS.

In the end it would be useful to know when there is a problem which, as my reply above says, Windows 10 does.

Here is the output from the terminal from Linux:

/media/james/data/test$ ls -l

total 0

-rwxrwxrwx 1 james james 0 Jun 3 21:50 a_file.txt

drwxrwxrwx 1 james james 0 May 25 14:31 'directory?'

drwxrwxrwx 1 james james 0 May 25 14:31 directory.

-rwxrwxrwx 1 james james 0 May 25 14:32 'file?'

-rwxrwxrwx 1 james james 0 May 25 14:32 file.

and the same drive when mounted under MacOS

james$ cd /Volumes/data/test

natsuki:test james$ ls -al

total 24

drwxr-xr-x 0 james staff 4096 3 Jun 21:50 .

drwxr-xr-x 0 james staff 8192 3 Jun 21:51 ..

-rwxr-xr-x 1 james staff 0 3 Jun 21:50 a_file.txt

Notice that on the Mac only one file is present and the other files and directories are not.

In finder dragging the test folder to the desktop and:

james$ cd ~/Desktop/test/

natsuki:test james$ ls -al

total 0

drwxr-xr-x 3 james staff 96 3 Jun 21:50 .

drwx------+ 17 james staff 544 3 Jun 21:55 ..

-rwxr-xr-x 1 james staff 0 3 Jun 21:50 a_file.txt

The other files and directories were not copied and no warning given. Hence data loss.

Is that a bit clearer?

The copying was done on MacOS from the external NTFS drive to my MacOS desktop. The files that exist (under Linux) are not seen my the Mac (a bug) and hence the copy on the desktop is incomplete.

The files are empty as they were created to illustrate the problem. They were created using touch/mkdir on Linux. There were no additional attributes added. I'm at a loss as to how you view flags and extended attributes (on the Mac) when they don't show up when listed. ls -a shows all files including those starting with a .

The Mac is copying the files and directories it sees. Since it is not spotting all files and directories it can't perform a complete copy. i.e. a bug and quite a serious one. What should be a copy of the test directory and its contents on my Desktop isn't.

so specifically "They appear to have copied, but are not shown in macOS"; the last ls output is of the "copy" under MacOS. The files are not there - MacOS is not showing files/directories on a APFS partition. So I struggle to see how you can claim they have copied.

my "solution" is to copy the files over a network. Attaching and relying on MacOS to mount the NTFS drive + contents is not reliable.