I've also had weird issues between my Macs and my QNAP NAS. I THINK I may have finally closed in on a solution. I wish I had stumbled on this thread before now; I've been thinking I'm the only one with issues.
Anyway, for what it's worth, here are some conclusions I've reached (for now) in no particular order:
1. Do NOT use more than one protocol unless you need to and you take great care. OS X considers a share via SMB to be different than a share via AFP. (This doesn't appear to be true between two Macs, just between a Mac and another type of machine). You can prove this by creating a Photos or iTunes library on your NAS with AFP, then switch to SMB and try to start Photos or iTunes. It will complain that the library is missing and you'll have to point to it manually before you can use it. I found this out the hard way when an iTunes library got completely hosed. Several songs/movies/podcasts would be recreated with a version number (file, file 1, file 2, etc.) because of this. I could clearly see what was going on by looking at "iTunes Library.xml". What a mess!! Good thing I still had my original library.
2. AFP works much better than SMB ... until it doesn't. QNAP's AFP implementation is apparently some third party or open-source non-Apple solution that works pretty well, but not 100% reliably. Since Apple is deprecating AFP, you could say that it's as good as it's going to get, so maybe you should stay with it. I did for a while but got tired of issues continuing to crop up, so I figured I'd go with SMB. Besides, SMB is the future for both Windows and OS X.
3. SMB is a work in progress. Apple is dumping GPL-licensed SAMBA and rolling their own solution. It has been improving and I'm sure it will continue to do so. That said, there are some issues and a major one appears to be NTLM authentication. Starting with Yosemite (apparently), Apple uses only NTLMv2 to authenticate SMB connections. The default on ALL Windows installations prior to Windows 8 is to use NTLMv1 but allow negotiating to v2. What's weird is that Microsoft has tech notes out starting in the 90's!! talking about how insecure v1 is and you should use v2; why is that not the default? So I don't really blame Apple for this one. Anyway, long story short, I checked the box on my NAS that said "Allow only NTLMv2 authentication" and voìla, my SMB issues went away ( ... almost, see below). I had erroneously assumed that meant forcing the use of an older method as opposed to say "NTLMv3" (which doesn't exist as far as I know). Anyway, for those whose troubles started with Yosemite, this might be the problem. Regardless, knowing what I know now, I would avoid NTLMv1 whenever possible.
[As an aside, it's really easy to reconfigure Windows NT - Windows 7 to only use NTLMv2. It's just a registry entry that can be controlled by the "local security policy" control panel. Just Google it, it's easy to find. Windows 95, 98, and ME can be configured that way also but you've got to install the Active Directory Client Extension. I've done this for Windows 98 that I run in Virtual PC under XP, Win 8, and Win 2000 under Parallels desktop. It just works awesomely. Windows 8 seems to work just fine out of the box and that registry entry doesn't exist.]
4. After a few blissful days of absolutely no SMB issues, I had Finder lock up and extreme weirdness when trying to back up some large files to my NAS, some of which happened to be packages (mostly Parallels virtual machines). Things were really hosed and re-launching Finder just made everything hang. What fixed that and left me where I am now was UN-checking QNAP's SMB setting: "Enable asynchronous I/O". This had been un-checked by default, but I had enabled it at some point. Interestingly, in testing this, every file so far copies much faster with this unchecked. I believe the expected behavior is for async to be somewhat faster on average. Possibly with a bunch of very small files, that may be true - I just haven't tried.
So here I am and everything appears to be working well so far. I hope this helps others.