Same think here. I bought a new MBA 11" (Early 2015) at start of this year that already shipped with 10.11.3 and I had no problems whatsoever.
Than I installed 10.11.4 and the hole system starts to freeze completely from time to time while watching YouTube-Videos with Safari.
First time it happen I thought as a long time OSX user "well that can happen, even if it is extremely rare(once for 2 or 3 years or less) on a Mac"
(but normally you see the debug output of a kernel freeze too).
Second time I was wondering if my Mac had an hardware issue? So I start to check with Apple Hardware check and a Ram-test, but both found no hardware issues?
Third time I start to lock into the console log files and found that log entry of watchdogd that WindowServer is not responding.
"service (com.apple.WindowServer) reported as unresponsive"
I google for it and founds a lot of forums and threads with people having the same issue since 10.11.4 so it is probably an issue of the operating system. Since the update 10.11.4 changes the intel graphics drivers this would be my first guess what causes the problems or at least is involved in it.
After 4 freeze I start to become a bit angry and enable the ssh-server on my mac that in case of next freeze I can try to kill the hanging windowserver and restart the GUI. I test in normal operation and a kill of the WindoServer restarts the GUI as expected.
During the 5 freeze I really could connect to the frozen system via SSH and so we can say that not the entire system is frozen some processes still running, but all the GUI is frozen.
I try to kill WindowServer as I tried before but the „killall -HUP WindowServer“ was not working, also an „top“ was hanging maybe due to the hanging WindowServer. After restart of system again I realized that I not tried the harder „killall -KILL WindowServer“.
I did not wait for the 6 freeze and roll back 10.11.3 system from an older time machine backup. And than rollback my user folder and applications from a newer backup. I restored my newer applications, because an older version iTunes (like the one of 10.11.3) normally not works with an iTunes-Library that was opened with an newer version of iTunes (like the ne from 10.11.4). But after restoring this mixed version of the two backups iBooks was not longer working. And after I try to reconnect my account in iTunes I saw that also a connection was not longer possible with iTunes. In system logs I saw a problem with the PrivateFramework BookKit.framework. So I restore from my older backup Version 1.4 (I had 1.5 before). Now iTunes Account could reconnect, but iBooks was still not working. I need to delate my local iBooks-Library first and start a new one before iBooks was again up and running.
So final conclusion is that Backup with 10.11.3 is only than a good solution if you can life without newer data. But if you mix a backup of system from 10.11.3 with user and application data backup of 10.11.4 you can end up with a lot of mess as I did. Maybe my main mistake was that I roll back my applications from 10.11.4. So take care if you try.
Now my 10.11.3 system with newer user data is up and running and I had no more freezes in the last 3 days. Fingers crossed. 😉