I have been having the same problem periodically in Quicktime. I have been able to take up to 5 hour long recordings and they save just fine, as long as you have the space for it. But what I have noticed is that I only have this problem, "The Operation Could Not Be Completed" when saving, if I start a new screen recording before saving the first one. It is the first one that I am then unable to save.
I have had some luck that if I trim the first recording I am sometimes able to save it. But this time I did not have enough extra recording to trim much and so it still wouldn't save, which is when I found this thread.
I resorted to making a new screen recording of my recording (silly I know, and it probably reduced my resolution and sound quality). Since the recording that would not save still plays in Quicktime, I was able to make a recording of my recording and that saved just fine. So I was at least able to keep the content, which for me is much more important than the quality.
I realize this tread started a while ago, but I hope it may help anyone else struggling with the same issues.