Can't get remote cluster machine to encode video - getting 'Media Server application unexpectedly quit'

Question

Level 1

47 points

Can't get remote cluster machine to encode video - getting 'Media Server application unexpectedly quit'

Hi

Using Compressor 4 / Apple QuarterMaster admin etc, I have set up a Cluster and it works well to Share > Export Using Compressor Settings. That took some doing, and I'm happy that it works.

If I do all my processing in the foreground, or alternatively in the background on my FCPX computer, ie without running or using any cluster in the background, it all goes well. Quick, error free.

If I activate the cluster and only put my local machine's compressor services into it, and then send my compressor batch to the cluster, it works perfectly well. But that defeats the point of having a cluster. I want the remote machine to do all the work so my local machine doesn't slow down.

Unfortunately, if I add the remote machine's services into the cluster, the encoding always fails with 'Media Server application unexpectedly quit' in the error log. I know that the cluster is distributing segments of the file to the remote machine - this can be seen in the Share Monitor, but they never get processed and sit there waiting until I get the error message.

The remote machine is a modern 4G core 2 duo MacBook. It's never been used for this before. It has Compressor 4 and the Pro Apps Update installed and both appear to work. Both machines are running 10.6.8. The remote machine has Compressor services initialised in Apple QMaster sharing. The QMasterAdmin sees these services. They are accessible over Bonjour and appear in the Cluster. Compressor services from both the local and the remote machine are configured exactly the same way. The same QT codecs exist on both machines. I've restarted, shift-restarted, etc etc.

So I am at a loss here. The remote machine just won't compress anything it is sent.

Any ideas? Must both have FCPX installed?

Anyone actually got a remote machine in a cluster to work with Compressor 4? I can't figure it out.

Chris.

Posted on Jun 28, 2011 12:31 AM

Reply

Answer 1

ctzsnooze Author

Level 1

47 points

Jun 28, 2011 7:04 AM in response to ctzsnooze

OK... finally sorted it out.

It's a bug, as far as I can tell.

Any job entered directly into Compressor 4 in the normal Compressor manner will be successfully rendered by any working cluster from any machine that can access the cluster. That's good. It means that the underlying distributed processing model works well.

HOWEVER - any job forwarded to a Compressor 4 cluster that includes non-local compressor services (ie compressor services not resident on the same machine), using Share > Export Using Compressor Settings direct from FCPX will fail.

To confirm this bug, I made a cluster on a remote machine. It was a dual core machine, so I enabled 2 compressor services on that machine, and that's all the cluster was. Simple.

I then manually entered a video file (ProRess 422 720p) into Compressor on the remote machine. I did this by physically setting up new job using the compressor user interface. A bog standard ProRes422 720p file rendered fine this way on the remote machine. As did anything else I gave it. Good.

Then on my main machine, I opened Compressor and made a job based on the same file and settings, and sent it to the remote machine's cluster. No problem at all! Great!

So now I know that both Compressor versions, and the clustering model, are working fine. In fact I can send all sorts of files to the cluster, from any other version of compressor, and have them processed on the remote machine, and get the result back on my desktop later on. Excellent.

But, if I try put this same file into a FMPX timeline, and go Share > Export Using Compressor Settings.... and select a cluster with remote (non-local) compressor services, it does not work. Rendering the video segments on the remote machine times out and fails, every time. It doesn't matter what file format I use, it just fails.

So it's a bug.

From what I can tell, Final Cut Pro X somehow messes up the Share > Export Using Compressor settings where the cluster includes non-local compressor services, causing all jobs to fail. The same Share > Export Using Compressor Settings will work quite happily if all the services on the cluster are on the same machine as FCPX, or if the job is sent to This Computer in the background. But any attempt to send files to a cluster using any remote services will fail.

I hope this saves some people from wasting as much time as I have!

One workaround is to export to ProRes then put this file manually into Compressor, sending the job to the remote cluster. This is a two step process with a large intermediary ProRes file. If one goes Export as QuickTime movie, generating the intermediary file prevents further FCPX work being done.

To get the intermediary in the background one could use Share > Export Using Compressor Settings via either This Computer, or to a cluster using only local services. Then once complete, manually add it to a compressor job.

So this is a FCPX bug as far as I can tell.

Chris.

Reply

Answer 2

Corbywan

Level 1

64 points

Jul 22, 2011 5:00 PM in response to ctzsnooze

Thank you for this. I'm seeing the same behavior. I can drop in a file and send it to the cluster, works fine. If I Share it from FCPX, chose the same preset, send it to the cluster, I get the same errors. I'm in the process of exporting my movie from FCPX and will drop it in and see if it works. Thanks again for duplicating what I'm experiencing.

Reply

Answer 3

Aug 12, 2011 8:21 AM in response to Corbywan

I'm also having a similar problem which I brought up in this thread https://discussions.apple.com/thread/3255880 before seeing this one.

In my case I was able to render but it just produces a video with a missing meda icon.

I am able to get it to work by rendering locally to a ProRes file and submitting that manually to compressor but that's a step that involves extra time and quality loss.

I'll try looking in to the "Export Using Compressor Settings" idea. Are you saying there is a way to get it to produce some kind of job file or reference movie that can be submitted directly to compressor?

Reply

Answer 4

Aug 23, 2011 12:47 PM in response to ctzsnooze

Ugh. Thank you so much. I was driving myself crazy trying to figure out why this wouldn't work. Hopefully a .1 update to FCPX isn't too far off.

Reply

Answer 5

Aug 23, 2011 5:41 PM in response to Alex Serriere

I don't know if we'll be seeing an update anytime soon. They have ALOT of stuff to fix.

Anyways in the meantime I export from FCPX to the cluster storage as ProRes422 and then I feed that to Compressor. I have the destination set to cluster storage as well. In Compressor Preferences under Cluster Options you can select "Never copy files to/From cluster". It's pretty fast and exporting to the cluster storage isn't so bad as it would have had to copy it there at some point anyways.

I also found for longer pieces I can export multiple projects to the cluster and then join them all together in a Quicktime reference movie. As long as everthing is on the cluster quicktime deals with it fine.

Reply

Answer 6

pkdev

Level 1

0 points

Sep 30, 2011 1:26 PM in response to ctzsnooze

I'm glad I came across your post! Just installed Compressor 4 last night and was about to pull my hair out from having the same problems. Just ran a couple of tests and confirmed that sending a job from FCPX fails, while an intermediary export processed directly in compressor succeeds. Has anyone reported this bug to Apple?

Reply

Answer 7

drevil02

Level 1

0 points

Nov 25, 2011 11:43 AM in response to ctzsnooze

Guys.... this is making me crazy ALSO. I have a 12 core primary machine and an 8 core accessory machine. The whole reason I bought the 12 core was to be able to utilize the distributed processing capability of compressor 4 and FCPX and leverage my existing machine. This is a real disappointment as is stands it takes me about 2.5 hours for every hour of 1080p/30 video that I have to encode. While this is not excessive and I have it better than most with Apple's highest performance machine, I would really like to be able to utilize my "old" machine to help with these processor intensive task. Plus the whole idea of having this done over the network is cool.

On using an intermediate file: it takes me longer to create the intermediate 422HQ quicktime .mov file and encode that to the finished BluRay files (264 and ac3) than it does encode that than it does to go straight to a .264 file with ac3.audio... my primary output is BluRay DVDs at 1080p. I could see how the intermediate file would help if you needed to go multiple output formats. However, as described above that process does take over FCPX and if you pull up the activity window it only uses about 15% of the processing capacity.

How do we get apple's attention on this????

Reply

Answer 8

Feb 21, 2012 7:51 PM in response to drevil02

I'm still having this issue and agree also that it's definetly a bug after performing some tests of my own. To get Apple's attention submit feedback for Compressor from the menu bar using "Compressor --> Provide Compressor Feedback."

If you're more interested in a link then here.

Reply

Answer 9

Mar 22, 2012 5:34 AM in response to ctzsnooze

Just adding my 2 cents to say exactly the same problem here. 4 Mac Pro's (Edit Machines) Hooked up to a Qmaster cluster comprising 8 XServes all working drom shared XSan storage and I can send a job straight from FCPX as it times out with error "Media Server application unexpectedly quit".

If I Export a ProRes intermediate from FCPX then throw that at the cluster it always works but sometimes that export can take over an hour if its a long project so it kinda kills the point of the cluster a bit.

Anyway have send feedback, lets wait and see. These sorts of things are not usually high on apples agenda though.

Reply

Answer 10

Mar 26, 2012 11:46 AM in response to MatthewDove

I have been searching for a solution for over a week now. It is really annoying. I cannot think of anything I have changed recently to have caused this. Other than a recent FCP X update.

Reply

Answer 11

Mar 27, 2012 3:49 PM in response to ibBenjamin

Sorry, I don't think anyone's really found a solution. If it was working for you before and now it's not then it probably has something to do with permissions. Unfortunately Apple doesn't really provide anyway to find out what the problem is.

All I can offer is a workaround. Export from FCPX in ProRes to the cluster storage and then run the job in compressor from there. I've been doing that and it's actually a nice way to work as you can submit a bunch of jobs and it's not tying up any local resources on your computer. You can even shut down the computer you submited from and log back in to the cluster later to get the finished files.

Reply

Answer 12

Mar 27, 2012 4:40 PM in response to Haggardly

I was doing that, but then it started working again, then stopped again. Business is good, so I'm unable to go back to that and not be able to use FCP for a few hours. It seems to be solved by making the computer submitting to the server to render as an administrator, and making sure both have equal permissions to each other.

But yes, it's a permissions issue it seems.

Reply

Answer 13

Apr 6, 2012 8:26 AM in response to ibBenjamin

It's still not working. This is the latest FCP X and Compressor as well as all the other latest updates. Submitting from Final Cut Pro X to Compressor would be such a time saver.

Reply

Answer 14

Apr 24, 2013 9:04 AM in response to ibBenjamin

hmmm. Im getting a similar problem, with both compressor and Motion 5. Whenever I try to export other than movie or publish a generator in Motion I get the "media server unexpectedly quit" error message. It was working fine until yesterday, and I cant seem to find out what caused it. I didnt update or anything, so it must be something else than a bug. Permissions or else. It seems this propagated inti compressor and Motion.

Any ideas?

Thanks!

Reply

Answer 15

Apr 24, 2013 9:05 AM in response to ctzsnooze

hmmm. Im getting a similar problem, with both compressor and Motion 5. Whenever I try to export other than movie or publish a generator in Motion I get the "media server unexpectedly quit" error message. It was working fine until yesterday, and I cant seem to find out what caused it. I didnt update or anything, so it must be something else than a bug. Permissions or else. It seems this propagated inti compressor and Motion.

Any ideas?

Thanks!

Reply