0 Replies Latest reply: Jan 28, 2013 1:58 PM by achourasia
achourasia Level 1 Level 1 (0 points)

Folks,

 

I am curious to hear about your experience with compressor's managed cluster.

  Here is my experience with a setup that consists of

a) 8 quadcore mac mini, with Mountain Lion and Compressor 4.06

b) NFS storage connected with 1gb link to each mac mini

c) Compressor preference set to never copy to/from cluster storage

 

******************************************************************************** ******************************************************************************** ************

Experiment 1: One managed cluster with 4 compressor service instances on eight mac mini. So a total of 32 compressor instances.

The instances are set to send an email when they go down.

Source: 4K Image sequence with 1850 images

Target: Apple ProRes 422 (default)

Output: NFS storage

 

Observations:

1) The target codec determines the number of segments the job gets broken down into. For example the HD 1080p setting will break the job into atmost 6 segments (regardless of 32 instances that I have available). In case of ProRes 422 the number of segment were 64 ie 2x more than the number of instances available.

 

2) The compressors service instances are extremely temperamental and flimsy. I got over 100 error notification for this experimental job, about a dozen instances of compressor were in bad state/dead and the job failed with many errors as follows.

 

The error seems to indicate an issue with file naming, but I am unsure what it points towards

 

1/26/13 7:03:54.815 PM compressord[12399]: *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSPlaceholderString initWithFormat:locale:arguments:]: nil argument'

*** Call stack at first throw:

(

          0   CoreFoundation                      0x982c712b __raiseError + 219

          1   libobjc.A.dylib                     0x92eea52e objc_exception_throw + 230

          2   CoreFoundation                      0x98226bbb +[NSException raise:format:] + 139

          3   Foundation                          0x93885d02 -[NSPlaceholderString initWithFormat:locale:arguments:] + 99

          4   Foundation                          0x93886e8c +[NSString stringWithFormat:] + 88

          5   AECore                              0x000805da -[ImageSequenceMedia fileURLForFrame:] + 399

          6   AECore                              0x00082908 -[ImageSequenceMedia readImageInfo] + 118

          7   AECore                              0x00080968 -[ImageSequenceMedia isNativePixelFormat:] + 34

          8   StompTypes                          0x0032b791 _ZNK27CExtendedImageSequenceMedia19isNativePixelFormatEi + 47

          9   StompTypes                          0x003202d1 _ZN5stomp13IMediaDecoder20IsDecodablePixelTypeEiRK7CURLRefRKNS_18CFilterSetting sRefE + 71

          10  Transcoding                         0x004c4ac4 _ZN5stomp17CQTSpitTranscoder16optimalPixelTypeERKNS_18CFilterSettingsRefE + 856

          11  Transcoding                         0x004c5ca3 _ZN5stomp22CQTSpitMovieTranscoder22configureSpitComponentEm + 761

          12  Transcoding                         0x004c38eb _ZN5stomp17CQTSpitTranscoder3runEv + 327

          13  Transcoding                         0x004a8498 _ZN5stomp16CStompTranscoder15_processRequestERKN5swamp18IServiceRequestRefE + 3740

          14  ServiceControl                      0x00cbde00 _ZN5swamp24CServiceControllerThread7runOnceEv + 2068

          15  AECore                              0x00023364 _ZN6aecore7CThread9runThreadEv + 36

          16  AECore                              0x00020ffc _Z10threadProcPv + 17

          17  libsystem_c.dylib                   0x95b5d557 _pthread_start + 344

          18  libsystem_c.dylib                   0x95b47cee thread_start + 34

)

******************************************************************************** ******************************************************************************** ************ Experiment 2: Experiment 2: One managed cluster with 2 compressor service instances on seven mac mini. The master controller is not running any compressor instances its just taking care of house keeping. So a total of 14 compressor instances.

Source: 4K Image sequence with 1850 images

Target: Apple ProRes 422 (default)

Output: NFS storage

 

Observations: job failed, two dead compressor instances and about 20 error notifications of instance down

******************************************************************************** ******************************************************************************** ************

 

Conclusions:

Overall, the jobs which transcode movies from one format to another have a good chance of success. However the same is not true for Image sequences.

The distributed compressor instances are extremely flaky and their revival scheme of four times(as per documentation) is not enough.

 

 

I am eager to know your experience with distributed encoding using compressor.

a) What are your thoughts on stability of distributed encoding using compressor for a large number of jobs

b) Is there an automatic way to revive dead compressor instances?

c) What other tools are out there which perform better? I am looking for a way to setup unmanned operation.

 

Cheers

-A


Compressor, OS X Mountain Lion (10.8.2)