5 Replies Latest reply: Jan 28, 2013 2:01 PM by achourasia
achourasia Level 1 Level 1 (0 points)

Hello,

 

I have couple of questions for compressor (ver 4 )

 

1) Is there a way to specify framerate for image sequence using command line interface.

The compressor help lists the following but does not provide a way how such an option work. I tried few different ways in vain

 

-jobpath  -- url to source file.                        -- In case of Image Sequence, URL should be a file URL pointing to directory with image sequence.

-- Additional parameters may be specified to set frameRate (e.g. frameRate=29.97) and audio file (e.g. audio=/usr/me/myaudiofile.mov).

 

 

2) I have a managed cluster with 8 nodes each running 4 instances. For some reason compressor only uses 6 instances spread across many nodes and not all 32.

 

 

3) Is there a way to specify and process just one job for a given cluster? This is equivalent of serialize option but it does not seem to be available.

Currently when I submit multiple jobs few of them run at once and create a havok on NFS mounts and fail. I can limit the job queue to one but that is not ideal.

 

I would appreciate any pointers.

 

Thanks

--Amit


Compressor, OS X Mountain Lion (10.8.2)
  • Warwick Teale Level 3 Level 3 (580 points)

    Hi Amit, just saw your post. I assume you are passing the settings via the "-settingspath" <my_specific_job_settings_in_a_file_path_somwhere.settings" on the compressor command?

     

    If so, it's a very simple matter to specify the frame rate etc etc on the VIDEO setting itself, save it and use it.

     

    I don't recall that such "atomic settings" we're actually available in the v3 of compressor.app. I'll check later for v4. I'd be surprised  if they are. :)

     

    What I've done in the past is to simply make my own personalised settings (and destinations) up using  the Compressor.app V4 UI (save as ... my name .. i.e.  prores_noaudio_PAL" and path these file paths on the -settingspath" parm on the compressor cli. In your case I'd imagine a simple VIDEO setting for your frame rate and you're set!

     

    Compressor.app v4 makes any of your own R.Y.O settings on your ~/library/application support/compressor/settings folder. You can copy or move or link these anywhere you like and pass these customised Parms to compressor cli as above.

     

    I doubt also if these atomic settings are available as AppleScript properties, there's likely no vars like that there me thinks. . I recall the same ones exist as they do for the cli and now in compressor.app 4 they probably support Qmaster .. Yeah.. Compressor.app is apple scriptable.. It's not in the default list library so just add it...

     

    Lastly as a guess these compressor ".setting" files are probably XML base.. So you might consider to tinker with one with an editor.

     

    Anyway.. Try then"-settingspath" operand and see how u go.

     

    2) the way Qmaster schedules across unmanaged transcode nodes is likely based on how busy each node is. You should be able to see if there is a pattern simply by looking on the Qmaster job controller.log .. See share monitor.app info or use console.app look on

    your ~/library/application support/apple Qmaster/logs directory. There will be something in there.

     

    Also have a look for any errors on the cluster services logs in case the services you expects to be there are actually not.

     

    Are you using a managed cluster? Personally I have found this quite stable. Make sure u insure that those services are for managed climates only..

     

    3) yes you can specify a specific cluster on the using the "-clusterid" operand. Should you have more than one managed cluster in the farm, this is a cool way to go. Also considering the "-priority" operand usage as well for hot jobs. Make sure all submissions are low priority... It's batch.. Works great!!

     

    4) NDS mounts.. Well the simple rule is to keep them mounted on their own subnet, maker sure all destinations and sources are available to all hosts, set compressor options to copy to cluster only when ya must, and make sure the service time for NDS io READ Request is as fast as u can make it. ( jumbo frames, dedicated NICs and sinners and optimum fast read file systems... Keep other traffic away for it.. Works a treat!

     

    Should you turn something up pleased at it here for others to see. I'm certainly interested.

     

    Sorry for typos.. Just on MTR with iPhone on way home. :)

     

    Hth

     

    Warwick

    Hong Kong

  • Warwick Teale Level 3 Level 3 (580 points)

    Hi Amit, yeah I just had a dig around. Regarding the compressor.app personalised settings, it is XML based. The frame rate operate u might wanna chase  for frame rate is unsurprisingly "frame-rate"

     

    So say a frame rate of 25fps would be specified in the video conversion section  as "<frame-rate>R25/1</frame-rate>"

     

    Cheers mate.

    Warwick

  • achourasia Level 1 Level 1 (0 points)

    Hi Warwick,

    Thanks for your reponse.

     

    1) Yes I can change the frame rate in settings file either by editing XML or creating new one through GUI. This is not what I am after, I am looking for a more automated/scriptable way to set this for image sequences. Command line seems to provide an option for frameRate and -info none of which are further documented

     

    2) I am using a managed cluster ( 8 machines each with 4 instances of compressor, so total 32 compressor instances). Everything looks fine, I can see all 32 instances which are idle. When I submit a job only 4-6 instances are used the rest remain idle. When more than one job is submitted other instances are used, so the cluster is set fine its just compressor is somehow automatically deciding how many instances to use.

     

    3) I would like only one job to be processed by the cluster at a time, there is no predictable way to set this.

    a) I have tried setting  "Apple Qmaster > Share This Computer > Advanced> Maximum Active Targets> 1" which does not help

    b) I have tried setting the priority to low which does hold few jobs but not all.

     

    4) NFS and other network settings are set properly.

     

    5) Is there a programmatic way to create setting XML files?

  • Warwick Teale Level 3 Level 3 (580 points)

    Hi Amit, thanks for your reply. Yeah I appreciate what you are attempting to go. I recant that I'm unaware of an obvious method to dynamically pass any "framerate" parm value to the cli for compressor.app

     

    I would however state that it is VERY plausible to dynamically construct strings, values, parm files and any goodies programmatically. This as usual is a decision of your choosing. For me if I need to do a particular workflow many times, I will decide if I must automate it somehow and the subsequent trade off of supporting it , developing enhancements and the rigourous testing if it is to be used by others

     

    Your call :)

     

    In this case, I'd opt for simplicity of APPLESCRIPT or any UI or service on osx , any text manipulation, finder, compressor and file object manufacture... Speed is not a requirement as long as your workflow is automated I imagine

     

    I'd do this over messing with PERL or even UNIX bash/shell scripting because you can at least AppleScript compressor.app and for that matter the additional Qmaster stuff

     

    Now on this order:

    5) yes there sure are many XML creation apps and even within osx. I'm assuming here you'd want to build the compressor setting dynamically.  Sure you can

     

    4) NFS mounts I mentioned this because others may have a setup consideration and this is a weak link in the service transcode time when distributed.

     

    2) in regards to the algorithm and conditions deemed by Qmaster to schedule a batch over the available node services, I am not entirely convinced that its too sensitive to the actual work on a particular host, merely what it thinks it is running or has scheduled on those service nodes. I doubt it considers any load balancing other that trying to equalise work over all nodes in a cluster and their services

     

    As you are using managed cluster , why not try the TICK BOX  in the Qmaster administrator for that cluster to utilise also any "unmanaged services" . Them test a node with some services that are unmanaged and try from there? I'd be curious to know.

     

    3) targets = 1 doesn't work eh? I'll try this over the weekend.  Good idea. Of not you could again programmatically build a simple batch scheduler.  Yep now we are back to coding again :(

     

    I'll get back to I over weekend

     

    Hrh

    Warwick

  • achourasia Level 1 Level 1 (0 points)

    An update to the above discussion. Compressor sets the segmentation automatically for different codecs. There does not appear to be a way to set them in an explicit way.