Martin Pace

Q: How to set up Qmaster with Terminal in Mavericks

I have a Mac Pro 5,1, 12-core with 24 GB of RAM and am trying to set up Qmaster to fully use my CPUs as a quickcluster. My current Qmasterprefs as indicated in Terminal are listed below and I'm looking for someone who can help me with the Terminal commands needed to be entered to set it up.

 

Martys-Mac-Pro:~ mepace$ qmasterprefs -list

Current settings:

-allowBonjourDiscovery on

-sharingType servicesAndQuickCluster

-launchContentAgent off

-log 3 truncate on

-statusMenu on

Exception: no config file entry for service "jobcontroller:com.apple.qmaster.cluster.admin"

Compressor, OS X Mavericks (10.9)

Posted on Nov 18, 2013 4:19 PM

Close

Q: How to set up Qmaster with Terminal in Mavericks

  • All replies
  • Helpful answers

Previous Page 2
  • by Fred Turner,

    Fred Turner Fred Turner Nov 20, 2013 10:57 AM in response to Martin Pace
    Level 1 (85 points)
    Nov 20, 2013 10:57 AM in response to Martin Pace

    I'm at a bit of a loss at the moment-- it should be working w/ all 12 cores at this point, I think. One thing that throws me is the process name that's handling the crunching for you: compressord. And just a single process at that. On my 8-core Mac Pro running 10.6.8 (sorry, I can't test it w/ Mavericks, it's a 2006), I see 8 processes called "CompressorTranscoderX" in Activity Monitor when I use QuickCluster. When I use "This Computer" instead, I only see 1 process working called "CompressorTranscoder". Checking ps in Terminal shows that they both are running "compressord" processes, but from 2 different framework bundles.

     

    While you are processing, try this in Terminal:

     

    ps -ax | grep Compressor

     

    My guess is you'll see 1 process from the CompressorTranscoder.bundle and 12 from CompressorTranscoderX.bundle. And my further guess is that the process ID of the single process will match the single process you show in Activity Monitor (233 in your latest example). Can you confirm those things for me?

     

    Some more basics to check also:

     

    - After the restart you mentioned, do you still have the Qmaster menu and all 12 Compressor instances showing?

    - Can you post the latest from "qmasterprefs -list" (after the restart)?

    - What does Batch Monitor show? Do you see an entry in the "Martys Mac Pro Cluster" tab that shows the job broken into many parts and processing concurrently?

    - I tried the same preset that you are using on my 8-core Mac Pro (10.6.8), and I only got 7 total job parts when using QuickCluster on a 15-min video, despite getting 20 or so using a different H.264 LAN preset. Perhaps the preset choice is affecting you (although I can't imagine why w/ two H.264 settings it'd be different)...

     

    Fred

  • by Fred Turner,

    Fred Turner Fred Turner Nov 20, 2013 12:07 PM in response to Fred Turner
    Level 1 (85 points)
    Nov 20, 2013 12:07 PM in response to Fred Turner

    Update: I installed the full Compressor on my C2D MacBook Pro running Mavericks; previously I had only installed the Qmaster components. It seems to work right for me, but may perhaps require some tweaking to make it it operate as expected.

     

    In your screenshot, you show "compressord" as the active process w/ some, but not all of your cores being used. This tells us you are NOT making use of the cluster and all instances. On test runs I just performed w/ both the "H.264 LAN" and "AppleTV 5Mbps" presets, my MBP shows 2 "CompressorTranscoderX" processes both running at 95+%. Oddly, though, the AppleTV preset uses fewer job "chunks" when observed via Batch Monitor.

     

    Anyway, I think for some reason, after you restarted, maybe the Cluster wasn't fully functional, or somehow Compressor didn't properly hand off to it.

     

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 20, 2013 1:24 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 20, 2013 1:24 PM in response to Fred Turner

    I was testing with a 2 minute video clip previously. Just plugged in a 2 hour test clip and all cylinders are firing.

     

    Thanks for your help!

     

    Screen Shot 2013-11-20 at 1.18.01 PM.png

    Screen Shot 2013-11-20 at 1.24.27 PM.png

  • by Fred Turner,

    Fred Turner Fred Turner Nov 20, 2013 1:31 PM in response to Martin Pace
    Level 1 (85 points)
    Nov 20, 2013 1:31 PM in response to Martin Pace

    Fantastic! Glad we got that all sorted out. It may require some care and feeding at times w/ those Terminal commands to get it to cooperate, but the improvement in compression speed should be well worth it.

     

    Interesting to note that 4 instances show as responding while the other 8 show as not responding. I wonder if that changes over the course of the batch. Also interesting that you're definitely getting some use out of your HyperThreading there, w/ 11 of your 12 of your instances getting well over 100%. Not sure if it would help you (may, in fact, hurt the results), but it might be fun to try the "-service" line for Compressor w/ >12 instances. :-D

     

    Keep us posted!

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 20, 2013 1:40 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 20, 2013 1:40 PM in response to Fred Turner

    Fred Turner wrote:

     

    Interesting to note that 4 instances show as responding while the other 8 show as not responding. I wonder if that changes over the course of the batch. Also interesting that you're definitely getting some use out of your HyperThreading there, w/ 11 of your 12 of your instances getting well over 100%. Not sure if it would help you (may, in fact, hurt the results), but it might be fun to try the "-service" line for Compressor w/ >12 instances. :-D

     

    Keep us posted!

    FT

    Eventually all of the instances show not responding but the % CPU still changes.

     

    The manual recommends 1 instance for every 2 GB of installed memory, since I just happen to have 24 GB 12 instances makes sense. Activity Monitor shows that only 10.7 GB are being used though. CPU usage seems to be increasing as the job progresses. Fairly steady in the mid 80% range for my user right now, even saw 96% briefly so I don't think I need to up the # of instances.

  • by Berend de Meyer,

    Berend de Meyer Berend de Meyer Apr 6, 2014 1:34 PM in response to Martin Pace
    Level 1 (14 points)
    Mac OS X
    Apr 6, 2014 1:34 PM in response to Martin Pace

    Thanks for this GREAT thread Martin & Fred! Just moved to Mavericks, so your thread came to the rescue for my Qmaster issue.

     

    Cheers

  • by Berend de Meyer,

    Berend de Meyer Berend de Meyer Apr 11, 2014 1:56 AM in response to Fred Turner
    Level 1 (14 points)
    Mac OS X
    Apr 11, 2014 1:56 AM in response to Fred Turner

    -deleted

  • by Berend de Meyer,

    Berend de Meyer Berend de Meyer Apr 11, 2014 7:35 AM in response to Fred Turner
    Level 1 (14 points)
    Mac OS X
    Apr 11, 2014 7:35 AM in response to Fred Turner

    ... and a little Terminal Shell Script copied into the auto-login prefs pane:

     

    #!/bin/bash

    qmasterprefs -service "Compressor Processing" on instances 8

    qmasterprefs -cluster on quickclusterservername "MACPRO"

    qmasterprefs -statusMenu off

    qmasterprefs -statusMenu on

     

    Thanks again. Cheers

  • by Berend de Meyer,

    Berend de Meyer Berend de Meyer Apr 15, 2014 8:37 AM in response to Berend de Meyer
    Level 1 (14 points)
    Mac OS X
    Apr 15, 2014 8:37 AM in response to Berend de Meyer

    Updated the script for reset cluster when wake-up after sleep!

     

    #!/bin/bash

    qmasterprefs -resetCluster

    qmasterprefs -service "Compressor Processing" on instances 8

    qmasterprefs -cluster on quickclusterservername "MACPRO"

    qmasterprefs -statusMenu off

    qmasterprefs -statusMenu on

  • by Maria A Martin,

    Maria A Martin Maria A Martin Apr 22, 2014 12:13 AM in response to Martin Pace
    Level 1 (0 points)
    Apr 22, 2014 12:13 AM in response to Martin Pace

    You have to be careful when copying and pasting formatted text into the  Terminal window. It looks like the speech marks may have been fancy  "curly" ones that the Terminal doesn't recognize. I'd recommend typing  the command out manually just to be sure.

  • by Berend de Meyer,

    Berend de Meyer Berend de Meyer Apr 22, 2014 12:33 AM in response to Maria A Martin
    Level 1 (14 points)
    Mac OS X
    Apr 22, 2014 12:33 AM in response to Maria A Martin

    Not sure I fully understand what you mean by formatted text?

  • by hope2win,

    hope2win hope2win Oct 1, 2014 2:53 AM in response to Fred Turner
    Level 1 (0 points)
    Oct 1, 2014 2:53 AM in response to Fred Turner

    Hi Fred

     

    I have been using this thread to try and get Compressor to work with Mavericks, and learning a lot about Terminal while doing so.

     

    But no matter what I try I cannot get things to work - every time I tweak the settings then try and submit a job via Compressor, the button is greyed out.  I just cannot get a Cluster to appear as an option.

     

    I've struggled with Batch Monitor, QAdministrator and am about to give up but thought I would see if you could help in any way at all.

     

    I have not posted any details as I didn't want to be presumptious or make demands on your time.

     

    But thank you for reading this...

     

    Peter

Previous Page 2