Martin Pace

Q: How to set up Qmaster with Terminal in Mavericks

I have a Mac Pro 5,1, 12-core with 24 GB of RAM and am trying to set up Qmaster to fully use my CPUs as a quickcluster. My current Qmasterprefs as indicated in Terminal are listed below and I'm looking for someone who can help me with the Terminal commands needed to be entered to set it up.

 

Martys-Mac-Pro:~ mepace$ qmasterprefs -list

Current settings:

-allowBonjourDiscovery on

-sharingType servicesAndQuickCluster

-launchContentAgent off

-log 3 truncate on

-statusMenu on

Exception: no config file entry for service "jobcontroller:com.apple.qmaster.cluster.admin"

Compressor, OS X Mavericks (10.9)

Posted on Nov 18, 2013 4:19 PM

Close

Q: How to set up Qmaster with Terminal in Mavericks

  • All replies
  • Helpful answers

Page 1 Next
  • by Fred Turner,

    Fred Turner Fred Turner Nov 18, 2013 8:22 PM in response to Martin Pace
    Level 1 (85 points)
    Nov 18, 2013 8:22 PM in response to Martin Pace

    Hey Marty--

     

    That last line indicates that we have a file or part of a file missing. Can you look to see if you have this file:

     

    /Library/Application Support/Apple Qmaster/qmasterservices.plist

     

    If so, please post its contents here.

     

    Thx,

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 9:19 AM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 9:19 AM in response to Fred Turner

    It's there.

     

    Screen Shot 2013-11-18 at 11.10.18 PM.png

    I think the file I'm missing is jobcontroller/com.apple.qmaster.cluster.admin.plist, it's in my Time Machine backup from when I was messing around with settings. I think that once we set up a cluster a new file should be generated.

  • by Fred Turner,

    Fred Turner Fred Turner Nov 19, 2013 9:25 AM in response to Martin Pace
    Level 1 (85 points)
    Nov 19, 2013 9:25 AM in response to Martin Pace

    Hmmm...I'm not so sure about that. On my test machine, I didn't do anything like that (setting up a cluster on it), and it had that file. It's only a .plist file, so I think I'd restore it from your Time Machine backups and run the "-list" option again, then post that.

     

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 10:21 AM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 10:21 AM in response to Fred Turner

    Here's what I get after I restore the file.

     

    Martys-Mac-Pro:~ mepace$ qmasterprefs -list

    Current settings:

    -allowBonjourDiscovery on

    -sharingType servicesAndQuickCluster

    -launchContentAgent off

    -log 3 truncate on

    -statusMenu on

    -cluster off autorestart off servername "Marty’s Mac Pro" maxactivetargets 40 maxactivesegments 400 storagepath "/var/spool/qmaster" privatestorage off publishedstorage on storagecleanupthreshold 7 unmanagedservices on unmanagedmulticapturethreshold 0 networkinterface allinterfaces log 3 truncate on

        -service "Compressor Processing" off sharing on instances 1 autorestart off unmanaged on log 3 truncate on

        -service "Rendering" off sharing on instances -1 autorestart off unmanaged on log 3 truncate on

  • by Fred Turner,Helpful

    Fred Turner Fred Turner Nov 19, 2013 11:30 AM in response to Martin Pace
    Level 1 (85 points)
    Nov 19, 2013 11:30 AM in response to Martin Pace

    Hey Marty--

     

    Looks like the "sharingType" flag is set correctly. "statusMenu" is also on, so that's good...hopefully you see it in the menu bar. If not, type these 2 commands back-to-back:

     

    qmasterprefs -statusMenu off
    qmasterprefs -statusMenu on
    

     

    Next, let's adjust the Compressor settings (assuming you want 12 instances, 1 per core):

     

    qmasterprefs -service "Compressor Processing" on instances 12
    

     

    The other settings for that service should be retained, even though you didn't type them all out. Finally, we turn on the cluster:

     

    qmasterprefs -cluster on

     

    Shortly after typing these commands, the Qmaster menu should show 12 Compressor services ready to go, as well as the QuickCluster. Give Compressor a try now, and you should see the "Marty's Mac Pro" QuickCluster available as an option after you submit. If not, or if you run into errors, post any errors and changes to "qmasterprefs -list".

     

    HTH!

    Fred

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 11:56 AM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 11:56 AM in response to Fred Turner

    I do see 12 Compressor services. When I submit a job I see this:

     

    Screen Shot 2013-11-19 at 11.41.53 AM.png

     

    And I'm still not getting full usage of my 12 cores. In  Activity Monitor I see:

     

    Screen Shot 2013-11-19 at 11.40.44 AM.png

    This is essentially identicle to what I was getting before we started. Others seem to be able to get significantly higher CPU usage and faster compression.

     

    Here's what I ended up with in qmastersprefs:

     

    Martys-Mac-Pro:~ mepace$ qmasterprefs -list

    Current settings:

    -allowBonjourDiscovery on

    -sharingType servicesAndQuickCluster

    -launchContentAgent off

    -log 3 truncate on

    -statusMenu on

    -cluster on autorestart off servername "Marty’s Mac Pro" maxactivetargets 40 maxactivesegments 400 storagepath "/var/spool/qmaster" privatestorage off publishedstorage on storagecleanupthreshold 7 unmanagedservices on unmanagedmulticapturethreshold 0 networkinterface allinterfaces log 3 truncate on

        -service "Compressor Processing" on sharing on instances 12 autorestart off unmanaged on log 3 truncate on

        -service "Rendering" off sharing on instances -1 autorestart off unmanaged on log 3 truncate on

  • by Fred Turner,Solvedanswer

    Fred Turner Fred Turner Nov 19, 2013 11:57 AM in response to Fred Turner
    Level 1 (85 points)
    Nov 19, 2013 11:57 AM in response to Fred Turner

    Tested some more, and found that I had to do a little more for QuickClustering (I had only set my test system up as a node). We'll need to set the QuickCluster name, so change the last part above to:

     

    qmasterprefs -cluster on quickclusterservername "Martys Mac Pro Cluster"

     

    If you don't see that cluster name in your Qmaster menu, try this too:

     

    qmasterprefs -resetCluster

     

    Let me know how it goes!

     

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 12:05 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 12:05 PM in response to Fred Turner

    Here's my current settings:

    Martys-Mac-Pro:~ mepace$ qmasterprefs -list

    Current settings:

    -allowBonjourDiscovery on

    -sharingType servicesAndQuickCluster

    -launchContentAgent off

    -log 3 truncate on

    -statusMenu off

    -cluster on autorestart off servername "Marty’s Mac Pro" quickclusterservername "Martys Mac Pro Cluster" maxactivetargets 40 maxactivesegments 400 storagepath "/var/spool/qmaster" privatestorage off publishedstorage on storagecleanupthreshold 7 unmanagedservices on unmanagedmulticapturethreshold 0 networkinterface allinterfaces log 3 truncate on

        -service "Compressor Processing" on sharing on instances 12 autorestart off unmanaged on log 3 truncate on

        -service "Rendering" off sharing on instances -1 autorestart off unmanaged on log 3 truncate on

     

    Unfortunately I'm still seeing the same performance. I also tried a

    sudo qmasterprefs -restart

    and did not see any change in performance.

  • by Fred Turner,Helpful

    Fred Turner Fred Turner Nov 19, 2013 12:08 PM in response to Martin Pace
    Level 1 (85 points)
    Nov 19, 2013 12:08 PM in response to Martin Pace

    Do you see the Cluster now in either the Qmaster menu or when you go to submit? If not, try the command:

     

    qmasterprefs -resetCluster

     

    It'll take a few seconds, but you should see all 12 instances and the cluster in the Qmaster menu.

     

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 12:12 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 12:12 PM in response to Fred Turner

    No, I don't. I see:

     

    Screen Shot 2013-11-19 at 12.09.19 PM.png

    Screen Shot 2013-11-19 at 11.41.53 AM.png

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 12:28 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 12:28 PM in response to Fred Turner

    I restarted my computer and the cluster now shows up.

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 12:48 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 12:48 PM in response to Fred Turner

    Submitting jobs to the cluster still gives about the same performance.

     

    Screen Shot 2013-11-19 at 12.34.43 PM.png

    Screen Shot 2013-11-19 at 12.43.58 PM.png

     

    This is what has frustrated me in the past even when the Qmaster preference pane was available, I would set everything as described and see no change in performance.

  • by Fred Turner,

    Fred Turner Fred Turner Nov 19, 2013 1:24 PM in response to Martin Pace
    Level 1 (85 points)
    Nov 19, 2013 1:24 PM in response to Martin Pace

    Hmmm...looks like we've made progress, but not quite yet able to utilize it! Can you describe your workflow? Are you dumping straight from FCP to Compressor, or are you saving an intermediate reference file? What type of compression are you applying and what format is the source?

     

    FT

  • by Martin Pace,

    Martin Pace Martin Pace Nov 19, 2013 1:33 PM in response to Fred Turner
    Level 5 (5,118 points)
    Nov 19, 2013 1:33 PM in response to Fred Turner

    I typically capture video using Blackmagic's Media Express (via a Blackmagic Intensity Pro PCIe card) to hard disk. Source is using QuickTime Apple ProRes 422 HQ in an HD 720p 59.94 format.

     

    I'll edit the video using either Quicktime Player 7 (if only minor trimming is required) or FCP 7. I save the changes to disk.

     

    Then I drag the video file from the location in Finder to my Compressor 3.5 icon in the Dock. Compressor 3.5 is set with default presets for Apple TV (H.264 for Apple TV @ 5 Mbps). Then I submit the video for compression.

Page 1 Next