Steffen Bendix

Q: How to share a job with Compressor 4.1?

Can anyone explain how to set up Compressor on two or more computers to share a coding job? I was never successful, neither with the old versions nor the new one. I have connected two computers running Mavericks via Ethernet. They appear in the preferences list of Compressor as inactive and can be selected (with a tick). Starting a job produces no error. Only the little network monitor window shows some activity: "inactive" (in white or yellow), sometimes: "not found" (in red). The computer which sends the job waits endlessly.

 

I deactivated the firewall, connected the computer with DHCP or fixed IP but no success. What else do I have to do?

iMac (27-inch, Late 2012), OS X Mountain Lion (10.8.4), 16 GB, Nvidia GeForce GTX 680MX

Posted on Dec 19, 2013 3:23 PM

Close

Q: How to share a job with Compressor 4.1?

  • All replies
  • Helpful answers

first Previous Page 3 of 3
  • by Warwick Teale,

    Warwick Teale Warwick Teale Feb 6, 2014 2:49 AM in response to Steffen Bendix
    Level 3 (588 points)
    Feb 6, 2014 2:49 AM in response to Steffen Bendix

    Hi Steffen, regarding your most recent post:

     

    Steffen Bendix wrote:

    .....e but when the last piece is almost finished (approx. 80%) the last computer finishes its work. The final movie can't be completed by the master computer. The individual work pieces remain uncombined.

    It's likey that access to the SEGMENTS from one host (the master or the slave) has been removed

     

    If you are able to re-run the job again and check the following ON BOTH Hosts (master and slave 1.1.1.1. and 1.1.1.2):

    1. the (console/compressor) ~/logs/compressor/servicecontroller/com.apple.stomp.transcoderxx.logs - there will be an error in here. Might be QT or encoding error.
    2. check the system.log too for any errors. Look for things like:
      • spinlock smashed, error AFP or NFS mount failure, loss of path,, NIC error (error on ETHERNET ) etc ec
    3. also check these places for the assempled segments:
      • /users/shared/Library/application support/compressor/storage/xxxxxx-xxxxx/ for partial segments
      • (your home's) ~/Library/application support/compressor/storage/xxxxxxx-xxxxx/Jobs and look for segments if they exist

     

    The error or resource issue will be in here somewhere.

     

    Post your reults for others to see.

     

    W

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Feb 7, 2014 12:50 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Feb 7, 2014 12:50 AM in response to Warwick Teale

    On the master 1.1.1.1 the last chunk out of 4 has been finished but Share Monitor reports that the computer has finished only 55%. The stomp.log does not show any errors but the system.log has many. Here is an extract:

     

    Feb  7 09:22:18 Idefix.local mDNSResponder[44]: SSLHandshake failed: -9806

    Feb  7 09:22:18 Idefix.local mDNSResponder[44]: tcpCallback: stream connection for LLQ _t2FCB9F2B04DB._tcp.12140584.members.btmm.icloud.com. (PTR) failed 7 times, retrying in 900000 ms

    Feb  7 09:22:18 Idefix.local mDNSResponder[44]: SSLHandshake failed: -9806

    Feb  7 09:22:18 Idefix.local mDNSResponder[44]: tcpCallback: stream connection for LLQ _qmp6._tcp.12140584.members.btmm.icloud.com. (PTR) failed 7 times, retrying in 900000 ms

    Feb  7 09:22:54 Idefix.local compressord[2875]: CGSGetWindowType: Invalid (NULL) window

    Feb  7 09:22:54 Idefix.local compressord[2875]: CGSGetWindowResolution: Invalid window 0x0

    Feb  7 09:22:54 Idefix.local compressord[2875]: CGSGetWindowDepth: Invalid window

    Feb  7 09:22:54 Idefix.local compressord[2875]: CGSLockWindowRectBits: Invalid window 0x0

    Feb  7 09:22:54 Idefix.local compressord[2875]: CGSUnlockWindowBits: Invalid window 0x0

    Feb  7 09:23:00 Idefix.local compressord[2875]: CGSGetWindowType: Invalid (NULL) window

    Feb  7 09:23:00 Idefix.local compressord[2875]: CGSGetWindowResolution: Invalid window 0x0

    Feb  7 09:23:00 Idefix.local compressord[2875]: CGSGetWindowDepth: Invalid window

    Feb  7 09:23:00 Idefix.local compressord[2875]: CGSLockWindowRectBits: Invalid window 0x0

    Feb  7 09:23:00 Idefix.local compressord[2875]: CGSUnlockWindowBits: Invalid window 0x0

    Feb  7 09:23:13 Idefix.local com.apple.IconServicesAgent[543]: main Failed to composit image for binding VariantBinding [0x5d3] flags: 0x8 binding: FileInfoBinding [0x3f5] - extension: m4v-4, UTI: dyn.age804rd0fy4a, fileType: MooV.

    Feb  7 09:23:13 Idefix.local quicklookd[7273]: Warning: Cache image returned by the server has size range covering all valid image sizes. Binding: VariantBinding [0x203] flags: 0x8 binding: FileInfoBinding [0x103] - extension: m4v-4, UTI: dyn.age804rd0fy4a, fileType: MooV request size:16 scale: 1

    Feb  7 09:29:13 Idefix.local WindowServer[263]: MPAccessSurfaceForDisplayDevice: Set up page flip mode on display 0x042c0140 device: 0x7fcee1d17b20  isBackBuffered: 1 numComp: 3 numDisp: 3

    Feb  7 09:29:13 Idefix.local WindowServer[263]: Display 0x042c0140: GL mask 0x1; bounds (0, 0)[800 x 600], 60 modes available

              Main, Active, on-line, enabled, built-in, boot, captured, OpenGL-accel, Vendor 610, Model b005, S/N 0, Unit 0, Rotation 0

              UUID 0x1cbf8fef38ae9dfec93508341d780c47, ColorProfile { 2, "iMac"}

    Feb  7 09:29:13 Idefix.local WindowServer[263]: GLCompositor: GL renderer id 0x01022724, GL mask 0x0000000f, accelerator 0x0000449b, unit 0, caps QEX|MIPMAP, vram 2048 MB

              texture max 16384, viewport max {16384, 16384}, extensions FPRG|NPOT|GLSL|FLOAT

    Feb  7 09:29:13 Idefix.local WindowServer[263]: Display 0x003f003f: GL mask 0x8; bounds (1824, 0)[1 x 1], 2 modes available

              off-line, enabled, OpenGL-accel, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 3, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:13 Idefix.local WindowServer[263]: GLCompositor: GL renderer id 0x01022724, GL mask 0x0000000f, accelerator 0x0000449b, unit 0, caps QEX|MIPMAP, vram 2048 MB

              texture max 16384, viewport max {16384, 16384}, extensions FPRG|NPOT|GLSL|FLOAT

    Feb  7 09:29:13 Idefix.local WindowServer[263]: Display 0x003f003e: GL mask 0x4; bounds (1825, 0)[1 x 1], 1 modes available

              off-line, enabled, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 2, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:13 Idefix.local WindowServer[263]: Display 0x003f003d: GL mask 0x2; bounds (1826, 0)[1 x 1], 1 modes available

              off-line, enabled, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 1, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x042c0140: Unit 0: Mode 2560 x 1440, CGSThirtytwoBitColor, Resolution 1, ioModeID 0x80001000, ioModeDepth 0x2, IOReturn 0x0

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display mode changed

    Feb  7 09:29:14 Idefix.local WindowServer[263]: MPAccessSurfaceForDisplayDevice: Set up page flip mode on display 0x042c0140 device: 0x7fcee1d17b20  isBackBuffered: 1 numComp: 3 numDisp: 3

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x042c0140: GL mask 0x1; bounds (0, 0)[2560 x 1440], 60 modes available

              Main, Active, on-line, enabled, built-in, boot, captured, OpenGL-accel, Vendor 610, Model b005, S/N 0, Unit 0, Rotation 0

              UUID 0x1cbf8fef38ae9dfec93508341d780c47, ColorProfile { 2, "iMac"}

    Feb  7 09:29:14 Idefix.local WindowServer[263]: GLCompositor: GL renderer id 0x01022724, GL mask 0x0000000f, accelerator 0x0000449b, unit 0, caps QEX|MIPMAP, vram 2048 MB

              texture max 16384, viewport max {16384, 16384}, extensions FPRG|NPOT|GLSL|FLOAT

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x003f003f: GL mask 0x8; bounds (3584, 0)[1 x 1], 2 modes available

              off-line, enabled, OpenGL-accel, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 3, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:14 Idefix.local WindowServer[263]: GLCompositor: GL renderer id 0x01022724, GL mask 0x0000000f, accelerator 0x0000449b, unit 0, caps QEX|MIPMAP, vram 2048 MB

              texture max 16384, viewport max {16384, 16384}, extensions FPRG|NPOT|GLSL|FLOAT

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x003f003e: GL mask 0x4; bounds (3585, 0)[1 x 1], 1 modes available

              off-line, enabled, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 2, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x003f003d: GL mask 0x2; bounds (3586, 0)[1 x 1], 1 modes available

              off-line, enabled, Vendor ffffffff, Model ffffffff, S/N ffffffff, Unit 1, Rotation 0

              UUID 0xffffffffffffffffffffffffffffffff

    Feb  7 09:29:14 Idefix.local WindowServer[263]: Display 0x42c0140 released by conn 0x1609b

    Feb  7 09:29:16 Idefix.local WindowServer[263]: **DMPROXY** (2) Found `/System/Library/CoreServices/DMProxy'.

    Feb  7 09:29:16 Idefix.local WindowServer[263]: Display 0x042c0140: Unit 0; ColorProfile { 2, "iMac"}; TransferFormula (1.000000, 1.000000, 1.000000)

    Feb  7 09:29:46 --- last message repeated 1 time ---

    Feb  7 09:30:02 Idefix.local rpc.statd[32]: Failed to contact host idefix.local: RPC: Port mapper failure - RPC: Timed out

    Feb  7 09:30:02 Idefix.local rpc.statd[32]: Failed to contact rpc.statd at host idefix.local

     

     

    The slave 1.1.1.2 has no errors in stomp either and these in the system.log:

     

    Feb  7 09:16:00 Friedward.local rpc.statd[27]: Failed to contact host idefix.local: RPC: Port mapper failure - RPC: Timed out

    Feb  7 09:16:00 Friedward.local rpc.statd[27]: Failed to contact rpc.statd at host idefix.local

    Feb  7 09:16:18 Friedward.local com.apple.IconServicesAgent[670]: main Failed to composit image for binding VariantBinding [0x5db] flags: 0x8 binding: FileInfoBinding [0x3e7] - extension: m4v-3, UTI: dyn.age804rd0fy3u, fileType: MooV.

    Feb  7 09:16:18 Friedward.local quicklookd[6551]: Warning: Cache image returned by the server has size range covering all valid image sizes. Binding: VariantBinding [0x203] flags: 0x8 binding: FileInfoBinding [0x103] - extension: m4v-3, UTI: dyn.age804rd0fy3u, fileType: MooV request size:16 scale: 1

    Feb  7 09:16:55 Friedward.local com.apple.IconServicesAgent[670]: main Failed to composit image for binding VariantBinding [0x4e3] flags: 0x8 binding: FileInfoBinding [0x1ef] - extension: m4v-2, UTI: dyn.age804rd0fy3a, fileType: MooV.

    Feb  7 09:16:55 Friedward.local quicklookd[6551]: Warning: Cache image returned by the server has size range covering all valid image sizes. Binding: VariantBinding [0x403] flags: 0x8 binding: FileInfoBinding [0x303] - extension: m4v-2, UTI: dyn.age804rd0fy3a, fileType: MooV request size:16 scale: 1

    Feb  7 09:23:23 Friedward.local com.apple.IconServicesAgent[670]: main Failed to composit image for binding VariantBinding [0x5dd] flags: 0x8 binding: FileInfoBinding [0x2df] - extension: m4v-4, UTI: dyn.age804rd0fy4a, fileType: MooV.

    Feb  7 09:23:23 Friedward.local quicklookd[7555]: Warning: Cache image returned by the server has size range covering all valid image sizes. Binding: VariantBinding [0x203] flags: 0x8 binding: FileInfoBinding [0x103] - extension: m4v-4, UTI: dyn.age804rd0fy4a, fileType: MooV request size:16 scale: 1

     

     

    The master started re-encoding chunk #2 after a while, which was already finished before by the slave computer. This happened after the slave finished its work and I cut the connection between the two computers. Then the final file is created at last. Weird. I will run another test later.

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Feb 7, 2014 1:33 AM in response to Steffen Bendix
    Level 1 (60 points)
    Mac OS X
    Feb 7, 2014 1:33 AM in response to Steffen Bendix

    I checked it again. The slave stops its work half way through. I have to cut the network connection and then the master re-encodes the work piece. So, distributed encoding is not really an advantage. Am I the only one with these clustering problems? I can't believe.

  • by Warwick Teale,

    Warwick Teale Warwick Teale Feb 11, 2014 11:56 PM in response to Steffen Bendix
    Level 3 (588 points)
    Feb 11, 2014 11:56 PM in response to Steffen Bendix

    Hi Steffen, Apple just put an update out a coupe of hours ago to compressor.app V4.1.1 - http://support.apple.com/kb/HT4590?

     

    might be worth a look to see if it eases the previous "pain" wth distributed transcoding that you've had.

     

    BR

    warwick

  • by Steffen Bendix,Solvedanswer

    Steffen Bendix Steffen Bendix Feb 12, 2014 12:51 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Feb 12, 2014 12:51 AM in response to Warwick Teale

    Yes, I am testing all morning. The update fixes most of the issues (I knew that three different computers can't have the same problem). Automatic file sharing works via Ethernet. I tried the following setups:

     

    • Master on WIFI, 2 slaves connected by Ethernet: works
    • Master connected by Ethernet with 1 slave, another slave on WIFI: Ethernet connection works, WIFI does not work
    • all 3 computers on WIFI: does not work

     

    I think I we can close this thread unless we want to find out why it does not fully work with WIFI.

     

    Thank you very much for your help, Warwick!

  • by Warwick Teale,

    Warwick Teale Warwick Teale Feb 12, 2014 12:59 AM in response to Steffen Bendix
    Level 3 (588 points)
    Feb 12, 2014 12:59 AM in response to Steffen Bendix

    Hi Steffen yes this was a great thread. You might wanna close it. As for the WIFI LAN and distributed transcode processing, it shouldn't matter what the SUBNET is s long as the resources are there. I think you realise that. Maybe just a local glitch.

     

    If you have every thing work to your satisfaction you might want to enable AUTOMATIC FILE SHARING as well. Care that the /users/shared/Library/compressor/storage is available on your 'slaves" too on those NETWORK interfaces you use. Personally I don't mix the NICs because its can be way too slow to assemble the quicktime components at the end. I try to avoid having Qmaster copy the object.

     

    Good Luck

    W

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Feb 13, 2014 2:49 PM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Feb 13, 2014 2:49 PM in response to Warwick Teale

    I am back to the problem again. After the update I could run several distributed transcoding jobs successfully, it does not work anymore today. The same problem occurs: The slave can't find the shared volume. I'm not sure whether Apple's engineer's advice to delete the /users/shared/Library/compressor folder on both machines broke it. We are still on the issue why it does not work via WIFI. Now it does not work via Ethernet again. It is just rediculous! Ich habe die Schnauze langsam voll!

  • by Warwick Teale,

    Warwick Teale Warwick Teale Feb 13, 2014 9:26 PM in response to Steffen Bendix
    Level 3 (588 points)
    Feb 13, 2014 9:26 PM in response to Steffen Bendix

    Hi Steffen, that's a bummer I can image you are fed up with it. For what it's worth, and as I've mentioned din several previous posts, your issue is merely one of having the SOURCE and TARGET file systems available on all your service nodes (aka service hosts, ala service slaves).. in your case your macbookpro on 1.1.1.2.

    Steffen Bendix wrote:

    The same problem occurs: The slave can't find the shared volume.

     

    I'm not sure whether Apple's engineer's advice to delete the /users/shared/Library/compressor folder on both machines broke it.

    This advice is only a circumvention or workaround however it is not a remedy. This action would impact only the CLUSTER definition. If you have redefined it (as you and and I have detailed in this thread). then it should reappear.

     

    BTW the advice by the Apple people on only utilising filesystem on external volumes is as you appreciate now, IS rather misleading and inconsistent (wrong ?  ) with Apple Qmaster/Compressor documentation..... moving on.....

    We are still on the issue why it does not work via WIFI. Now it does not work via Ethernet again. It is just rediculous!

    Yes I agree it is frustrating. Again I will emphasise this is merely a setup issue with the FILE system availability for Compressor. If WIFI is your "primary" network subnet, then ofcourse this is where Bonjour enabled devices/machines(hosts) are automatically exposed as a ".local"..  Simply use the netstat -r command.. wait 10 seconds and you will all of these devices AND hopefully the two ( without machinenames)on your 1.1.1.x subnet.

     

    Diagnosis procedure:

    1. Refer to the system.log (console.app) and to the STOMP logs in compressor. You will see the BEGIN and END groups and within the errors the Qmastr has in accessing the cluster resources.
    2. using some of the earlier posts in this thread, verify the set up , especially the file system sharing and mount  points, authisaion and access by the servce (slaves ) node(s) . 1.1.1.2.
    3. try NOT to use the EXTRA options yet (automating file sharing, additional instances- use a single instance on each, on MASTER (1.1.1.1) allow others to submit job and so on in order to localise the errors for earlier diagnosis
    4. use a SIMPLE codec that supports multipass and segmenting and make the source clip  10 minutes or more to enable some attempt by Qmaster to distribute te work. (Short clips dont seem to utilise segmenting and more over .. transcoding distribution). Some logic stops this.. a sensible action Ithink.


    I'd hope that you may persist in making this work. It works really well for most of us how rely on it for reducing the transcode (and render .. on old days) duration time for jobs.

     

    Lastly, if there is a BACK CHANNEL (soemwhere else off these forums) you can utilise, then I can assist you over that. Your call.

     

    Warwick

    Hong Kong

     


  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Feb 14, 2014 4:56 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Feb 14, 2014 4:56 AM in response to Warwick Teale

    I think I got it now. After a long troubleshooting session this morning with endless tests and setups I localized a process of DNSCrypt, an extension by OpenDNS. It was hidden and difficult to find. It never made any trouble before. I deinstalled it and now the jobs are distributed just fine via Ethernet.

     

    I created a setup of the master connected to one slave via Ethernet, and to one slave connected via WIFI. Both the slaves got their work pieces and send them back after finishing the jobs. At a second try the WIFI slave lost its connection to the shared volume again. I had no time to investigate this any further and will do this later.

     

    I wonder why DNSCrypt made no trouble when the source file is on an external volume or when the Update of Compressor has been released. Anyway. I will run more tests later and report back.

  • by aidassup,

    aidassup aidassup Mar 21, 2014 7:07 AM in response to Warwick Teale
    Level 1 (0 points)
    Mar 21, 2014 7:07 AM in response to Warwick Teale

    Went so well, up until 5.4:

    Slaves do not show up on the right, nor I can add them by entering IPs (Add button is greyed out)

    Help!!!

first Previous Page 3 of 3