Steffen Bendix

Q: How to share a job with Compressor 4.1?

Can anyone explain how to set up Compressor on two or more computers to share a coding job? I was never successful, neither with the old versions nor the new one. I have connected two computers running Mavericks via Ethernet. They appear in the preferences list of Compressor as inactive and can be selected (with a tick). Starting a job produces no error. Only the little network monitor window shows some activity: "inactive" (in white or yellow), sometimes: "not found" (in red). The computer which sends the job waits endlessly.

 

I deactivated the firewall, connected the computer with DHCP or fixed IP but no success. What else do I have to do?

iMac (27-inch, Late 2012), OS X Mountain Lion (10.8.4), 16 GB, Nvidia GeForce GTX 680MX

Posted on Dec 19, 2013 3:23 PM

Close

Q: How to share a job with Compressor 4.1?

  • All replies
  • Helpful answers

Previous Page 2 of 3 last Next
  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 26, 2014 12:26 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Jan 26, 2014 12:26 AM in response to Warwick Teale

    I checked everything and every volume is mounted. There is read/write access from the slave. The Macs are connected over direct Ethernet. "Ethernet only" is set in the preferences of both Compressor.apps. To avoid conflicts I disabled WIFI on the slave Mac. Here is the output of the terminal.

     

    Friedward:~ steffen$ ls -l /Volumes

    total 8

    drwxrwxr-t  38 root     admin  1360  1 Dez 13:41 Backup

    drwx------   1 steffen  staff   264 26 Jan 09:09 Finished

    lrwxr-xr-x   1 root     admin     1 26 Jan 08:54 Friedward -> /

    drwxrwxrwx   0 root     wheel     0 26 Jan 09:07 MobileBackups

    drwx------   1 steffen  staff   264 24 Jan 21:37 Source

    drwx------   1 steffen  staff   264 24 Jan 22:04 Storage

    drwx------   1 steffen  staff   264 24 Jan 22:04 Storage-1

    Friedward:~ steffen$ df

    Filesystem                        512-blocks       Used Available Capacity   iused    ifree %iused  Mounted on

    /dev/disk0s2                       877175552  381377104 495286448    44%  47736136 61910806   44%   /

    devfs                                    365        365         0   100%       632        0  100%   /dev

    /dev/disk0s4                        97656248   94704872   2951376    97%  11838107   368922   97%   /Volumes/Backup

    map -hosts                                 0          0         0   100%         0        0  100%   /net

    map auto_home                              0          0         0   100%         0        0  100%   /home

    localhost:/NgXXEf4A4YasM2yTNDBPSd  877175552  877175552         0   100%         0        0  100%   /Volumes/MobileBackups

    //steffen@1.1.1.1/Source          1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Source

    //steffen@1.1.1.1/Finished        1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Finished

    //steffen@1.1.1.1/Storage         1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Storage

    //steffen@1.1.1.1/Storage-1       1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Storage-1

     

    One of the Storage folders is automatically titled "Storage-1" to avoid names conflicts I think. May this be the reason?

     

    On either Mac I can't enable any instances.

     

     

      • confirm for this post that the file path in the error message is actually ar Master (@ 1.1.1.1)  @1.1.1.1/Users/Downloads/Source/Test.mov?

    No, it is the path without "@1.1.1.1" at the beginning. By the way, I can't mount any volume with "

    • afp://steffen@1.1.1.1/users/steffen/Downloads/Compressor/Source".

     

    This brings up an error that there were no such sharing. I only can mount with "afp://steffen@1.1.1.1/".

  • by Warwick Teale,

    Warwick Teale Warwick Teale Jan 26, 2014 1:38 AM in response to Steffen Bendix
    Level 3 (588 points)
    Jan 26, 2014 1:38 AM in response to Steffen Bendix

    HI Steffen looks good! THe error you describe is my doing. I my haste I typed in the INCORRECT specification (typo) for your path as

    (@ 1.1.1.1)  @1.1.1.1/Users/Downloads/Source/Test.mov?

     

    It should have been:

    Master (@ 1.1.1.1)  steffen@1.1.1.1/Users/steffen/Downloads/Source/ 

     


    Steffen Bendix wrote:

     

    I checked everything and every volume is mounted. There is read/write access from the slave. The Macs are connected over direct Ethernet. "Ethernet only" is set in the preferences of both Compressor.apps. To avoid conflicts I disabled WIFI on the slave Mac. Here is the output of the terminal.

     

    Friedward:~ steffen$ ls -l /Volumes

    total 8

    drwxrwxr-t  38 root     admin  1360  1 Dez 13:41 Backup

    drwx------   1 steffen  staff   264 26 Jan 09:09 Finished

    lrwxr-xr-x   1 root     admin     1 26 Jan 08:54 Friedward -> /

    drwxrwxrwx   0 root     wheel     0 26 Jan 09:07 MobileBackups

    drwx------   1 steffen  staff   264 24 Jan 21:37 Source

    drwx------   1 steffen  staff   264 24 Jan 22:04 Storage

    drwx------   1 steffen  staff   264 24 Jan 22:04 Storage-1

    Friedward:~ steffen$ df

    Filesystem                        512-blocks       Used Available Capacity   iused    ifree %iused  Mounted on

    /dev/disk0s2                       877175552  381377104 495286448    44%  47736136 61910806   44%   /

    devfs                                    365        365         0   100%       632        0  100%   /dev

    /dev/disk0s4                        97656248   94704872   2951376    97%  11838107   368922   97%   /Volumes/Backup

    map -hosts                                 0          0         0   100%         0        0  100%   /net

    map auto_home                              0          0         0   100%         0        0  100%   /home

    localhost:/NgXXEf4A4YasM2yTNDBPSd  877175552  877175552         0   100%         0        0  100%   /Volumes/MobileBackups

    //steffen@1.1.1.1/Source          1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Source

    //steffen@1.1.1.1/Finished        1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Finished

    //steffen@1.1.1.1/Storage         1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Storage

    //steffen@1.1.1.1/Storage-1       1654970944 1072194680 582776264    65% 134024333 72847033   65%   /Volumes/Storage-1

     

    One of the Storage folders is automatically titled "Storage-1" to avoid names conflicts I think. May this be the reason?


    From the slave over Ethernet 1.1.1.2 this looks pretty good. Yes you are right you will have Storage-1 and Storage to resolve the duplicate names. You certain can name these properly to avoid confusion... however let's do this later.

     

    I can see drwx------   1 steffen  staff   264 24 Jan 21:37 Source is mounted. So your own mounts worked well!

     

    For our discussion, the file system "Finished" is the TARGET of the transcode? IF yes, then it looks like you have performed admirably   and it all should work because every thing looks in place and the slave (1.1.1.2) may surely access these file systems in read/write mode (rw) from your listing.

     

     

    Steffen Bendix wrote:

     

    On either Mac I can't enable any instances.

     

     

    Instances of the compressord subtask: is this to which you refer?

    comressor.instances.png

     

    If you can't ENABLE ADDITIONAL instances  it nor SELECT (form the listbox) any more instances, it's quite normal.

     

    REASON: I understand it's a NEW feature of Qmaster/Compressor for thoughput balancing. Its seems that this part of the UI examines the computer resources you have for host to base this for further additions. This is the main reason I proposed you DON'T bother to set it on whilst you are still undertaking to stabilise your new cluster.

     

    FWIW, grand central dispatch will put you available to cores to work... Suggest you dont be concerned.

     

    So, have a look at what's running now on your slave and your master:


    Can you see some compressd subtasks running on your slave & your master?


    USe the Activity Monitor.app to have a look. Use the filter compressord in  to locate only those.

    actity-monitor.filter.png

      • BTW... Choose Activity Monitor.app / View Menu/ All Processes (my example is idle so it wont show in active processes)
      • set the filter to compressord and you should see at least one instance of compressord on your slave (1.1.1.2)
      • please verify
      • Here's what I have on one of the mac-minis service nodes current;y. It has only 2 instances and when running utilise the whole 8 cores of the i7 to nearly 100%.

     

    compresord_instances.png

    As an example on our MAC PRO nehalem with an ATI 5870 graphics card, 16 virtual cores and 32GB of ram, the new Compressor.app 4.1 UI qmaster will only permit 3 additional instances. The 2011 mac minis are only permitted ONE additional instance (16GB ram i7)... These run flat out maximum with qmaster.

     

    Lastly, were you able to submit a batch form your master using your CLUSTER?

     

    IF not, would you post any errors you see in ~/logs/compressor (console.app)

     

    You have it all working soon enough mate!

     

    Post your results for others to see.

     

    Warwick

    Hong Kong

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 26, 2014 4:43 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Jan 26, 2014 4:43 AM in response to Warwick Teale

    Warwick Teale wrote:

     

    HI Steffen looks good! THe error you describe is my doing. I my haste I typed in the INCORRECT specification (typo) for your path as

    (@ 1.1.1.1)  @1.1.1.1/Users/Downloads/Source/Test.mov?

     

    It should have been:

    Master (@ 1.1.1.1)  steffen@1.1.1.1/Users/steffen/Downloads/Source/ 

     

    It doesn't matter. I can mount the network volumes as I wrote.

     

     

    For our discussion, the file system "Finished" is the TARGET of the transcode? IF yes, then it looks like you have performed admirably   and it all should work because every thing looks in place and the slave (1.1.1.2) may surely access these file systems in read/write mode (rw) from your listing.

     

    Yes, "Finished" is the folder where the results of Compressor should saved to, "Source" is the folder, where the footage to be transcoded is lying.

     

     

     

    Instances of the compressord subtask: is this to which you refer?

    comressor.instances.png

     

    If you can't ENABLE ADDITIONAL instances  it nor SELECT (form the listbox) any more instances, it's quite normal.

    This is what I mean and I know that this option is hardware specific. My Macs are not able to run more instances.

     

     

     

    Can you see some compressd subtasks running on your slave & your master?

     

     

    Here are screenshots of the master and the slave computer while computing:

     

    Aktivity iMac.png

    Detail:

    Detail Activity iMac.png

     

     

    Activity MacBook Pro.png

     

    Detail:

    Activity MacBook Pro.png

     

    You see, the slave (MacBook Pro) is idle and the iMac (master) runs on full speed. It devides the job in four chunks and works on them one after another for itself.

     

    The process "compressord" crashes on the master when I cancel the transcoding job in Compressor or Share Monitor.

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 26, 2014 7:48 AM in response to Steffen Bendix
    Level 1 (60 points)
    Mac OS X
    Jan 26, 2014 7:48 AM in response to Steffen Bendix

    Weird. When I create a "Source" and "Finished" folder on the slave computer (same folder structure) and put an alias of the source file into it, it works. Somehow. The slave transcodes its piece of work but put it into the "Finished" folder localy. This finished work piece is later added to the final transcoded movie properly. It is a dirty but working solution and far from being user friendly. But better than nothing.

  • by Warwick Teale,

    Warwick Teale Warwick Teale Jan 26, 2014 8:54 PM in response to Steffen Bendix
    Level 3 (588 points)
    Jan 26, 2014 8:54 PM in response to Steffen Bendix

    HI Steffen, thanks for posting your screen views. I think you are measuring / monitoring using the wrong (our out of date tool). I think also you stuff works and you may be looking in the wrong place to see it working . I also don't thing you have been sumitting the jobs on your own defined cluster.

     

    LEt's see.... FWIW you've done all the correct things to get this going. I think you'll kick yourself when you see... ,

     

    My Deutsch is worse than dreadful..,  however does "Deiser Computer" mean "This Computer" in English? If so then I think you are looking at the wrong thing mate!.

     

    What gives it away is that the icon (looks like a pee-sea) is not an icon for a defined cluster for distributed nodes cluster

     

    Some Observations:

    1. The utility Share Monitor.app is hidden away (buried)  these days in Compressor.app 4.1. I noticed myself that it now WONT SEE the clusters (groups) defined in Compressor.app v4.1. I will stand corrected on this though.
    2. Note yourself that SHARE MONITOR.app is showing the job running on Deiser Computer/This computer.. This is NOT your defined/created CLUSTER name (aka "Group" ,them now used in Compressor.app V4.1) .. therefore the job will not utilise any distibuted nodes.
      • I'll be pleasantly surprised if it is..! Please verify for my curiosity.
    3. The Compressor.app V4.1 UI "Active" certainly shows segments transcoding on your envirorment...
    4. and lastly you've confirmed you and instance of compressord running on it.

    Here are screenshots of the master and the slave computer while computing:

     

    Aktivity iMac.png

    1. Detail:

    Detail Activity iMac.png

     

     

    The other screen shots of Compressor.app V4.1 UI on both the MAster (1.1.1.1) and the Slave (1.1.1.2)  certinly look correct as you have pointed out.

     

    SO.. Im very very curious if you have:

    1. successfully defined a CLUSTER (group) that is a UNION of the master (1.1.1.1) and slave (1.1.1.2) and named it something like "steffencluster"?
      • if that was true, AND Share Monitor.app (that is now buried away) was correctly showing the cluster 'steffencluster" and not what is  "Diesier Computer/This Computer"
      • steffens_cluster.png
      • the above if from Page 2) of the new  Apple Compressor V4.1 USers Guide (Dec 2013).
    2. SUMBITTED the "Test.mov" job using the group defined (cluster)?
      • you MUST submit the job 'tets.movie" and its distibutions (u may use more than one) , to a GROUP or CLUSTER that represents the distirbution nodes (1.1.1.1 and 1.1.1.2 etc)
      • here is an example (copied from PAge 102 of the  Apple Compressor V4.1 USers Guide (Dec 2013).
      • Looking at Compressor.app V4.1. UI in the Current Tab  where you set up the job. look down at the middle lower left for a "process on: list box.
      • Do you see you "steffencluster" in the list? IF so , select it and use THAT "steffencluster" for your distributed processing and not "Deiser Computer" (This Computer)?

    submit_clusterpdf.png

     

    DId you submit your 'TEst'.mov" using the CLUSTER you defined???.. it doesn't look like it mate !

     

    Please verify and post back. I'm rather curious...

     

    Post your results for others to see.

     

    Warwick

    Hong Kong

  • by Warwick Teale,

    Warwick Teale Warwick Teale Jan 26, 2014 8:57 PM in response to Steffen Bendix
    Level 3 (588 points)
    Jan 26, 2014 8:57 PM in response to Steffen Bendix

    Hi steffen, please refer to my previous post.

     

    I believe you're nearly 99.99% there to getting this all working.

     

    When you do you will be very pleeeeaaaaaased!

     

    You will appreciate that Compressor.app is your friend again.

     

    Warwick

    Hong Kong

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 27, 2014 12:19 AM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Jan 27, 2014 12:19 AM in response to Warwick Teale

    Hi Warwick,

     

    yes, I see my cluster in the preferences but in the activity section of Compressor and in Share Monitor there are only the entries "This Computer" ("Dieser Computer" in German). The once and only time sharing worked some days ago, Share Monitor successfully showed which Computer worked on which chunk but there was never any other entry than "This Computer" in the list.

     

    I found out something more yesterday. When I start the job with the test.mov lying on the SLAVE computer, the log on the slave shows the error that this file cannot be found at "/Volumes/Users/steffen/Downloads/Compressor/Source/Test.mov". When the mov-file is situated on the MASTER computer, the error reads "/Users/steffen/Downloads/Compressor/Source/Test.mov. Compressor on the slave searches for the file always in the wrong location.

     

    And here is something more: I created a new user on the slave computer. I can access the 1.1.1.1 network without any problems but I can't see the slave in Compressors clustering preferences. The slave doesn't see the master, too.  I don't have the firewall or any other similar software running which could prevent the network connection.

  • by Warwick Teale,

    Warwick Teale Warwick Teale Jan 27, 2014 12:55 AM in response to Steffen Bendix
    Level 3 (588 points)
    Jan 27, 2014 12:55 AM in response to Steffen Bendix

    HI Steffen,

    Steffen Bendix wrote:

     

    yes, I see my cluster in the preferences but in the activity section of Compressor and in Share Monitor there are only the entries "This Computer" ("Dieser Computer" in German). The once and only time sharing worked some days ago, Share Monitor successfully showed which Computer worked on which chunk but there was never any other entry than "This Computer" in the list.

    Share Monitor.app..

    • hmm I porpose you don't rely / use Share MOnitor.app  for Compressor.app V4.1. 2013.
    • From our experience, it has been ("purposely buried' - like in the movie  2001!)
    • There is a NEW item Network Encoding Monitor.app (command+E , +E) that is barebone and shows you the instances that are active. I mentioned this in a previous post. USe the "SHOW LOGS" to watch the STOMPxxx compressord status progress
    • Share Monitor.app at its best was often unreliable.. (socket errors)  suggest you don't "bank" on its improvement especially that it's been moved out of the /Applications folder to oblivion in a Contents somewhere  i mentioned on these forums LoL!
    • The ACTIVE pane in  Compressor.app V4.1 UI is your best bet. It currently lacks showing where the job is being dispatched on what nodes. PErsonally I think this is part of simplifying it for everyones use.
      • also using the Activity Monitor.app and
      • also the ~/Library/Logs/Compressor/ as I mention above
      • these will show you if you  segments have been dispatched on the 1.1.1.2 slave node.

     

    I found out something more yesterday. When I start the job with the test.mov lying on the SLAVE computer, the log on the slave shows the error that this file cannot be found at "/Volumes/Users/steffen/Downloads/Compressor/Source/Test.mov". When the mov-file is situated on the MASTER computer, the error reads "/Users/steffen/Downloads/Compressor/Source/Test.mov. Compressor on the slave searches for the file always in the wrong location.

     

    Yes this is very strange, seeing that these are mounted, THis is certinaly the error tha twill stop the SLAVE 1.1.1.2 from having those segments dispatched on them.

     

    I dunt know what else I can suggest here. Unless it's some envirnment steffen-only circumstance, it shoudl be able to be resolved using some of the methodology we and have have offered to you.

     

    Regarding the above would you verify that the AFP failure (access by the cluster) is for 1.1.1.2/volumes/Source/ that should be the mount point it shoudl be accessing.. not the /users/steffen/downloads/compressor path... seems strange that the slave would ask fo that.

     

    Steffen Bendix wrote:

    And here is something more: I created a new user on the slave computer. I can access the 1.1.1.1 network without any problems but I can't see the slave in Compressors clustering preferences. The slave doesn't see the master, too.  I don't have the firewall or any other similar software running which could prevent the network connection.

     

    Interesting... fwiw

    I would have thought the cluster (steffenscluster" was global especially when that host 1.1.1.2 is part of it.. I dont know.

    Slave: recall you shoudl have "Allow others to process.." = OFF on that slave 1.1.1.2 host, thus that SLAVE cannot  submit job to the 1.1.1.1 master. MAybe a reason.

     

     

    Additional docs:

    Steffen, I believe you are already using this correctly and the failures in teh set up are resolvable.

     

    Would you be so kind as ather some screen shots (shift+cntl+command+4), then Preview.app and +N , make and crop a .PNG's and upload to this thread for:

     

    On the master 1.1.1.1

    1. Compressor.app V4.1 Preferences / Shared Computers
    2. Failed part of any STOMP and CONTROLLER logs for one job in your ~/Library/Logs/Compressor
    3. a grab of the Compressor.app V4.1 Current pane with all the options you have selected to SUBMIT the "test.mov' Batch so we can see what you are submitting
    4. NEtwork Encoder Monitor - a screen grab of that  - there should be two instances as IDEL (one on each node 1.1.1.1 and 1.1.1.2)

     


    On the slave/service node 1.1.1.2

     

    1. Compressor.app V4.1 Preferences / Shared Computers
    2. relevent Failed part of any STOMP and CONTROLLER logs for one job in your ~/Library/Logs/Compressor
    3. Network Encoder Monitor - a screen grab of that  - there should be two instances as IDEL (one on each node 1.1.1.1 and 1.1.1.2)

     

    See how you go mate!

     

    Warwick

    Hong Kong

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 27, 2014 11:26 PM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Jan 27, 2014 11:26 PM in response to Warwick Teale

    Share Monitor.app..

    • hmm I porpose you don't rely / use Share MOnitor.app  for Compressor.app V4.1. 2013.

    Well, it showed which node was running when clustering worked once.

     

     

     

    I dunt know what else I can suggest here. Unless it's some envirnment steffen-only circumstance, it shoudl be able to be resolved using some of the methodology we and have have offered to you.

     

    I tested it with three different computers in any combination. It is the same error on all of them.

     

     

     

    Regarding the above would you verify that the AFP failure (access by the cluster) is for 1.1.1.2/volumes/Source/ that should be the mount point it shoudl be accessing.. not the /users/steffen/downloads/compressor path... seems strange that the slave would ask fo that.

    I don't really understand. Do you mean I should check that the network volumes (Storage folders etc.) can be accessed by the slave computer? Yes they can.

     

     

     

    Slave: recall you shoudl have "Allow others to process.." = OFF on that slave 1.1.1.2 host, thus that SLAVE cannot  submit job to the 1.1.1.1 master. MAybe a reason.

     

    You mean the master should have switched OFF "Allow others to process" and the slave switched it ON? Otherwise the master could not submit anything. And yes, the settings are like that.

     

     

     

    On the master 1.1.1.1

    1. Compressor.app V4.1 Preferences / Shared Computers
    2. Failed part of any STOMP and CONTROLLER logs for one job in your ~/Library/Logs/Compressor
    3. a grab of the Compressor.app V4.1 Current pane with all the options you have selected to SUBMIT the "test.mov' Batch so we can see what you are submitting
    4. NEtwork Encoder Monitor - a screen grab of that  - there should be two instances as IDEL (one on each node 1.1.1.1 and 1.1.1.2)

     

    1. Preferences iMac:

    4 iMac.png

    5 iMac.png

    6 iMac.png

     

    2. Logs of STOMP iMac (the other logs didn't show something for the test job.

     

    <mrk tms="412584901.470" tmt="01/28/2014 07:55:01.470" pid="18626" kind="begin" what="service-request" req-id="08623C1A-19CA-4C8A-A87C-234A674CEDAC:1" msg="Preflighting."></mrk>

    <log tms="412584901.492" tmt="01/28/2014 07:55:01.492" pid="18626" msg="Virtual service count = 0 (srcType = 'quicktime', settingsType = 'H264iPod')"/>

    <mrk tms="412584902.506" tmt="01/28/2014 07:55:02.506" pid="18626" kind="end" what="service-request" req-id="08623C1A-19CA-4C8A-A87C-234A674CEDAC:1" msg="Preflighting service request end."></mrk>

    <mrk tms="412584902.541" tmt="01/28/2014 07:55:02.541" pid="18626" kind="begin" what="service-request" req-id="08623C1A-19CA-4C8A-A87C-234A674CEDAC:3" msg="Preprocessing."></mrk>

    <log tms="412584902.549" tmt="01/28/2014 07:55:02.549" pid="18626" msg="preProcess for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v"/>

    <log tms="412584902.577" tmt="01/28/2014 07:55:02.577" pid="18626" msg="done preProcess for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v"/>

    <mrk tms="412584903.593" tmt="01/28/2014 07:55:03.593" pid="18626" kind="end" what="service-request" req-id="08623C1A-19CA-4C8A-A87C-234A674CEDAC:3" msg="Preprocessing service request end."></mrk>

    <mrk tms="412584903.651" tmt="01/28/2014 07:55:03.651" pid="18626" kind="begin" what="service-request" req-id="28FAAB59-0B83-41B0-9408-A03337807371:1" msg="Processing."></mrk>

    <log tms="412584903.667" tmt="01/28/2014 07:55:03.667" pid="18626" msg="starting _processRequest for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <log tms="412584903.667" tmt="01/28/2014 07:55:03.667" pid="18626" msg="Writing transcode segment: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <log tms="412584903.685" tmt="01/28/2014 07:55:03.685" pid="18626" msg="QuickTime Transcode, rendering in YUV 8 bit 422"/>

    <mrk tms="412585019.945" tmt="01/28/2014 07:56:59.945" pid="18626" what="service-request" req-id="28FAAB59-0B83-41B0-9408-A03337807371:1" msg="Cancelling."></mrk>

    <log tms="412585019.993" tmt="01/28/2014 07:56:59.993" pid="18626" msg="QuickTime Transcode aborted by user: 116.288 seconds.  status = 0"/>

    <log tms="412585019.997" tmt="01/28/2014 07:56:59.997" pid="18626" msg="Done _processRequest for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <mrk tms="412585020.910" tmt="01/28/2014 07:57:00.910" pid="18626" kind="end" what="service-request" req-id="28FAAB59-0B83-41B0-9408-A03337807371:1" msg="Processing service request end."></mrk>

    <mrk tms="412585250.750" tmt="01/28/2014 08:00:50.750" pid="23163" kind="begin" what="log-session"/>

    <mrk tms="412585286.407" tmt="01/28/2014 08:01:26.407" pid="23163" kind="begin" what="service-request" req-id="89C46569-BD55-49B8-B1D7-2FBB49F213E2:1" msg="Preflighting."></mrk>

    <log tms="412585286.449" tmt="01/28/2014 08:01:26.449" pid="23163" msg="Virtual service count = 0 (srcType = 'quicktime', settingsType = 'H264iPod')"/>

    <mrk tms="412585287.415" tmt="01/28/2014 08:01:27.415" pid="23163" kind="end" what="service-request" req-id="89C46569-BD55-49B8-B1D7-2FBB49F213E2:1" msg="Preflighting service request end."></mrk>

    <mrk tms="412585287.452" tmt="01/28/2014 08:01:27.452" pid="23163" kind="begin" what="service-request" req-id="89C46569-BD55-49B8-B1D7-2FBB49F213E2:3" msg="Preprocessing."></mrk>

    <log tms="412585287.461" tmt="01/28/2014 08:01:27.461" pid="23163" msg="preProcess for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v"/>

    <log tms="412585287.521" tmt="01/28/2014 08:01:27.521" pid="23163" msg="done preProcess for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v"/>

    <mrk tms="412585288.486" tmt="01/28/2014 08:01:28.486" pid="23163" kind="end" what="service-request" req-id="89C46569-BD55-49B8-B1D7-2FBB49F213E2:3" msg="Preprocessing service request end."></mrk>

    <mrk tms="412585288.544" tmt="01/28/2014 08:01:28.544" pid="23163" kind="begin" what="service-request" req-id="9B67E8B2-D7D8-4239-9860-6701521E2004:1" msg="Processing."></mrk>

    <log tms="412585288.561" tmt="01/28/2014 08:01:28.561" pid="23163" msg="starting _processRequest for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <log tms="412585288.561" tmt="01/28/2014 08:01:28.561" pid="23163" msg="Writing transcode segment: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <log tms="412585288.582" tmt="01/28/2014 08:01:28.582" pid="23163" msg="QuickTime Transcode, rendering in YUV 8 bit 422"/>

    <mrk tms="412585322.137" tmt="01/28/2014 08:02:02.137" pid="23163" what="service-request" req-id="9B67E8B2-D7D8-4239-9860-6701521E2004:1" msg="Cancelling."></mrk>

    <log tms="412585322.266" tmt="01/28/2014 08:02:02.266" pid="23163" msg="QuickTime Transcode aborted by user: 33.6588 seconds.  status = 0"/>

    <log tms="412585322.271" tmt="01/28/2014 08:02:02.271" pid="23163" msg="Done _processRequest for job target: file:///Users/steffen/Downloads/Compressor/Finished/Test.m4v-1"/>

    <mrk tms="412585322.626" tmt="01/28/2014 08:02:02.626" pid="23163" kind="end" what="service-request" req-id="9B67E8B2-D7D8-4239-9860-6701521E2004:1" msg="Processing service request end."></mrk>

    <mrk tms="412585333.551" tmt="01/28/2014 08:02:13.551" pid="23390" kind="begin" what="log-session"/>

     

    3. Compressor options iMac:

    1 iMac.png

    2 iMac.png

    4. Network Encoder Monitor busy iMac:

    3 iMac.png

     

     

     

    On the slave/service node 1.1.1.2

     

    1. Compressor.app V4.1 Preferences / Shared Computers
    2. relevent Failed part of any STOMP and CONTROLLER logs for one job in your ~/Library/Logs/Compressor
    3. Network Encoder Monitor - a screen grab of that  - there should be two instances as IDEL (one on each node 1.1.1.1 and 1.1.1.2)

    1. Compressor preferences MacBook Pro (MBP):

    3 MBP.png

    4 MBP.png

    5 MBP.png

     

    2. Log of transcoderx MBP:

     

    <mrk tms="412584903.688" tmt="01/28/2014 07:55:03.688" pid="18591" kind="begin" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Processing."></mrk>

    <mrk tms="412584903.742" tmt="01/28/2014 07:55:03.742" pid="18591" kind="end" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg=" error: Ausgangsdatei „/Users/steffen/Downloads/Compressor/Source/Test.mov“ wurde nicht gefunden"></mrk>

    <mrk tms="412584904.689" tmt="01/28/2014 07:55:04.689" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412584989.550" tmt="01/28/2014 07:56:29.550" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412585019.952" tmt="01/28/2014 07:56:59.952" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412585165.979" tmt="01/28/2014 07:59:25.979" pid="23199" kind="begin" what="log-session"/>

    <mrk tms="412585288.583" tmt="01/28/2014 08:01:28.583" pid="23199" kind="begin" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg="Processing."></mrk>

    <mrk tms="412585293.626" tmt="01/28/2014 08:01:33.626" pid="23199" kind="end" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg=" error: Source file /Users/steffen/Downloads/Compressor/Source/Test.mov not found"></mrk>

    <mrk tms="412585322.120" tmt="01/28/2014 08:02:02.120" pid="23199" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg="Error: unrecognized request."></mrk>

     

    3. Network Encoder Monitor idle MBP:

    1 MBP.png

     

    To be continued in another post because I can't upload more pictures in this one.

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 27, 2014 11:27 PM in response to Steffen Bendix
    Level 1 (60 points)
    Mac OS X
    Jan 27, 2014 11:27 PM in response to Steffen Bendix

    4. Network Encoder Monitor busy MBP:

    2 MBP.png

     

    I canceled the job after a while because it would have taken too long to wait for the iMac (master) to finish work.

  • by Warwick Teale,

    Warwick Teale Warwick Teale Jan 29, 2014 12:21 AM in response to Steffen Bendix
    Level 3 (588 points)
    Jan 29, 2014 12:21 AM in response to Steffen Bendix

    Hi Steffen, thanks for the logs and screen images:

     

    Firstly, i WILL STAND CORRECTED that Share Monitor.app DOES show a cluster correctly. I simulated my setup something like yours and launched the setup, and to my surprise (and self-admonishment!) .... in fact you can see Share Monitor.app showing the DISTRIBUTED nodes (node in this case) running. The unusual (as per Compressor V4.0) it shows the active cluster as THIS COMPUTER which I regret telling you it was an error..

     

    You can see in this job below that compressor.app v4.1 has divided this job into 4 segments across the cluster:

    • the main node (mac pro)
    • and the slave (the macmini).
    • The macpro is the CONTROLLER (can submit jobs). click to enlarge.

     

    share_monitor.png

     


     

    Steffen Bendix wrote:

     

    I dunt know what else I can suggest here. Unless it's some envirnment steffen-only circumstance, it shoudl be able to be resolved using some of the methodology we and have have offered to you.

     

     

    I tested it with three different computers in any combination. It is the same error on all of them.

     

     

    YES this is quite perplexing indeed. Its looking simply likely that the resources ARE NOT really available to all the cluster. see later..

     


    Regarding the above would you verify that the AFP failure (access by the cluster) is for 1.1.1.2/volumes/Source/ that should be the mount point it shoudl be accessing.. not the /users/steffen/downloads/compressor path... seems strange that the slave would ask fo that.

    I don't really understand. Do you mean I should check that the network volumes (Storage folders etc.) can be accessed by the slave computer? Yes they can.

     

     

     

    FILE SHARING: Well something YOU could ADD just to try out is to specify the folder as being able to be shared for all. I'm assuming you don't have OSX SERVER running. Try this in the Finder (it won't hurt anything).. like this example (click to enlarge)

    sharing_finder.png

    On each HOST (1.1.1.1 and 1.1.1.2) for each directory (folder) that you want to share in the cluster (between these two hosts including the /compressor/storage etc)

    1. Using the Finder, select the directory
    2. use File / Get Info (+i)
    3. then Tick (enable) Sharing
    4. the resulting SHARENAME (Sharepoint) will be the name of the folder that the hosts can see.
    5. do this for ALL folders on all hosts used by the cluster (you can turn it off later)

     

    Re do again the folder mounts of the master 1.1.1.1 filesystems you need.. do this form the Slave Finder:

    Now...on the slave (1.1.1.2) dismount (eject | +e) and remount / share them as you have been doing. In fact this time try this in the Finder to remount:

    1. On Slave (1.1.1.2) in Finder
    2. Connect to Server (+k)
    3. enter 1.1.1.1 as the host (because we want them mounted over this specific ethernet path)
    4. you will see a list of SHARED mount points (these appear now because of your actions above!)
    5. select the ones you need accessible by the host 1.1.1.2 (current host) for the cluster.

     

    Try your batch job again. Recall you can always return and disable these settings.

     

    Further...

     


     

    Slave: recall you shoudl have "Allow others to process.." = OFF on that slave 1.1.1.2 host, thus that SLAVE cannot  submit job to the 1.1.1.1 master. MAybe a reason.

     

    You mean the master should have switched OFF "Allow others to process" and the slave switched it ON? Otherwise the master could not submit anything. And yes, the settings are like that.

     

    YES you are correct!: Whoops another iPHONE typo .. sorry once again the SLAVE must be ON - , while you are testing this, there is no need to have the MASTER (1.1.1.1) must be OFF  for our testing, be the only one also be a method to submit jobs and its OWN.  I pointed out originally I have suggested this only to mitigate (reduce) any other logs and problem errors. (As you have done)

     

    Here's is the example of my simulated setup for you. Here you can see this mac-mini (service node) Compressor.app V4.1 preferences/My Computer (Deiser computer)

    allow_others_to_process.png

     

    The rest of the setup looks correct.

    • the job has JOB SEGMENTING enabled
    • and the job is performing a SEGMENTED TRANSCODE on the iMAC (1.1.1.1 master) ..
    • settigs look good...

     

     

    3. Compressor options iMac:

    1 iMac.png

     

     

    So.. the issue of course is that the slave (1.1.1.22 - the MBpro) cant access the resources and thus will not become engaged in the cluster for this job.

     

    Note that the slave node (1.1.1.2 - MPB) CANNOT see the cluster you defined! it will be "Untitled" as a default.( you can rename it on the slave)

     

    here's is the Compressor.app V4.1 Preferences/Shared Computers from MY SIMULATION setup for the SLAVE (mac-mini-01)

    shared_computers_mac-mini_01.png

     

    and here is the one BELOW from your MAcBook Pro (1.1.1.2 - slave) - I dont see the cluster?

     

     

    4 MBP.png

     

     

    Looking at the MPros compressor.STOMP log below in RED there is ofcourse the error that the path to an object (some resource - your Test.mov) wurde nicht gefunden (not found??) can be found on the current host (this host) and thus this node (1.1.1.2) is excluded from joining in the cluster for this job (839C7087-B6A2-4302-A505-70D020186B4B)

     

     

    2. Log of transcoderx MBP:

     

    <mrk tms="412584903.688" tmt="01/28/2014 07:55:03.688" pid="18591" kind="begin" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Processing."></mrk>

    <mrk tms="412584903.742" tmt="01/28/2014 07:55:03.742" pid="18591" kind="end" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg=" error: Ausgangsdatei „/Users/steffen/Downloads/Compressor/Source/Test.mov“ wurde nicht gefunden"></mrk>

    <mrk tms="412584904.689" tmt="01/28/2014 07:55:04.689" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412584989.550" tmt="01/28/2014 07:56:29.550" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412585019.952" tmt="01/28/2014 07:56:59.952" pid="18591" what="service-request" req-id="839C7087-B6A2-4302-A505-70D020186B4B:1" msg="Error: unrecognized request."></mrk>

    <mrk tms="412585165.979" tmt="01/28/2014 07:59:25.979" pid="23199" kind="begin" what="log-session"/>

    <mrk tms="412585288.583" tmt="01/28/2014 08:01:28.583" pid="23199" kind="begin" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg="Processing."></mrk>

    <mrk tms="412585293.626" tmt="01/28/2014 08:01:33.626" pid="23199" kind="end" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg=" error: Source file /Users/steffen/Downloads/Compressor/Source/Test.mov not found"></mrk>

    <mrk tms="412585322.120" tmt="01/28/2014 08:02:02.120" pid="23199" what="service-request" req-id="6416F3AF-D362-40F6-A9B4-2F9C0903D0DF:1" msg="Error: unrecognized request."></mrk>

    and .. as the English (the Poms say) ..'the plot thickens!"

     

    Additionally the Network Encoder Monitor on the MACBOOKPRO (1.1.1.2)  DOES NOT see the iMAC instances - looks bad.--

     

     

     

    3. Network Encoder Monitor idle MBP:

    1 MBP.png

     

    ANd the NETWORK Encodong Monitor is showing the RED message confirming the resources was not found. (last post from steffen https://discussions.apple.com/servlet/JiveServlet/downloadImage/370535/2+MBP.png)

     

    What to DO?:


    As in a few posts ago, I thought you had this mailed down pretty much.

     

    1. It looks to me as though DIRECTORIES you need from 1.1.1.1 (master) and simply NOT MOUNTED / SHARED on  slave (1.1.1.2 - MACBOOKPRO)
    2. On the SLAVE nodes (1.1.1.2) Try the "SET THE SHARING tick on " in the finder as I mentioned above. The FINDER is very easy to do like that.
    3. then use the Finder to check eth access and permissions and the path too while you are there (+i).

     

    (FWIW, I use OSX SERVER to administer the complex I have here. You definitely DO NOT need it BTW.

     

    Mate you're so close to getting this working.. Please don't give up (Warwick's pep-talk to Steffen!)

     

    Post your results for others to see.

     

    Warwick

    Hong Kong

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 29, 2014 12:40 PM in response to Warwick Teale
    Level 1 (60 points)
    Mac OS X
    Jan 29, 2014 12:40 PM in response to Warwick Teale

     

     

    and here is the one BELOW from your MAcBook Pro (1.1.1.2 - slave) - I dont see the cluster?

     

     

    4 MBP.png

    The cluster can be seen from the slave computer? I can't really remember, but I think there was nothing like this in the list when the very one successful distributed transcoding took place.

     


     

     

    1. It looks to me as though DIRECTORIES you need from 1.1.1.1 (master) and simply NOT MOUNTED / SHARED on  slave (1.1.1.2 - MACBOOKPRO)
    2. On the SLAVE nodes (1.1.1.2) Try the "SET THE SHARING tick on " in the finder as I mentioned above. The FINDER is very easy to do like that.
    3. then use the Finder to check eth access and permissions and the path too while you are there (+i).

     

    This is a problem and it was one of the first attempts I did before I posted here. The Shared volume, which appears after a job is started on the master Mac, can't be shared via Finder. There is no possibility to set a tick (see picture below).

    Bildschirmfoto 2014-01-29 um 21.11.20.png

     

    On each HOST (1.1.1.1 and 1.1.1.2) for each directory (folder) that you want to share in the cluster (between these two hosts including the /compressor/storage etc)

    I am a bit bemuddled as you write "host" and give both the IPs 1.1.1.1 and 1.1.1.2. I think the host is the master with 1.1.1.1, which distributes the job, and the other computer (the slave) has the 1.1.1.2, but is no host.

     

    One more thing. I opened the info dialog windows of the Storage folders of the master from the slave (wow, English is somewhat limited sometimes). I mean, I mounted the Storage folders of the master computer on the slave. The permissions look odd: the group "everyone" has no rights on the slave, although I set it to read&write on the master. Could this be the culprit?

     

    info on iMac:

    Storage iMac.png

     

    info on MBP:

    Storage MBP.png

     

    By the way. Apple support suggested to put the "source" and "finished" folders onto an external volume so that every computer has to mount the volume externaly. In addition I should try to mount the network with smb. Both things, also in combination, don't work. The only difference I can observe is a brief increase of processor power by the curl process. I asked the technician to have a look into this thread and I think he/she did. Hopefully we can solve this serious issue. Network transcoding with Compressor is far away from being user friendly. Thank you again for your endurance Warwick.

  • by Steffen Bendix,Helpful

    Steffen Bendix Steffen Bendix Jan 29, 2014 12:54 PM in response to Steffen Bendix
    Level 1 (60 points)
    Mac OS X
    Jan 29, 2014 12:54 PM in response to Steffen Bendix

    I could not alter my last passage because the Apple forum suddenly presented the "Back soon" yellow postit. The suggestion by Apple works! Running the distributed job from an external volume works. Now we have to find out why it does not from an internal one.

  • by Steffen Bendix,

    Steffen Bendix Steffen Bendix Jan 31, 2014 6:09 AM in response to Steffen Bendix
    Level 1 (60 points)
    Mac OS X
    Jan 31, 2014 6:09 AM in response to Steffen Bendix

    I tested the setup with different video files and codecs and there is another issue: All node computers (I used two) work at their specific work piece but when the last piece is almost finished (approx. 80%) the last computer finishes its work. The final movie can't be completed by the master computer. The individual work pieces remain uncombined.

  • by Warwick Teale,

    Warwick Teale Warwick Teale Feb 6, 2014 2:39 AM in response to Steffen Bendix
    Level 3 (588 points)
    Feb 6, 2014 2:39 AM in response to Steffen Bendix

    Hi Steffen, looks like you have had some success.

     

    Franky ghat suggestion from Apple as well meaning as it might seem is nonsense. By default if one was to use the ~/Movies directory as the examples Apple has in manuals from the past then their own advice would be incorrect.

     

    I believe you just have a simple setup issue with SHARING your file systems.

     

    Anywy you have it working by the sounds of things.

     

    Good work

     

    W

Previous Page 2 of 3 last Next