Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Compressor 4.1.1 fails to Cancel a batch process

I sent a bacth of 6 files from FCP X 10.1.1 to Compressor using a Custom Setting to transcode Apple ProRes 422 1920 x 1080 to x264 640 x 480. The first file failed showing "FAILED: 3x crash service down" in the status column. I attempted to cancel the entire group of files and today, after nearly 24 hours, the status of the projects remains "Canceling". I've run through the normal troubleshooting, but nothing seems to work. (ie. restarting the computer, restting permissions, etc.)


Grateful for any advise!

MacBook Pro, OS X Mavericks (10.9.2)

Posted on Feb 28, 2014 9:55 AM

Reply
Question marked as Best reply

Posted on Feb 28, 2014 9:42 PM

I assume Compressor > Preferences > Advanced > Reset Queue (cancel jobs) isn't working? I have found a couple of ways to canel stubborn batches. Easiest way would be to remove ~/Library/Application Support/Compressor/Storage and ~/Library/Application Support/Compressor/History, then restart.


If you want to preserve your history and settings, like most people, you can delete ~/Library/Application Support/Compressor/Storage, then go into ~/Library/Application Support/Compressor/History and delete the one that is giving you the problem. Most likely the last one. Then you can either restart or kill all Compressor processes (ex: "for pid in `ps -ax | grep -i compressor | awk '{print $1}'`; do kill -9 $pid; done"). If you dont know how to use terminal, just restart. As always YMMV, but either method should work.

32 replies
Question marked as Best reply

Feb 28, 2014 9:42 PM in response to Hogrider03

I assume Compressor > Preferences > Advanced > Reset Queue (cancel jobs) isn't working? I have found a couple of ways to canel stubborn batches. Easiest way would be to remove ~/Library/Application Support/Compressor/Storage and ~/Library/Application Support/Compressor/History, then restart.


If you want to preserve your history and settings, like most people, you can delete ~/Library/Application Support/Compressor/Storage, then go into ~/Library/Application Support/Compressor/History and delete the one that is giving you the problem. Most likely the last one. Then you can either restart or kill all Compressor processes (ex: "for pid in `ps -ax | grep -i compressor | awk '{print $1}'`; do kill -9 $pid; done"). If you dont know how to use terminal, just restart. As always YMMV, but either method should work.

Mar 1, 2014 12:05 AM in response to AnonimaKontonWTF

My method for killing all the Compressor processes assumes nothing else is named Compressor. I cant think of a practical reason why this would be a problem, but there is the possibility of some weird scenario where you have a user named compressor, or some important app with that name in its path, like "/Applications/Audio Compressor.app" or something. I guess this is my way of saying be careful, and if you dont understand what its doing, it might be better to just restart.

Mar 1, 2014 3:28 AM in response to Hogrider03

Hi, this Compressor.app V4.x symptom is VERY VERY simple to fix and is due the residual state information maintained in the Qmaster cluster structures for the failed job(s). This information was not or cannot be cleaned up by the QMaster termination substasks error recovery nor job termination nor manually through intervention through the Compressor.app V4.1.x UI (compressor.appV4.1.x / preferences/advanced/ 'reset queue') mentioned by other posters. The latter, which, attempts to QUIESCE and subsequently remove this state information I mentioned earlier.


So before just going in and mindlessly blasting away the compressor.v4.1x and qmaster structures and parmlists and preferences or using 3rd part .app removers , this symptom and fix (workaround) deserves some explanation so that others who seems to have these experiences can understand and fix it themselves.


As I've mentioned many times on this and other forums, blasting away this state information can be very quite disruptive if you have other jobs and like us, other clusters we use in our workflow.

be mindful of templates, droplets, settings, definitions/templates, restart/redo history, other active jobs and agan of OBLITERATING the Compressor.app V4.1 x HISTORY that is EXTREMELY useful for a RESTART or REDO (see the completed paned in the UI)

Thus being selective and surgical (and smarter) helps to maintain job deadlines and won't upset colleagues. 😉


EXPLANATION Qmaster Cluster Job state Information in Compressor.app V4.1.x (post Dec 2013):

Information for ALL jobs that are QUEUED UP, ACTIVE (and stalled - waiting) or Completed (successfully or not) are maintained for each defined cluster (distributed or not). YOU may define many clusters and therefore any host (node or service nodes) MAC that is part of the distributed cluster will have these structures. In larger workplaces, obviously not all macs (hosts aka service nodes) belong to the same cluster in either or both of two places on each host (service node):

  1. your home directory) ~/Library/Application Support/Compressor/Storage/aaaaaaaaa-bbbbbbbb/ or xxxxxxxx-yyyyyyyyy etc etc
    • where aaaaaaaaa-bbbbbbbb or or xxxxxxxx-yyyyyyyyy represents a CLUSTER you defined or had by default this computer) .. an example is:
    • ~/Library/Application Support/Compressor/Storage/68ED8F60-8936DA51

  2. and /Users/Shared/Library/Application Support/Compressor/Storage/aaaaaaaaa-bbbbbbbb/ or xxxxxxxx-yyyyyyyyy on the USUALLY however not always.host that is the cluster controller (where the cluster ws defined)
  • where aaaaaaaaa-bbbbbbbb or or xxxxxxxx-yyyyyyyyy represent a CLUSTER .. an example is:
  • /Users/Shared/Library/Application Support/Compressor/Storage/68ED8F60-8936DA51

  • The objects in these structures reveal active jobs, what's shared (distributed) and other information and some HIDDEN files. It's the hidden files. you need to clean up these little buggers as well. Here's an example ;


    mavericks-server:~ warwick$ ls -1aR ~/Library/Application\ Support/Compressor/Storage/68ED8F60-8936DA51/jobs

    .

    ..

    .qmaster.plist <------------------

    directory

    history

    requests

    mavericks-server:~ warwick$


    If you delete these objects, your compressor.app V4.1.x app and the Shared Monitor.app UI status will show nothing is active... which is what you have asked to do.


    Residual from Compressor.app V4.1.x 'Active" and Shared Montor.app residual information.

    Well ...if you look at the:

    • system via Activity Monitor.app (filter compressord) or ps, top etc, you will see that it's highly likely that the compressord subtasks (depending on the instances you have started) are not active (idle) and nothing related to the Compressor batch for those jobs is actually running!
    • You can also review the STOMP transcoder logs for each instance of compressor on each host in ~/librarylogs/compressor (or use Console.app). You'll note failures in there that are very easily seen... all is revealed


    These two UI interfaces seem to retrieve this STATE information from the above.


    SO.. here's what you might try. - delete some of these structure using the unix rm command. You may not have success using the Finder / trash because the empty of the trash may fail because these objects are enqued / serialised via the file system and wont be deleted until closed through normal or abnormal means or a restart. (again very disruptive). I recomend using the unix rm command. Step 1 is optional.


    Simply on each host where this symptom exists (hosts = service nodes = comprise a cluster)

    STEP 1: (OPTIONAL - for elegance and safety and disruption too only).. I don't do it .. your call.

    In Compressor.app V4.1.x UI / preferences/ my computer/ "Allow other computers.... on my computer" - slide = OFF

    User uploaded file

    Quit Compressor.app V4.1.x (shut it down)

    STEP 2: use the osx finder to NAVIGATE to your home directory library (+shift+g) and enter ~/library/application support/compressor/storage

  • note the cluster directory there. for example: ls -a1 ~/Library/Application\ Support/Compressor/Storage/68ED8F60-8936DA51
  • .

    ..

    .DS_Store

    jobs

    shared

    mavericks-server:~ warwick$

  • delete these two folders "jobs" and "shared" and anything else and their contents in one go. Launch the terminal.app and enter the the text below in green (make sure there's no blanks and include the asterisk '*' on the end where xxxxxxxx-yyyyyyyyyy is the cluster folder(s) you see in this structure..:
  • rm -rf ~/Library/Application\ Support/Compressor/Storage/xxxxxxxx-yyyyyyyy/* and press enter...
  • optionally repeat as above for other clusters. While your there remove any dead cluster storage directories (check the dates..)
  • STEP 3: use the finder to NAVIGATE to the SHARED home directory library (+shift+g) and enter /users/shared/library/application support/compressor/storage. There may be nothing there is this a host/ service node. To avoid further verbosity......

    • perform the same steps as in step 1. by examining the contents of the users/shared/library/application support/compressor/storage directory.
    • use the terminal command to selective remove the contents of the active clusters storages as:

      rm -rf /users/shared/library/application\ support/compressor/storage/xxxxxxxx-yyyyyyyy/* with care and discern as above

    • repeat as above for other clusters. While your there remove any dead cluster storages (check the dates..)

    STEP 4: (Optional as a result of Step 1 only if you did it...) on each host in the cluster, launch/start Compressor.app V4.1.x and

    • reinstate the cluster sharing
    • check you clusters in preferences/shared computers
    • check all the SHARED/MOUNTED file systems on each hosts that they are correctly mounted, available, accessible and authorised!



    Summary:

    • Although this procedure appears quite verbose and complex, it is not.
    • Knowing what to surgically remove and WHY is important in knowing how to maintain the transcoding system and make it stable and reliable and at utmost, USABLE.
    • Thus, there no need to mindlessly BLAST away 😠 the compressor.app V4.1.a app structure (its setting/parms etc) and nor disrupt others, work in progress in the process as well as yourself.


    HTH

    Post your results for others to see.


    Warwick

    Hong Kong

    Mar 1, 2014 9:39 AM in response to AnonimaKontonWTF

    First, I want to thank you for your thoughtful responses. I was so hopeful when I started down the path you describe, but unfortunately, I found no folders/files related to Compressor in the directory path ~/Library/Application Support/Compressor


    I don't have any other systems on my network so am stuck with doing all my editing and Compressor tasks on my MacBook Pro.


    I have a deadline coming up and these transcode tasks have to be completed. 😕

    Mar 1, 2014 10:44 AM in response to Hogrider03

    Hogrider03 wrote:


    I found no folders/files related to Compressor in the directory path ~/Library/Application Support/Compressor



    Are you sure about this? I dont want to seem patronizing, but the "~" means your home directory. So /Users/<your user name/Library/Application Support (unless using a network home directory). You must have that directory, or else I dont think you could ever submit a batch or create a custom setting.


    If you definitely dont have that directory, that would definitely be a problem. There are logs in ~/Library/Logs/Compressor and /var/log/system.log that might give a hint to the issue.

    Mar 1, 2014 12:52 PM in response to AnonimaKontonWTF

    I understand your point. When I didn't find the Compressor files earlier, I thought the same thing. The only place I have a Library file is in the home directory. I searched in the User Name directory and then searched the entire system looking for those files. Even now, I don't seem to have them. The Logs folder has no files referencing Compressor either. 😐


    Since my last reply, I cleared (deleted) the older version of Compressor from Final Cut Studio and rebooted the system twice. Then, on relaunching Compressor 4.1.1, the batch I was trying to get rid of was finally gone! So I decided to try transcoding just one of the projects that was in the original batch and it seems to be progressing OK, but really, really slow... It's a 13 min. 1080p Pro Res source converting to QT x264 640 x 480 "proof" copy (my original goal). After churning for more than 2 hrs. 45 min. its only about 2/3rds finished.

    Mar 1, 2014 2:07 PM in response to Hogrider03

    Hogrider03 wrote:


    I understand your point. When I didn't find the Compressor files earlier, I thought the same thing. The only place I have a Library file is in the home directory. I searched in the User Name directory and then searched the entire system looking for those files. Even now, I don't seem to have them. The Logs folder has no files referencing Compressor either. 😐


    Since my last reply, I cleared (deleted) the older version of Compressor from Final Cut Studio and rebooted the system twice. Then, on relaunching Compressor 4.1.1, the batch I was trying to get rid of was finally gone! So I decided to try transcoding just one of the projects that was in the original batch and it seems to be progressing OK, but really, really slow... It's a 13 min. 1080p Pro Res source converting to QT x264 640 x 480 "proof" copy (my original goal). After churning for more than 2 hrs. 45 min. its only about 2/3rds finished.

    You could have had an older version of the Compressor services running. Did you have an Apple Qmaster folder in ~/Library/Application Support? And I take it you still dont have the Compressor folder? Thats strange that you could create settings and submit batches if you didnt have that Compressor directory.


    About the slow encoding...are you exporting from Final Cut? If you did a Send to Compressor, it will be slow. Its better to export to a Master file, then use that in Compressor. Or use a Compressor Setting in Final Cut.

    Mar 1, 2014 2:15 PM in response to AnonimaKontonWTF

    I do have an Apple Qmaster folder.


    Still no Compressor folder.


    I have been sending the project to Compressor from FCP, but will try exporting Master files.


    After more than three hours of processing in Compressor, my latest test resulting in "Failed: 3x crash service down" in the Status column.


    To be clear, Compressor has worked fine for me for previous attempts. Both with this project and others.

    Mar 1, 2014 4:09 PM in response to Hogrider03

    Latest Update:

    I've exported "Masters" from FCP X and have run transcode tests which have all failed. ("Failed: 3x crash service down" status)


    It seems to me that there must be some corruption with this copy of the applicaiton. I'm at a point where I'm about to trash the application, download and reinstall it.


    Any thoughts? (does anyone from Apple monitor these discussions????)

    Mar 1, 2014 10:25 PM in response to Hogrider03

    Hi, a quick read of your posts "sounds like " you have a job deadline and with no time and a learning curve all not on your side.


    If you simply need to make one or more distributions and you are using FCPX , use SHARE facility from fcpx and avoid compressor.app?


    The error status "failed 3x service down" is a Qmaster error btw. The structure present is buggered and there is a great deal of tips and hints on this forum to fix or work around it. Again there will be logs of this event in places mention by posts on thus thread and by me. Worth posting them.


    You mentioned re- installation..really? This would imply that the app was corrupt. By all means try it but prepare for disappointment.


    Should you want to pursue compressor.app, would you be able to FURNISH SOME DIAGNOSTIC INFORMATION in subsequent? Else any assistance you get from this forum will continue to be tried measures and well meaning speculation!


    More from your posts indicate you may have fragments of old legacy fcp7 around or dead structures from compressor v3.x or v4.0...


    As others have pointed out you should at least haven't hear basic structures in the libraries ~/library/application support/compressor for v4.1. Other posts on this thread ponder if you have actually found the correct paths. I too am surprised that these structures don't exist. Perhaps you might furnish a ls -a1 of the "application support" in both your own home directory and /users/shared/library


    Further Diagnosis?

    Further start by turning off all compressor.app + Qmaster facilities and . In short disable all the options in compressor. Turn them off


    Delete any structures that contain the name "apple Qmaster" or "Qmaster"...

    Look here: and care with blanks .. Use the finder cmd+shift+g

    1. /var/spool/

    2. Your home directory ~/library/application support/

    3. /users/shared/library/application support/


    Either rm them or use rubbish (trash)


    This will remove state info of old and logs of old.


    Restart your mac n log in


    Delete the trash


    Start compressor.app v4.1 and ENABLE any option such as

    allow batched on my computer - ON

    FILE SHARING


    Use the finder to verify you have these structures!


    ~/library/application support/compressor


    /users/shared/library/application support/compressor


    ~/library/logs/compressor


    Post your results for others to see


    Warwick

    Hong hong

    Mar 2, 2014 2:15 AM in response to Hogrider03

    Hogrider03 wrote:


    Latest Update:

    I've exported "Masters" from FCP X and have run transcode tests which have all failed. ("Failed: 3x crash service down" status)


    It seems to me that there must be some corruption with this copy of the applicaiton. I'm at a point where I'm about to trash the application, download and reinstall it.


    Any thoughts? (does anyone from Apple monitor these discussions????)

    Well, reinstalling couldnt hurt. Be sure to search for Compressor in Spotlight, and delete any copies you might find. Maybe make a note of the different locations. Then just install from the app store. The first step to getting you working is getting the ~/Library/Application Support/Compressor folder created. This should be created when you open Compressor.

    Mar 3, 2014 9:09 AM in response to AnonimaKontonWTF

    I'm sharing this for the benefit of others who may be struggling with this problem.


    So it appears that Apple, in one of their recent OS updates, has determined that Users can't be trusted with access to the ~ /Library folder. As previously noted, searching the system for occurances of Compressor only yeilded results for the application itself and no results for Compressor in the Library/Applicaiton Support. A subsequent search for "Library" revealed that the ~/Library folder existed, but was hidden from routine access.


    To find that hidden folder from the finder select the "Go" Menu item while holding down the Option Key. This will reveal the ~/Library folder as an option which can then be selected. And sure enough, Compressor is available in the Applicaiton Support folder. Once there, the previous instructions about clearing the History and Storage folders can be executed.

    Compressor 4.1.1 fails to Cancel a batch process

    Welcome to Apple Support Community
    A forum where Apple customers help each other with their products. Get started with your Apple ID.