sherap

Q: Archiving in H264 while keeping folders/file creation date

*Hi, I just wanted to share with you a script that helps to convert the content of the FCP "capture scratch" folder into H264, to move the converted files into a Device, put the h264 files in the correct client folder.*
*Into this client folder, it creates folders with the original production name, while it keeps the original creation file date.*

*So it keeps the original folder hierarchy and date for my archives, because to browse with the final cut server client is great, but sometimes you just want navigate in your file system to work with your videos.*

*STEP 1* : Original Assets Cataloging (production scan)

I have a Scheduled Production scan of my "capture scratch",
(Capture scratch is on our SAN, so all Edit suites are capturing in the same "capture scratch" folder)

*STEP 2*: Detect to which client the asset belongs to (thanks to the location name) and set the proper [Owner ]metadata asset
NB: for that I ask all editors to put the name of the client at the beginning of the name of their FCP projects. ex CBS_snow in paris.fcp

I have Subscriptions that filters created assets location to see if the location "begins with" the client name, then give a "response" that set the correct [Owner] metadata, and also change the "statut" to "WAITINGFORENCODE" (you have to create it into Lookup/Status)

So I have as many Subscriptions/Reponse as actual Clients

*STEP 3 :* I have a subscription that filters Modified Assets that have a status of "WAITINGFORENCODE"
then gives 3 responses.


+Response 1:+ Set Asset Metadate Status to "ENCODING" ( yeah, Status are made for collaborating tags.. but here we are...)

+Response 2:+ Copy in H264 (that's where Matrox is great!) into a "transcode folder"

+Response 3+ (the tough one): Copy the transcoded asset to the right archive location and keep original creation date, create an asset and put it in the same production as the original, and set the original Asset Metatadata Status as "ENCODED"

Run Script

Command Path: //path to the script.
In the command parameters you'll need to enter: [File name] [Location] [Owner] [Asset ID]


So here is the Script. It's a LOT inspired from Andy Sykes http://fcsvoodoo.blogspot.com/2009/07/add-newly-created-asset-to-originating.htm l
A big thank to him for publishing it.





#!/bin/bash

FILENAME=$1
PLACE=$2
CLIENT=$3
IDFILE=$4

FILENAME_ESCAPED=$(echo -n "$FILENAME" | \
perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');


LOCATION_ESCAPED=$(echo -n "$PLACE" | \
perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');


# Set the location to the fcsvr_client binary
PATH_TO_BINARIES="/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/MacOS/fcsvr_client"

# Set the location of the Archives NAS Client Folder
PATH_TO_TARGET="//Volumes/NASARCHIVES/""$CLIENT""/Footages"

# Set the location of the Transcoding folder
PATH_TO_TRANSCODE="//Volumes/EditSAN/Transcode"

# Set the location of the Original File folder
PATH_TO_ORIGINAL="//Volumes/EditSAN/Capture Scratch/"


# Make a folder and move the file in it
mkdir "$PATH_TO_TARGET""$PLACE"

mv "$PATH_TO_TRANSCODE"/"$FILENAME" "$PATH_TO_TARGET""$PLACE"/

#Set Copied File and Folder Date from original
ORIGDATE=$(GetFileInfo -d "$PATH_TO_ORIGINAL""$PLACE"/"$FILENAME")
ORIGDATEFOLDER=$(GetFileInfo -d "$PATH_TO_ORIGINAL""$PLACE")
echo $ORIGDATE
echo $ORIGDATEFOLDER

Setfile -d "$ORIGDATE" "$PATH_TO_TARGET""$PLACE"/"$FILENAME"
Setfile -d "$ORIGDATEFOLDER" "$PATH_TO_TARGET""$PLACE"


# Determine original Production "address" by searching for the "location" (since this is the same as the Production name)
PROJECTNAME=$("$PATH_TO_BINARIES" search --crit "$PLACE" /project | grep -i -o -e "/project/[0-9]*")
echo $PROJECTNAME


# Create asset and associate it with the project

OUTPUT=$(sudo "$PATH_TO_BINARIES" createasset pa_asset_actua --projaddr $PROJECTNAME /dev/11/"$CLIENT""/Footages""$LOCATION_ESCAPED"/"$FILENAME_ESCAPED" CUST_ASSET_STATUS="INRAID6")
echo $OUTPUT


# Change Original metada asset status
SETMETA=$(sudo "$PATH_TO_BINARIES" setmd /asset/$IDFILE CUST_ASSET_STATUS="TRANSCODED")
echo $SETMETA




________________
Weekness of the script is when you're converting non .MOV files (or different extension in the H264 copy than the original), I could not figure out how Andy Skypes managed with the extensions in his script, so I just pass it.

Now it's working like a charm after I had a lot of Issues due to the escape characters, but thanks the the perl command ,it's good now. But you will still need to ask editors to avoid / and & and () in file names...

Well, I just make it work yesterday, so now I breath cause I couldn't figure out if investing if FCS would be more a sufferance than a relieve because lot of people saying that you have to totally rethink the way you work with assets, and forget about your folder hierarchy...but this, I couldn't hear It, especially when I saw the mess that was about to happen with thousands of P2 clips.


My conclusion would be: Metadatata is great, but folder hierarchy can save you if you loose your DB or want to migrate ...

au revoir.

Mac OS X (10.5.8), Final cut server

Posted on Jan 22, 2011 3:33 AM

Close

Q: Archiving in H264 while keeping folders/file creation date

  • All replies
  • Helpful answers

  • by A. Richards,

    A. Richards A. Richards Jan 22, 2011 6:57 PM in response to sherap
    Level 3 (625 points)
    Jan 22, 2011 6:57 PM in response to sherap
    Tip:

    Post your code using these formatting tags:


    Paste your code here.


    Which does this (including the colors):

    #! /bin/bash

    GREETING='Hello world!'

    echo "$GREETING"

    exit 0


    That way Apple's forums won't parse your code for formatting like strikethroughs, and we'll all be able to read your code properly.
  • by sherap,

    sherap sherap Jan 23, 2011 4:43 AM in response to A. Richards
    Level 1 (0 points)
    Jan 23, 2011 4:43 AM in response to A. Richards
    here we go



    #!/bin/bash

    FILENAME=$1
    PLACE=$2
    CLIENT=$3
    IDFILE=$4

    FILENAME_ESCAPED=$(echo -n "$FILENAME" |
    perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');


    LOCATION_ESCAPED=$(echo -n "$PLACE" |
    perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');


    # Set the location to the fcsvr_client binary
    PATH_TO_BINARIES="/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/MacOS/fcsvr_client"

    # Set the location of the Archives NAS Client Folder
    PATH_TO_TARGET="//Volumes/ARCHIVES/""$CLIENT""/Footages"

    # Set the location of the Transcoding folder
    PATH_TO_TRANSCODE="//Volumes/EditSAN/Transcode"

    # Set the location of the Original File folder
    PATH_TO_ORIGINAL="//Volumes/EditSAN/Capture Scratch/"


    # Make a folder and move the file in it
    mkdir "$PATH_TO_TARGET""$PLACE"

    mv "$PATH_TO_TRANSCODE"/"$FILENAME" "$PATH_TO_TARGET""$PLACE"/

    #Set Copied File and Folder Date from original
    ORIGDATE=$(GetFileInfo -d "$PATH_TO_ORIGINAL""$PLACE"/"$FILENAME")
    ORIGDATEFOLDER=$(GetFileInfo -d "$PATH_TO_ORIGINAL""$PLACE")
    echo $ORIGDATE
    echo $ORIGDATEFOLDER

    Setfile -d "$ORIGDATE" "$PATH_TO_TARGET""$PLACE"/"$FILENAME"
    Setfile -d "$ORIGDATEFOLDER" "$PATH_TO_TARGET""$PLACE"


    # Determine original Production "address" by searching for the "location" (since this is the same as the Production name)
    PROJECTNAME=$("$PATH_TO_BINARIES" search --crit "$PLACE" /project | grep -i -o -e "/project/[0-9]*")
    echo $PROJECTNAME


    # Create asset and associate it with the project

    OUTPUT=$(sudo "$PATH_TO_BINARIES" createasset pa_asset_actua --projaddr $PROJECTNAME /dev/11/"$CLIENT""/Footages""$LOCATION_ESCAPED"/"$FILENAME_ESCAPED" CUST_ASSET_STATUS="INRAID6")
    echo $OUTPUT


    # Change Original metada asset status
    SETMETA=$(sudo "$PATH_TO_BINARIES" setmd /asset/$IDFILE CUST_ASSET_STATUS="TRANSCODED")
    echo $SETMETA


  • by BenB,

    BenB BenB Jan 31, 2011 7:52 AM in response to sherap
    Level 6 (10,036 points)
    Audio
    Jan 31, 2011 7:52 AM in response to sherap
    Physical folders defeat the whole purpose of FCSvr.
  • by John F. Whitehead,

    John F. Whitehead John F. Whitehead Jan 31, 2011 9:09 PM in response to BenB
    Level 2 (380 points)
    Jan 31, 2011 9:09 PM in response to BenB
    Yes... though I could understand the argument of wanting to preserve the Capture Scratch directory structure so you could easily reconnect a FCP project file by pointing it to the H.264 directory.

    But why do it with scripts, which are brittle? Why not do it from within FCSvr by calling external commands?

    Or, why not just create Edit Proxies in H.264 upon project check-in -- then you can just archive your original media and you don't have to do anything else.
  • by nylonoxygen77,

    nylonoxygen77 nylonoxygen77 Feb 1, 2011 8:56 AM in response to sherap
    Level 1 (30 points)
    Feb 1, 2011 8:56 AM in response to sherap
    Thanks for sharing! Good job with escaping the path to convert it to a FCSvr representation address. Unfortunately, FCSvr doesn't subscribe to basic percent-encoding standards. It lets a lot of characters through without escaping. Your perl one-liner as it stands would encode everything but A-Za-z0-9. FCSvr lets other characters through !,;+=*&$'"/() without encoding. I ended up creating a function that seems to work. This escapes all the characters (that I know of) that FCSvr will allow to pass through. You can use this function standalone or pipe output from another command to it.


    percentEncode () {
    if [[ "$1" ]]; then
    encoded(echo -n "$1" |
    perl -pe 's/([^-_.~A-Za-z0-9!.,;=+*&$'\''/()])/sprintf("%%%02X", ord($1))/seg');
    echo $encoded
    else
    while read Input; do
    encoded(echo -n "$Input" |
    perl -pe 's/([^-_.~A-Za-z0-9!.,;=+*&$'\''/()])/sprintf("%%%02X", ord($1))/seg');
    echo $encoded
    done
    fi
    }


    so you can use it like this:

    percentEncode "$Path"

    Or this:

    echo "$Path" | percentEncode

    Use at your own risk.
  • by nylonoxygen77,

    nylonoxygen77 nylonoxygen77 Feb 1, 2011 9:01 AM in response to BenB
    Level 1 (30 points)
    Feb 1, 2011 9:01 AM in response to BenB
    BenB wrote:
    Physical folders defeat the whole purpose of FCSvr.


    FCSvr doesn't care where things are stored, but remember that path information is converted to searchable metadata, which IS a huge part of the point of FCSvr.
  • by John F. Whitehead,

    John F. Whitehead John F. Whitehead Feb 1, 2011 11:31 AM in response to nylonoxygen77
    Level 2 (380 points)
    Feb 1, 2011 11:31 AM in response to nylonoxygen77
    nylonoxygen77 wrote:
    FCSvr doesn't care where things are stored, but remember that path information is converted to searchable metadata, which IS a huge part of the point of FCSvr.


    But if you're using a file path on your disk to represent metadata, then you're not taking advantage of the power of FCSvr.

    I.e., don't rely on a path to represent a project name, or date, or anything else unrelated to its location in the filesystem. Instead, create custom metadata fields to store that info.
  • by sherap,

    sherap sherap Feb 1, 2011 12:11 PM in response to John F. Whitehead
    Level 1 (0 points)
    Feb 1, 2011 12:11 PM in response to John F. Whitehead
    Hey thanks so much for the missing escaped chr... it will help a lot avoid annoying crashs


    John F. Whitehead wrote:
    But if you're using a file path on your disk to represent metadata, then you're not taking advantage of the power of FCSvr.


    It's seems not going inconsistent dealing with folders and FCsvr to organise your data, especially when it comes to convert a whole capture scratch P2 assets directory..
    Even with a daily saved database, the day while come when seeing the coherence of your productions also in the file system might help a lot to relink Offlines or even survive to a server crash... who knows...
  • by John F. Whitehead,

    John F. Whitehead John F. Whitehead Feb 1, 2011 12:34 PM in response to sherap
    Level 2 (380 points)
    Feb 1, 2011 12:34 PM in response to sherap
    I'm not saying don't use folders.

    nylonoxygen77 said that an advantage of having folders is that they are represented as metadata that you can search. It sounds like he is implying there is something useful in the path names, such as a project name, or date, or some other piece of metadata.

    Sure that can be helpful sometimes, but a more powerful approach is to assign the project name into a Project Name metadata field, the date into a Date field, etc.

    Relying on overloading the filename and pathname as you would in the Finder before the days of Final Cut Server misses an opportunity you have with a real database at your disposal.

    Just to be clear: I use folders with descriptive names for my clients all the time. I just put that metadata info into custom fields so it can be more useful to Final Cut Server users.