essandess

Q: EyeTV with Commercial Skipping HD DVR Server

I use EyeTV HD and Turbo.264 HD to record and transcode h.264 shows for playback on a Mac, Apple TV, iPad, iPhone. I also use a self-modified version of etvcomskip to detect and mark commercials, which are then automatically skipped at playback. This makes tv viewing a lot more pleasurable and efficient, and is especially helpful controlling advertiser access to children. In case this is useful to others, I've outlined the steps here:

 

  1. Eric Kaashoek's website Comskip and download his HD-capable Comskip software. "Commercial" commercial-skipping companies (ReplayTV, TiVo, Dish, etc.) are quickly sued out of existence or into submission, even if they simply provide a basic "30-second skip" feature. Comskip is open source beerware that you can download, install, and use yourself, and you can't sue anyone for publishing source code. And rather than providing a simple 30-second skip feature, it detects, marks, and provides that capability for automatic commercial skipping.
  2. Comskip a Windows executable, so you'll need WINE (WINdows Emulator) on OS X to run it. First, download and install Xcode (free from the Apple App store), then download and install and install Macports, then in Terminal, run "sudo port selfupdate; sudo port install wine-devel". Finally, make sure that /Applications/Utilities/X11.app is added to your list of applications to run at login (WINE uses X11 to display Windows windows). Yes, this is windows emulation, but on a half-decent box you'll see that commercial skipping runs at around twice real-time for h.264 HD recordings.
  3. Download and install etvcomskip (and PyeTV if you want EyeTV to work with Front Row). The 2010 version of etvcomskip doesn't work with Comskip's latest HD capabilities, and the latest Comskip mpeg libraries do not allow live tv processing, so install these files to get everything working. Once you done this, you'll have a commercial-skipping HD DVR box on your Mac that can stream to Apple TV, iOS, or export to iTunes.

Comskip software

This lives in the directory /Library/Application Support/ETVcomskip.

 

Edit the file ./comskip/comskip.ini to use these HD-tuned settings, tailored for US cable:

 

detect_method=127;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
max_avg_brightness=21;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
non_uniformity=50; Set to 0 to disable cutpoints based on uniform frames
max_volume=20; any frame with sound volume larger than this will not be regarded as black frame
logo_at_bottom=1; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
intelligent_brightness=1 ; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA

volume_slip=150

output_dvrmstb=0; Set to 1 if you're running DVRMS-Toolbox
live_tv=0; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4; change to 16 when using live_tv in BTV, used for mpeg PS and TS

 

Edit the file ./MarkCommercials.app/Contents/Resources/MarkCommercials.py to use WINE to run comskip.exe nicely. Grep for these changes:

 

EyeTV.launch()

 

msg = 'Error: unable to communicate with %s\n' % options.app

 

#cmd = '"/Library/Application Support/ETVComskip/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile

# MacPorts 64-bit wine

#cmd = '"/Applications/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile

cmd = '"/opt/local/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile

 

# nice the wine command

cmd = "/usr/bin/nice -n 14 " + cmd

 

EyeTV software

EyeTV uses the triggered AppleScripts RecordingStarted and RecordingDone in the directory /Library/Application\ Support/EyeTV/Scripts/TriggeredScripts. Open these using the AppleScript Editor and use these scripts instead. You will probably have to Save As... on your desktop, then copy the updated script back into the correct directory to avoid permissions issues.

 

RecordingStarted.scpt

 

on RecordingStarted(recordingID)

    delay 10

    -- comskip81 uses ffmpeg and does not support live tv; put this in RecordingDone.scpt

    --    set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"

    -- display dialog cmd

    -- set cmd to "env > /tmp/etv_test.log &"

    -- do shell script cmd

 

    --disable this if you do not want a logfile written

    write_to_file((short date string of (current date) & " " & time string of (current date)) & "Recording Started run for ID: " & recordingID & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)

end RecordingStarted

 

on write_to_file(this_data, target_file, append_data)

    --from http://www.apple.com/applescript/sbrt/sbrt-09.html

    try

        set the target_file to the target_file as string

        set the open_target_file to open for access file target_file with write permission

        if append_data is false then set eof of the open_target_file to 0

        write this_data to the open_target_file starting at eof

        close access the open_target_file

        return true

    on error

        try

            close access file target_file

        end try

        return false

    end try

end write_to_file

RecordingDone.scpt

 

-- Run the python MarkCommercials script for the given recording

-- this must be run with the RecordingStarted script

-- it will check if there were multiple PIDs for the recording and runs MarkCommercials for each pid

-- requires updated MarkCommercials which allows specifying the pid

-- by Ben Blake, September 2009

 

-- modified for latest Comskip, which cannot be run until after recording is finished; waits for Turbo.264 HD to stop running as well

-- S.T.Smith

 

global LogMsg

 

on RecordingDone(recordingID)

 

    set ProcessName to "Elgato H.264 Decoder"

    set DelayTime to 60

    set MaxDelays to 4 * 60 -- four hours

 

    -- delay until the process slows down or stops running

    repeat MaxDelays times

        delay DelayTime -- delay at least once to give it a chance to start

        set pcpu to CPUPercentage(ProcessName)

        if pcpu is equal to "" or pcpu < 2.0 then exit repeat -- break out of delay loop if it's not running

    end repeat

 

    delay 10

    -- comskip81 uses ffmpeg and does not support live tv; take this from RecordingStarted.scpt

    set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"

    -- display dialog cmd

    -- set cmd to "env > /tmp/etv_test.log &"

    do shell script cmd

 

    set LogMsg to ""

    CheckMultiplePIDs(recordingID)

 

    --disable this if you do not want a logfile written

    if (count of LogMsg) > 0 then

        write_to_file((short date string of (current date) & " " & time string of (current date)) & LogMsg & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)

    end if

end RecordingDone

 

-- testing code: this will not be called when triggered from EyeTV, but only when the script is run as a stand-alone script

on run

    tell application "EyeTV.app"

        set rec to «class Unqu» of item 1 of every «class cRec»

 

        my RecordingDone(rec)

    end tell

end run

 

-- compute the percentage CPU used by ProcessName

on CPUPercentage(ProcessName)

    set ProcessPS to do shell script ("ps -axwwc | grep '" & ProcessName & "' | grep -v grep || true")

    if ProcessPS is not equal to "" then

        set ProcessID to word 1 of ProcessPS

        set ProcessPS to do shell script ("ps -xwwco pid,ppid,%cpu -p " & ProcessID & " | tail -1")

        set ProcessCPU to word 3 of ProcessPS

        return ProcessCPU as number

    else

        return ""

    end if

end CPUPercentage

 

on CheckMultiplePIDs(recordingID)

    --check if there are multiple Video PIDs in the file

 

    tell application "EyeTV.app"

        set input_text to my read_from_file((path to "logs" as string) & "ETVComskip" & ":" & recordingID & "_comskip.log")

        if (count of (input_text as string)) > 0 then

            set logdata to every paragraph of input_text

            set logdata_lastrow to (item ((count of logdata) - 1) of logdata) as string

 

            if (items 1 thru 19 of logdata_lastrow) as string = "Video PID not found" then

                --multiple Video PIDs, rerun MarkCommercials until successful

 

                set recrdingIDInteger to recordingID as integer

                set rec to «class cRec» id recrdingIDInteger

                set LogMsg to "RecordingDone found multiple PIDs for recording ID: " & recordingID & ", Channel " & («class Chnm» of rec) & " - " & («class Titl» of rec)

 

                set PIDs to (items 44 thru ((count of logdata_lastrow) - 2) of logdata_lastrow) as string

                set delims to AppleScript's text item delimiters

                set AppleScript's text item delimiters to ", "

                set PID_List to {}

                set PID_List to every word of PIDs

                set AppleScript's text item delimiters to delims

 

                repeat with pid in PID_List

                    my launchComSkip(recordingID, pid)

                    repeat while (my mcIsRunning())

                        delay 5

                    end repeat

                end repeat

 

            end if

        end if

    end tell

end CheckMultiplePIDs

 

on read_from_file(target_file)

    --return the contents of the given file

    set fileRef to (open for access (target_file))

    set txt to (read fileRef for (get eof fileRef) as «class utf8»)

    close access fileRef

    return txt

end read_from_file

 

on write_to_file(this_data, target_file, append_data)

    --from http://www.apple.com/applescript/sbrt/sbrt-09.html

    try

        set the target_file to the target_file as string

        set the open_target_file to open for access file target_file with write permission

        if append_data is false then set eof of the open_target_file to 0

        write this_data to the open_target_file starting at eof

        close access the open_target_file

        return true

    on error

        try

            close access file target_file

        end try

        return false

    end try

end write_to_file

 

on launchComSkip(recID, pid)

    if pid = "" then

        set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " &> /dev/null &"

    else

        set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " --pid=" & pid & " &> /dev/null &"

    end if

 

    do shell script cmd

end launchComSkip

 

on mcIsRunning()

    set processPaths to do shell script "ps -xww | awk -F/ 'NF >2' | awk -F/ '{print $NF}' | awk -F '-' '{print $1}' "

    return (processPaths contains "MarkCommercials")

end mcIsRunning



<Links Edited By Host>

Mac mini Server (Mid 2010), Mac OS X (10.7.4), Lion Server, EyeTV HD, Turbo.264 HD

Posted on Dec 5, 2015 1:28 PM

Close

Q: EyeTV with Commercial Skipping HD DVR Server

  • All replies
  • Helpful answers

first Previous Page 8 of 10 last Next
  • by Ste88,

    Ste88 Ste88 Dec 5, 2015 1:48 PM in response to essandess
    Level 1 (0 points)
    Dec 5, 2015 1:48 PM in response to essandess

    When I upgraded to 10.9 compskip stopped working. I just got it working again. I first installed the new MacPorts - but that wasn't enough for me to get commercial-marking to work. I had to reinstall winetricks, too.

     

    In terminal:

     

    # go to your home directory

    cd ~

     

    # make a backup of your current .wine directory, just in case

    cp -r .wine .wine_backup

     

    # remove the .wine directory

    rm -rf .wine

     

    # reinstall wine

    sudo port install winetricks

     

     

    This last step took longer than I thought it would while the program went and found and downloaded all the necessary, latest files and all the latest dependencies.

     

    I left EyeTV alone for a few days and the new recordings now have correctly-marked commercials. I haven't re-ran MarkCommercials.py (or whatever it's called) to apply this to old recordings, but there's no reason to think it wouldn't work like a fresh recording.

     

    Cheers,

    S

     

    <Edited by Host>

  • by tomar012,

    tomar012 tomar012 Feb 13, 2014 11:08 AM in response to essandess
    Level 1 (8 points)
    Mac OS X
    Feb 13, 2014 11:08 AM in response to essandess

    I have tried the ETVComskip_h264_HD_mods 1.0rc2 but the ExportDone.scpt is giving me the following error:

     

    error "EyeTV got an error: (ID=-1708)" number -1728 from title of recording id 380521861

     

    I only do manual recordings and I don't know if the script is expecting all the details of the show to be entered. Can someone help?

  • by Brandonshire,

    Brandonshire Brandonshire Feb 13, 2014 12:25 PM in response to Ste88
    Level 1 (6 points)
    Apple Music
    Feb 13, 2014 12:25 PM in response to Ste88

    I've been holding off on upgrading to 10.9 specifically becuase I've been dreading trying to make sure comskip keeps working. Have I missed someone posting clear step by step instrucitons for upgrading from 10.8 to 10.9 and getting all of this working in 10.9? I had a heck of a time getting everything running in 10.8 so I'm really hesitent to mess with anything without a clear set of instructions.

     

    [Edit to add] I'm sure all the info one would need is spread out among all the posts there but no one seems to have put it all together and/or made it clear that everything they're talking about is ALL the needs to be done. I'm just concerned I'm going to find that I've missed some step somewhere either becuase I missed a post or becuase everyone writing assumed that any idiot would know to do X Y or Z.

     

    Basically i'm trying to make sure all I have to do is upgrade to 10.9, install the updated macports (do I have to download it from the link mentioned before? or is there a comand line way to update it now?), and apparently reinstall winetricks. Is there anything else?

  • by robert pfile,

    robert pfile robert pfile Feb 14, 2014 10:18 PM in response to Brandonshire
    Level 2 (155 points)
    Feb 14, 2014 10:18 PM in response to Brandonshire

    i dunno... i was running 10.8 and upgraded to 10.9 without really thinking about it. i guess i must have upgraded macports after the upgrade to 10.9 but everything is still working using the last "original" release of ETVComskip.

     

    there are a couple of issues though; i find that comskip processes are sometimes left running forever (burning lots of CPU) and skipping in a program being recorded and viewed at the same time does not work (i am not sure it ever did). the original scripts launch comskip in RecordingStarted, which i assume was intended to make commercial skipping work in an in-progress recording. so i guess i need to adopt the 'new' scripts to keep this from happening.

     

    i think after updating to 10.9 you can try "port selfupdate" but i don't know if that's all that has to happen. probably since i don't remember downloading macports again.

  • by tomar012,

    tomar012 tomar012 Feb 22, 2014 3:03 PM in response to chicodaman
    Level 1 (8 points)
    Mac OS X
    Feb 22, 2014 3:03 PM in response to chicodaman

    Chicodaman or anyone else who can help.  I would like to export my .eyetv files without transcoding to maintain the highest possible quality.  From my research, a lot of people seem to be using Handbrake CLI which would provide a scripting option.  What handbrake command line setting would I use to convert my .eyetv files without transcoding so that it could be played on a Plex media server? Thanks

  • by Brandon Riffel,

    Brandon Riffel Brandon Riffel Feb 22, 2014 3:07 PM in response to tomar012
    Level 1 (0 points)
    Feb 22, 2014 3:07 PM in response to tomar012

    The files are already MPEG, which should be playable by plex.  They are just inside that eyetv wrapper.  Right click on of them and select show contents.  You'll see an MPEG file in there of some substantial size.  Copy it out, rename it, and put it into plex for testing.

     

    Then you just need an Automator script to do that for you after each recording finishes.

     

    Which for non-coders like me, is not so easy.

  • by chicodaman,

    chicodaman chicodaman Feb 23, 2014 9:50 AM in response to tomar012
    Level 1 (15 points)
    Feb 23, 2014 9:50 AM in response to tomar012

    tomar012 wrote:

     

    Chicodaman or anyone else who can help.  I would like to export my .eyetv files without transcoding to maintain the highest possible quality.  From my research, a lot of people seem to be using Handbrake CLI which would provide a scripting option.  What handbrake command line setting would I use to convert my .eyetv files without transcoding so that it could be played on a Plex media server?

     

    Brandon Riffel wrote:


    The files are already MPEG, which should be playable by plex.  They are just inside that eyetv wrapper.  Right click on of them and select show contents.  You'll see an MPEG file in there of some substantial size.  Copy it out, rename it, and put it into plex for testing.

     

    Brandon is correct, you can access the original MPEG file as he described. You can do this directly from the finder or with Applescript. Check out the various versions of "RecordingDone.scpt" floating around. You'll see examples of how that file is accessed via Applescript.

     

    Or: you can export the file from EyeTV without transcoding. Select "Export..." from the file menu and choose one of the native formats (which perform no transcoding).

     

    Or: you can tell EyeTV to perform the export command via Applescript. You use the Applescript "tell application EyeTV" command to perform the export. I don't have the exact syntax with me, but you can have this done automatically via the "RecordingDone" script. Again, you have to be familiar with Applescript, but if you are this is not a difficult thing to program.

     

    You don't need to use Handbrake to do what you're asking. Handbrake will compress your video to take up less disk space, at the cost of some video quality. Handbrake is very good at that but it sounds like you just want to use the original MPEG recording, not a compressed version of it.

  • by tomar012,

    tomar012 tomar012 Feb 23, 2014 2:10 PM in response to Brandon Riffel
    Level 1 (8 points)
    Mac OS X
    Feb 23, 2014 2:10 PM in response to Brandon Riffel

    Thanks Brandon and Chicodaman.  Brandon, I took you advice and move the mpg file, imported into Plex and it worked.  However, I realized that reason I was looking for a handbrake solution is that I wanted to take advantage of the decomb filter.  However, I still want to maintain the hightest quality.  I remember Chicodaman and others saying how bad the eyetv decoder performs so I was trying to stay away from it. I am starting to learn applescript and I know just enough to be dangerous. How do utilize the decomb filter to remove the jiggies without sacrificing quality?

  • by gmartine,

    gmartine gmartine Feb 24, 2014 6:47 AM in response to tomar012
    Level 1 (0 points)
    Feb 24, 2014 6:47 AM in response to tomar012

    Hi tomar012,

    I am not really familiar with HandBrake but as someone has already mentiloned eyetv is just a container sort of file format that "contains" an mpg file. You can easily extract this file automatically as yuo did.

     

    Here is a good blog post that describes how to implement this solution:

     

    http://zacharyflower.com/exporting-eyetv-files-to-plex/

     

    Let me know if you have any luck putting this in place. It is simple. I will be happy to assist you.

  • by nutw07,

    nutw07 nutw07 Mar 1, 2014 5:21 PM in response to tomar012
    Level 1 (0 points)
    Mar 1, 2014 5:21 PM in response to tomar012

    Hello tomar012,

     

    There is a really cool utility called the HandBrakeCLI which allows you to do everything you can do through the user interface (and more!) through the command line, which can be called through applescript.
    I've put together the most barebones script I could come up with to demonstate how this might look.

     

    Open the script in the applescript editor to see what it does.

    After you have downloaded HandBrakeCLI and moved it to the applications folder, toss this script in your /Library/Application Support/EyeTV/Scripts/TriggeredScripts/ folder (just create TriggeredScripts if its not there), and EyeTV will run RecordingDone.scpt when the recording finishes.

     

    I have been working on my EyeTV script (not this one) for around three years now. My current setup currently looks for and marks commercials in EyeTV using Comskip, gets the original air date from the content guide (generated by MC2XML),  exports an M4V file using ffmpeg , then calls SublerCLI to copy the commercial marks into the M4V file as chapter markers for easy skipping. I've had a lot of fun, but have spent an exorbant number of hours working on this as a hobby, so let me know if you have any other questions, and as always there are many others like gmartine who are willing to lend a hand.

     

    Have fun!

  • by tomar012,

    tomar012 tomar012 Mar 7, 2014 6:22 PM in response to essandess
    Level 1 (8 points)
    Mac OS X
    Mar 7, 2014 6:22 PM in response to essandess

    Has anyone install the latest update to the EyeTV software version 3.6.5 and does etvcomskip and associated scripts still work?

  • by tomar012,

    tomar012 tomar012 Mar 11, 2014 5:47 PM in response to nutw07
    Level 1 (8 points)
    Mac OS X
    Mar 11, 2014 5:47 PM in response to nutw07

    Hey nutw07 - thanks for the script. I have been reviewing with hopes of understanding it.  It is amazing what folks are doing with these scripts.

  • by davidb369,

    davidb369 davidb369 Apr 10, 2014 3:58 PM in response to tomar012
    Level 1 (0 points)
    Apr 10, 2014 3:58 PM in response to tomar012

    I use the normal ETVComSkip and my scripts apparently quit working after upgrading to 3.6.5 too (back on March 5th was my last successful automatic marking of commercials).  Any insight anyone has would be most appreciated.  I've tried re-copying the RecordingStarted.scpt and other scripts to the Eyetv Scripts library, to no avail.  Running 10.8

  • by tomar012,

    tomar012 tomar012 May 16, 2014 2:19 PM in response to essandess
    Level 1 (8 points)
    Mac OS X
    May 16, 2014 2:19 PM in response to essandess

    I tried recording a show manually and comskip produced an error. The show was recorded but the commercials were not marked. How can I run markcommercials manually on a specific show?  Thanks

  • by Ste88,

    Ste88 Ste88 Oct 1, 2014 9:02 PM in response to essandess
    Level 1 (0 points)
    Oct 1, 2014 9:02 PM in response to essandess

    I recently upgraded my hard drive and re-installed EyeTV (3.6.5) and copied over my ETVcomskip files from my old OSX installation. ETVcomskip on my old set-up used to just mark the commercials (rather than delete them) so I could use the advance button on the remote to skip the commercials using my Apple TV. This also had the side-effect that if comskip didn't quite get it right I could just manually find the commerical end/start. Now, however, my commericals are being deleting, and I'm missing chunks of shows (especially the part from the last commercial to the end of the show!).

     

    Question: How do I stop ETVcomskip deleting commercials, and go back to the "mark commercials" behavior? The shows in EyeTV are full-length, but in iTunes/AppleTV a 1 hour show is ~44 mins - so the deletion is happening at export I believe.

     

    Any clues??

     

    Thanks,

    Ste88

first Previous Page 8 of 10 last Next