Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

EyeTV with Commercial Skipping HD DVR Server

I use EyeTV HD and Turbo.264 HD to record and transcode h.264 shows for playback on a Mac, Apple TV, iPad, iPhone. I also use a self-modified version of etvcomskip to detect and mark commercials, which are then automatically skipped at playback. This makes tv viewing a lot more pleasurable and efficient, and is especially helpful controlling advertiser access to children. In case this is useful to others, I've outlined the steps here:


  1. Eric Kaashoek's website Comskip and download his HD-capable Comskip software. "Commercial" commercial-skippingcompanies (ReplayTV, TiVo, Dish, etc.) are quickly sued out of existence or into submission, even if they simply provide a basic "30-second skip" feature. Comskip is open source beerware that you can download, install, and use yourself, and you can't sue anyone for publishing source code. And rather than providing a simple 30-second skip feature, it detects, marks, and provides that capability for automatic commercial skipping.
  2. Comskip a Windows executable, so you'll need WINE (WINdows Emulator) on OS X to run it. First, download and install Xcode (free from the Apple App store), then download and install and install Macports, then in Terminal, run "sudo port selfupdate; sudo port install wine-devel". Finally, make sure that /Applications/Utilities/X11.app is added to your list of applications to run at login (WINE uses X11 to display Windows windows). Yes, this is windows emulation, but on a half-decent box you'll see that commercial skipping runs at around twice real-time for h.264 HD recordings.
  3. Download and install etvcomskip (and PyeTV if you want EyeTV to work with Front Row). The 2010 version of etvcomskip doesn't work with Comskip's latest HD capabilities, and the latest Comskip mpeg libraries do not allow live tv processing, so install these files to get everything working. Once you done this, you'll have a commercial-skipping HD DVR box on your Mac that can stream to Apple TV, iOS, or export to iTunes.

Comskip software

This lives in the directory /Library/Application Support/ETVcomskip.


Edit the file ./comskip/comskip.ini to use these HD-tuned settings, tailored for US cable:


detect_method=127;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
max_avg_brightness=21;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
non_uniformity=50; Set to 0 to disable cutpoints based on uniform frames
max_volume=20; any frame with sound volume larger than this will not be regarded as black frame
logo_at_bottom=1; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
intelligent_brightness=1; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA

volume_slip=150

output_dvrmstb=0; Set to 1 if you're running DVRMS-Toolbox
live_tv=0; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4; change to 16 when using live_tv in BTV, used for mpeg PS and TS


Edit the file ./MarkCommercials.app/Contents/Resources/MarkCommercials.py to use WINE to run comskip.exe nicely. Grep for these changes:


EyeTV.launch()


msg = 'Error: unable to communicate with %s\n' % options.app


#cmd = '"/Library/Application Support/ETVComskip/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile

# MacPorts 64-bit wine

#cmd = '"/Applications/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile

cmd = '"/opt/local/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile


# nice the wine command

cmd = "/usr/bin/nice -n 14 " + cmd


EyeTV software

EyeTV uses the triggered AppleScripts RecordingStarted and RecordingDone in the directory /Library/Application\ Support/EyeTV/Scripts/TriggeredScripts. Open these using the AppleScript Editor and use these scripts instead. You will probably have to Save As... on your desktop, then copy the updated script back into the correct directory to avoid permissions issues.


RecordingStarted.scpt


on RecordingStarted(recordingID)

delay 10

-- comskip81 uses ffmpeg and does not support live tv; put this in RecordingDone.scpt

-- set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"

-- display dialog cmd

-- set cmd to "env > /tmp/etv_test.log &"

-- do shell script cmd


--disable this if you do not want a logfile written

write_to_file((short date string of (current date) & " " & time string of (current date)) & "Recording Started run for ID: " & recordingID & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)

end RecordingStarted


on write_to_file(this_data, target_file, append_data)

--from http://www.apple.com/applescript/sbrt/sbrt-09.html

try

set the target_file to the target_file as string

set the open_target_file to open for access file target_file with write permission

if append_data is false then set eof of the open_target_file to 0

write this_data to the open_target_file starting at eof

close access the open_target_file

return true

on error

try

close access file target_file

end try

return false

end try

end write_to_file

RecordingDone.scpt


-- Run the python MarkCommercials script for the given recording

-- this must be run with the RecordingStarted script

-- it will check if there were multiple PIDs for the recording and runs MarkCommercials for each pid

-- requires updated MarkCommercials which allows specifying the pid

-- by Ben Blake, September 2009


-- modified for latest Comskip, which cannot be run until after recording is finished; waits for Turbo.264 HD to stop running as well

-- S.T.Smith


global LogMsg


on RecordingDone(recordingID)


set ProcessName to "Elgato H.264 Decoder"

set DelayTime to 60

set MaxDelays to 4 * 60 -- four hours


-- delay until the process slows down or stops running

repeat MaxDelays times

delay DelayTime -- delay at least once to give it a chance to start

set pcpu to CPUPercentage(ProcessName)

if pcpu is equal to "" or pcpu < 2.0 then exit repeat -- break out of delay loop if it's not running

end repeat


delay 10

-- comskip81 uses ffmpeg and does not support live tv; take this from RecordingStarted.scpt

set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"

-- display dialog cmd

-- set cmd to "env > /tmp/etv_test.log &"

do shell script cmd


set LogMsg to ""

CheckMultiplePIDs(recordingID)


--disable this if you do not want a logfile written

if (count of LogMsg) > 0 then

write_to_file((short date string of (current date) & " " & time string of (current date)) & LogMsg & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)

end if

end RecordingDone


-- testing code: this will not be called when triggered from EyeTV, but only when the script is run as a stand-alone script

on run

tell application "EyeTV.app"

set rec to «class Unqu» of item 1 of every «class cRec»


my RecordingDone(rec)

end tell

end run


-- compute the percentage CPU used by ProcessName

on CPUPercentage(ProcessName)

set ProcessPS to do shell script ("ps -axwwc | grep '" & ProcessName & "' | grep -v grep || true")

if ProcessPS is not equal to "" then

set ProcessID to word 1 of ProcessPS

set ProcessPS to do shell script ("ps -xwwco pid,ppid,%cpu -p " & ProcessID & " | tail -1")

set ProcessCPU to word 3 of ProcessPS

return ProcessCPU as number

else

return ""

end if

end CPUPercentage


on CheckMultiplePIDs(recordingID)

--check if there are multiple Video PIDs in the file


tell application "EyeTV.app"

set input_text to my read_from_file((path to "logs" as string) & "ETVComskip" & ":" & recordingID & "_comskip.log")

if (count of (input_text as string)) > 0 then

set logdata to every paragraph of input_text

set logdata_lastrow to (item ((count of logdata) - 1) of logdata) as string


if (items 1 thru 19 of logdata_lastrow) as string = "Video PID not found" then

--multiple Video PIDs, rerun MarkCommercials until successful


set recrdingIDInteger to recordingID as integer

set rec to «class cRec» id recrdingIDInteger

set LogMsg to "RecordingDone found multiple PIDs for recording ID: " & recordingID & ", Channel " & («class Chnm» of rec) & " - " & («class Titl» of rec)


set PIDs to (items 44 thru ((count of logdata_lastrow) - 2) of logdata_lastrow) as string

set delims to AppleScript's text item delimiters

set AppleScript's text item delimiters to ", "

set PID_List to {}

set PID_List to every word of PIDs

set AppleScript's text item delimiters to delims


repeat with pid in PID_List

my launchComSkip(recordingID, pid)

repeat while (my mcIsRunning())

delay 5

end repeat

end repeat


end if

end if

end tell

end CheckMultiplePIDs


on read_from_file(target_file)

--return the contents of the given file

set fileRef to (open for access (target_file))

set txt to (read fileRef for (get eof fileRef) as «class utf8»)

close access fileRef

return txt

end read_from_file


on write_to_file(this_data, target_file, append_data)

--from http://www.apple.com/applescript/sbrt/sbrt-09.html

try

set the target_file to the target_file as string

set the open_target_file to open for access file target_file with write permission

if append_data is false then set eof of the open_target_file to 0

write this_data to the open_target_file starting at eof

close access the open_target_file

return true

on error

try

close access file target_file

end try

return false

end try

end write_to_file


on launchComSkip(recID, pid)

if pid = "" then

set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " &> /dev/null &"

else

set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " --pid=" & pid & " &> /dev/null &"

end if


do shell script cmd

end launchComSkip


on mcIsRunning()

set processPaths to do shell script "ps -xww | awk -F/ 'NF >2' | awk -F/ '{print $NF}' | awk -F '-' '{print $1}' "

return (processPaths contains "MarkCommercials")

end mcIsRunning

<Links Edited By Host>

Mac mini Server (Mid 2010), Mac OS X (10.7.4), Lion Server, EyeTV HD, Turbo.264 HD

Posted on Jun 4, 2012 7:57 AM

Reply
144 replies

Jul 5, 2012 7:25 PM in response to essandess

This is awesome! I've been an EyeTV/aTV user for 3 years and lost the ability to edit out the commercials automagically. I got it marking them again on my most recent install, but would have to the compress in EyeTV and export to Apple tv manually to get rid of them.


How can I contact you more directly for help on this? I am happy to make a donation to commskipper, and I'd make an in kind gift of some iTunes credit to you for assistance!


Thanks!

Jul 9, 2012 12:35 PM in response to Brandon Riffel

I got it marking them again on my most recent install, but would have to the compress in EyeTV and export to Apple tv manually to get rid of them.


Unfortunately, I don't believe this is possible yet with EyeTV products -- they have six different triggered AppleScripts, none of which allow running comskip before transcoding starts -- there's only ExportDone, but no ExportStarted, which would allow you to run comskip beforehand.


This may be a (very thin) silver lining because if you mark commercials before transcoding, EyeTV/Turbo.264 reads comskip's .edl file (that has the start/stop times of all commercials it found) and clips out all marked regions, including any false positives from comskip. FPs happen. I've run it this way in the past with lores iPhone exports (that required absolutely no transcoding at all) and it sure is nice and saves a good amount of disk space, but you have to go back to the original if you ever have a false positive at an important part.


It would also be nice if iOS/OSX/Apple TV respected .edl files for video playpack, thus allowing iOS/Apple TV commercial skipping on the device if you also provide the .edl, but I don't believe that this is supported at all.


The only options I see are to request that Elgato support an ExportStarted triggered script, or to request that Apple support .edl video playback on iOS/Apple TV.

Dec 5, 2015 1:30 PM in response to essandess

Have you made this work in Mountain Lion? I am having some difficulty.


1. X11 apparently is now being run by XQuartz?? (at least that is what I was guided to install after some error messages). Message said that ML no longer fully supports X11, and XQuartz is needed.


2. In step 3, did you just install the mac version (ETVComskip-2.0.2-10.6.dmg), or did you copy the files from the donor's version (comskip81_048_donators.zip) which has the latest comskip.exe files.


3. Which part of the RecordingStarted.scpt did you edit. My existing RecordingStarted.scpt is quite long. Is your script intended to be the entire script of a segment that needs to be changed from the original?


4. Same question as 3 but for the RecordingDone.scpt.


<Link Edited By Host>

Oct 7, 2012 7:39 PM in response to Niccum

Yes, it works great in ML. The three changes are: download Xcode from the App Store, download XQuartz, and upgrade all the outdated MacPorts, specifically wine-devel:


$ sudo port selfupdate

$ sudo port upgrade outdated


I installed the latest googlecode ETVComskip, then made the changes described above, including swapping in the files from the donor's version of Comskip into the directory /Library/Application Support/ETVComskip/comskip.


That's my entire RecordingDone script. I also modified the Python script /Library/Application Support/ETVComskip/MarkCommercials.app/Contents/Resources/MarkCommercials.py to nice +15 all the wine jobs, though this isn't necessary.


In fact, you've caught me coding up some Python to add the .edl times as chapter times in the h.264 files that EyeTV exports for iPad streaming and iTunes, so you don't have to fiddle with the annoying and innaccurate iOS video time swiper. This will, I hope, provide Comskip information on iPads. I'll give this a week or two to finish and test, then I'll post that too. The relevant command is mp4chaps, available via:


$ sudo port install mp4v2

Oct 8, 2012 5:30 PM in response to essandess

(It was faster and easier to finish and this than I expected.)


ETVComskip works great for EyeTV's original recordings, but the Comskip information isn't included with exports to iPad/iPhone/Apple TV files created by EyeTV or Turbo.264. Result: iTunes exports and wifi streams have ads. This modification addresses this issue by adding Comskip .edl information as chapters within the H.264 .m4v files exported by EyeTV and Turbo.264, providing a mobile Comskip solution: when the ads appear, skip to the next chapter. This is a lot easier and faster than using the (barely functional) time swiping capability to guess where the show starts. It also avoids the problem of false detections by not deleting any program material.


See the script and details at code.google.com/p/etv-comskip/issues/detail?id=54.

Oct 8, 2012 5:41 PM in response to essandess

Great. Thanks for sharing all of this information. It will really help to simplify the whole process.


Interestingly, if I read you last post correctly, my EyeTV setup does indeed already skip the marked commercials in the 1080p AppleTV export. I just check to ensure the skip areas are correct and simply export by pressing the 1080p AppleTV Export button. I have not had to compress prior to export. I have not tried anything but the 1080p Export.


Sorry, but I don't understand the "I also modified the Python script /Library/Application Support/ETVComskip/MarkCommercials.app/Contents/Resources/MarkCommercials.py to nice +15 all the wine jobs, though this isn't necessary."


So, both of the above recording scripts are in their entirety. RecordingStart.scpt and RecordingDone.scpt


Again, thanks for the info.

Oct 8, 2012 6:03 PM in response to Niccum

> my EyeTV setup does indeed already skip the marked commercials in the 1080p AppleTV export.


It depends. I've observed that if EyeTV is transcoding a new file (takes a long time, hence the desirability of a Turbo.264 dongle), AND there's an existing .edl file from Comskip, the the transcoded respects the .edl edit decision list and deletes the (hopefully) commercials. Too bad if there are false detects. If, however, the export to iTunes is the same format at the EyeTV sharing format, then the .edl is ignored.


The scripts above do trivial load balancing by making sure that transcoding and/or exports are done first, then Comskip is run. The .m4v files in EyeTV's archive will have Comskip chapter markings, but any exports to iTunes prior to running Comskip will not have these chapters. My personal solution is to turn off automatic iTunes exports and export using the same quality as EyeTV sharing files, which ensures a fast simple copy to iTunes with no new transcoding when I export manually.


Now that I think of it, a simple mod to take care of previously exported iTunes files would be to write an ExportDone script that saves the full file name of the iTunes exported file within its originating .eyetv directory, then when Comskip is run in the Python script MarkCommercials.py, look for this file and update its chapter information. This would be useful for people who save both 720p files for iPad streaming and 1080p files for Apple TV. That's a lot of disk space and transcoding cpu cycles. I just use 720p for all streaming and exports, and save the 1080p for an HDMI cable.

Oct 12, 2012 1:35 PM in response to essandess

I think this will take care of my issues with videos that are recording getting messed up while commercials are being blocked out in Mountain Lion (an issue I never had in Lion oddly).


I've also taken to exporting for iPhone now as well.


I wonder if, instead of calling the routine on RecordingDone and waiting for the encoder to finish, you could just call the script on ExportDone? I guess it all depends on if you do a lot of other exporting.


Calling it on ExportDone would also cause it to re-run (with --force) after compacting. I haven't messed around to see if the chapter marks mess with what gets shown if they aren't re-calculated after compacting.


Also, I fixed up the Growl notification (--growl) from within the python script of MarkCommercials if anyone else is interested in having this work.

Oct 12, 2012 8:09 PM in response to gardnern

A lot of people, including me, only export a subset of their shows to iTunes, so handling commercial skipping with ExportDone would ignore commercial skipping for a lot of shows. And to avoid sucking all CPU cycles into h.264 trancoding and commercial marking, the RecordingDone script above waits for exports to be finished, which has the same functionality as putting this code in ExportDone.


I've been using commercial skipping on an iphone/ipad for a week now and it's great!—just hit the right video arrow and it works like a charm. I've been exporting EyeTV to itunes by hand, which is getting old already. I'd like to wrote an ExportDone script that remembers the exported show filenames in a text file within the source .eyetv directory, then adds the mp4 chapter info to those files when MarkCommercials.py is invoked. If anyone already has such a script, please post it.


And it would be great if you could post your updated Growl notification code for Mountain Lion. I'll update the ETVComskip post with it.

Oct 13, 2012 5:16 AM in response to essandess

Ok, so I'm trying to get caught up. Are you all saying that recording in EyeTV, marking commercials, compressing, and exporting to AppleTV/iTunes, is working well. Working in Mountain Lion?


Is having the Turbo 264 a requirement?


I have a current generation mid-range Mini acting as my media server with 2 USB tuners. Elgato support said that I would probably only see marginal speed improvements with T264, so I haven't invested yet.


I'm current recording and marking commercials with EyeTv Comskip, but then after the recordings are done, I go in and manually compress. Then go back and manually export to ATV. The compressing of a 30 minute HD program (down to 20 minutes after editing) tales about 5 minutes. Export to ATV about 30, assuming I'm not recording or compressing anything else at the time.


I'd love to be able to automate any or all of this. I do like being able to fine tune the commercial markers, I can usually shave off at least 3 more minutes or network commercials for other shows I don't care about, and cut off the endings and beginnings of the surrounding programs that sometimes are included.


Can somebody give a very basic breakdown of exactly what is working and in what scenario? As we are all Apple owners, we are ok with paying for form and functionality, we just want it all to work seemlessly!


Thanks again to all of you for this hard work.


Again, I'm offering a $25 iTunes credit to whomever wants to remote into my setup and get it working/give me a tutorial.

Oct 13, 2012 5:28 AM in response to essandess

Brandon,

I am doing a similar routine as I also like to manually tweak the commercial markers. I have found on my system that I can skip the compress step and go straight to the export. The export to AppleTV recognizes the commercial markers on my setup. So at least you can eliminate the compress step if you want to.


For those of us that are anal about fine tuning the markers, it would be nice to be able to be able to check them off as prepared for export and then have an export routine kick in automatically in the middle of the night to do the longer export work. That is about as automated as I can think of if still needing to fine tune the commercial marks.


Eric

EyeTV with Commercial Skipping HD DVR Server

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.