Terminal Shell Task/Coding Issue

Try

/usr/bin/time -pl srm ...

change the srm options and observe which options seem to do the most work. "voluntary context switches" seems to be proportional the amount of work. I suspect this is because each time srm does a write, ti is going into a wait state until the I/O is complete which is basically a voluntary context switch.

For me the most work happened with neither -s nor -m specified.

I suggest testing with a large file that you keep making a copy of so that you are testing erasing the same sized file each time.

I guess I should also mention that srm is only going to erase the information in the file. It will not erase any information that is an a replaced sector should the disk decide a sector was no longer reliable. It will not erase any file information that is a scratch file that a program use used with the information to process it. srm will not erase any cache information created while using the information in the file. srm will not erase any information that my have been written to page/swap files because the OS needed to move information out of RAM to disk because of other demands on RAM. If an editor you used to work on the file decides to write a new file and then rename the new file to the original name, as part of the rename the file system will just release the original file's storage to the file system free list, and srm will never touch that.

You already mentioned you are not using an SSD, so you are not subject to the fact that srm on an SSD just shortens the life of the SSD without touching any of the original data which is just sitting in the SSD hardware.

srm is not a fully reliable way to protect information. Depending on how paranoid are about your information, you should really reconsider the FileVault so that delete files (even scratch, cache, renamed) are just random bits.

hiccup wrote:

Any other coding ideas? how about Sudo srm -rf -m or Sudo srm -m?

As Bob suggested, test on a large file (i.e. a GB) (that you don't need to use sudo, you can use sudo later after you find out why it's erasing so fast)

Also, srm will print a % completed as it is erasing, are you seeing this?

Try using srm -m first (you can use the recursive option later.)

Well /usr/bin/time -pl srm ... should have given you some comparison information between doing a simple rm vs an srm vs an srm -m vs and srm -s that will give you an indication of how much work was being done.

You can try the following against rm, srm, srm -s, srm -m

In one Terminal session issue the following command:

sudo fs_usage | grep rm >tmp.rm

In another Terminal session issue the command

cp test.file tmp.tmp

rm tmp.tmp

Go back to the first terminal session and Control-C to abort the fs_usage.

Now start a new

sudo fs_usage | grep srm >tmp.srm

in the another Terminal session

cp test.file tmp.tmp

srm tmp.tmp

go back to the first and Control-C the fs_usage

Next

sudo fs_usage | grep srm >tmp.srm_m

in the other session

cp test.file tmp.tmp

srm -m tmp.tmp

Control-C the fs_usage

finally sudo fs_usage | grep srm >tmp.srm_s

cp test.file tmp.tmp

srm -s tmp.tmp

Control-C fs_usage

Now you should have 4 files of system call trace data for the rm command, the srm command (35 pass), the srm -m (7 pass), the srm -s (simple overwrite).

Now compare the 4 files to see how much extra work is done for each kind of delete.

If you need to zero, unused space in your filesystem, you can create a large file then erase it. That's how apple used to do it anyway.

Here is one way, from terminal.

dd if=/dev/zero of=zero bs=1024k count=100

and

dd if=/dev/random of=zero bs=1024k count=100

count is the number of input blocks

bs is input and output block size

For a 1 gig file how long does it take you?? (I know it depends on how many files and what kind of files, just want to know if theres an average)

A) I have a SSD, so

A.1) the srm command will not securely erase a file on an SSD, it will just keep writing to SSD revectored physical sectors to my logical sectors

A.2) it would needlessly shorten the life of my SSD.

B) I use FileVault, so when I delete a file, it is just a bunch of random bits to begin with.

C) I understand why secure erase does not guarantee the data is really erased.

It's a lot faster than I'd have imagined. I did only 100m. Got these numbers.

mac $ dd if=/dev/random  of=zero  bs=1024k  count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 7.336857 secs (14291897 bytes/sec)
mac $ ls -l zero 
-rw-r--r--  1 mac  staff   100M Nov  8 22:52 zero
mac $ \time -lp  srm -siv  zero 
Remove zero? y
removing zero
done
real         6.56
user         0.45
sys          0.09
   1609728  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
       402  page reclaims
         0  page faults
         0  swaps
         0  block input operations
        97  block output operations
         0  messages sent
         0  messages received
         0  signals received
       119  voluntary context switches
       179  involuntary context switches
mac $ dd if=/dev/random  of=zero  bs=1024k  count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 7.368458 secs (14230603 bytes/sec)
mac $ \time -lp  srm -miv  zero 
Remove zero? y
removing zero
done
real        18.17
user         0.91
sys          0.20
   1609728  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
       402  page reclaims
         0  page faults
         0  swaps
         0  block input operations
        87  block output operations
         0  messages sent
         0  messages received
         0  signals received
       725  voluntary context switches
       652  involuntary context switches
mac $

hiccup wrote:

When SET was on Finder menu it usually took about 15 minute for so, not 5 minutes, something odd's going on, I me its simply to fast to write a 35 pass on 1 GB file, even the 7 pass /usr/bin/time -pl srm -m should take longer than just under two minutes.

I wouldn't compare the time to Finder.

Try copying the 1G with cp, then multiply by 7 and then compare the time to srm -m

The Finder.app uses /System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Resources/Locum , a helper application that elevates privileges to root during the secure erase process.

rccharles

1. 119 voluntary context switches
2. 725 voluntary context switches

A voluntary context switch can be caused by making a system call that results in waiting for I/O to complete.

The first erase was a single pass of zeros (119) context switches

The second erase was a medium 7 passes of random data (725) context switches, which is 6 times as many.

In other words it took about 100 context switches to write a single pass of zeros, and 700 context switches to write 7 passes of random data. Now some of the context switches may have been for getting the random data, but I'm guessing most was to write 7 times over the file.

However, the bottom line is that srm is not insuring your data is securely erased. It is just doing a lot of I/O hoping it is catching all of your bits.

And if you get a new Mac with an SSD, you will need to find some other way to insure your data is secure, because srm is not going to do anything good for an SSD, except shorten its life and get you to spend more money when it needs to be replaced early (well good for someone else, such as OWC or Apple that will get your money 🙂).

I don't no what other coding options there are, maybe I need to reinstall El Capitan and see if Terminal was corrupted on the original install. Seems a bit drastic for such a tiny app.

It is unlikely anything is corrupt. And the 'srm' command is totally independent from the Terminal app. The Terminal app just draws a window and passes keyboard input to the shell or program running under the shell, and it displays the output from the shell or the program running under the shell.

And while it is possible the Finder used 'srm' under the covers, it is also possible the Finder rolled its own secure erase an that 'srm' is either more efficient, or it is less effective (for example not forcing the random data out of the operating system file system cache between each pass, so that it ends up just overwriting the memory cache and not the file).

You can download the open source 'srm' command and study how it works.

Time reported:

For one pass...

97 block output operations

for seven pass

87 block output operations

& That's the number I got. I figured the command was optimized to chained all the writes together which would reduce Output counts.

R