Removing leading "./" in file paths

Question

Removing leading "./" in file paths

Hi

I am writing a script to compare the existence of files in <source> directory with <dest> directory, and if so the size of each. Passed experience has told me that a bash shell buffer cannot hold all the names of files. (there are more than 102,000 files).

Luckily we use a strict folder structure for 3 levels: Client/Project/Stage. (After this the folder names are not consistent.)

So I wrote a script as follows:
sourcepath=$'/Volumes/SOURCE/ProjectWork'
destpath=$'/Volumes/DEST/ProjectWork'
logfile='~/Desktop/file_compare.txt'
for client in $(find $sourcepath -type d -depth 1 ! -name '.DS_Store'); do
for proj in $(find $client -type d -depth 1 ! -name '.DS_Store'); do
for stage in $(find $proj -type d -depth 1 ! -name '.DS_Store'); do
for f in $(find . -type f ! -name '.*'); do
# check file exists in destination folder
done
done
done
done

Problem is the $f variable contains a "./" at the front, which I need to strip off before checking exists in destination folder, otherwise folder cannot be found, i.e. /Volumes/DEST/Projects/Client/Project/Stage/./Folder/Filename

Any ideas how to handle this or improve upon it please? (I only really know how to dabble with bash and use of sed or awk were not successful.)

Thanks in advance.

Cheers
C.

MacBook Pro 15" 2.16GHz Intel Core 2 Duo, Mac OS X (10.5)

Posted on Jan 12, 2009 9:59 AM

Reply

Answer 1

Best reply

Cole Tierney

Level 4

1,380 points

Jan 12, 2009 10:33 AM in response to Craig Roberts1

Hello,

You could have sed strip those characters:
<pre style="padding-left: .75ex; padding-top: .25em; padding-bottom: .25em; margin-top: .5em; margin-bottom: .5em; margin-left: 1ex; max-width: 20ex; overflow: auto; font-size: 10px; font-family: Monaco, 'Courier New', Courier, monospace; color: #444; background: #eee; line-height: normal">sed 's#^\./##'</pre>
--
Cole

Reply

Answer 2

Jan 12, 2009 10:40 AM in response to Craig Roberts1

Try modifying your find statement thus:

for f in $(find . -type f ! -name '.*' | cut -c 3- ); do
echo $f
done

EDIT Or what Cole says 😉

Message was edited by: Michael Conniff

Reply

Answer 3

BobHarris

Level 9

53,267 points

Jan 12, 2009 10:48 AM in response to Craig Roberts1

would the diff command be useful?


diff -r /Volumes/SOURCE/ProjectWork /Volumes/DEST/ProjectWork

Reply

Answer 4

BobHarris

Level 9

53,267 points

Jan 12, 2009 10:51 AM in response to Craig Roberts1

This will remove the ./


f=${f#./}

If the variable does not contain ./ then the variable will be unchanged.

This is a bash, ksh, zsh, POSIX sh built-in, so it will be faster then invoking an external program such as sed or cut

Reply

Answer 5

Jan 12, 2009 10:56 AM in response to Craig Roberts1

Craig Roberts1 wrote:
Problem is the $f variable contains a "./" at the front, which I need to strip off before checking exists in destination folder, otherwise folder cannot be found, i.e. /Volumes/DEST/Projects/Client/Project/Stage/./Folder/Filename

It just occurred to me that you may not need to remove the leading ./. While it's not that clean looking, I think it is a valid path.

--
Cole

Reply

Answer 6

Jan 13, 2009 5:30 AM in response to Cole Tierney

Hi Cole

Thanks for this, but i found it a was beyond my sed knowledge. Would you be so kind as to explain how it works - actually, what # does - and where the most comprehensive resource on sed can be found, as my O'Reilly book on OS X In a Nutshell is not good enough.

Regards
C.

Reply

Answer 7

Jan 13, 2009 6:37 AM in response to Craig Roberts1

Hi Craig,

What I posted was a single sed substitution command (s) that matches a leading (^) period followed by a forward slash. If a leading ./ is found, it is replaced by nothing. Usually the regex delimiter is /, but sed will allow alternates. I used # so I wouldn't have to escape the forward slash. Here are couple examples that may be a little easier on the eyes:
<pre style="border: 1px solid #ddd; padding-left: .75ex; padding-top: .25em; padding-bottom: .25em; margin-top: .5em; margin-bottom: .5em; margin-left: 1ex; max-width: 50ex; overflow: auto; font-size: 10px; font-family: Monaco, 'Courier New', Courier, monospace; color: #444; background: #eee; line-height: normal">echo 'Ho Ender' | sed 's/^H/Y/'
Yo Ender

echo 'Ho Ender' | sed 's/^H//'
o Ender
</pre>
A few sed tutorials:
http://www.ibm.com/developerworks/linux/library/l-sed1.html
http://www.grymoire.com/Unix/Sed.html
http://www.panix.com/~elflord/unix/sed.html

--
Cole

Reply

Answer 8

BobHarris

Level 9

53,267 points

Jan 13, 2009 11:07 AM in response to Craig Roberts1

This is the inner most loop of your code:


...
for f in $(find . -type f ! -name '.*'); do
 # check file exists in destination folder
done
...

You may find it easier to use some shell magic to strip off the ./


...
for f in $(find . -type f ! -name '.*'); do
 f=${f#./}
 # check file exists in destination folder
 ...
done
...

The *${variable#match}* variable substitution operation. The # here says if the beginning of the variable's value begins with the *match* string, remove that string (the smallest match). If there is no match, just substitute the entire variable's value.

In *${variable##match}* the ## says starting at the beginning of the variable's value, look for the largest match, and remove that.

What makes the powerful is that you can use file selection wildcards. For example you could remove all the just the first directory in a path with


new=${file#*/}

And you can remove *ALL* the directories in the file name and leave just the file name using:


new=${file##*/}

There is also *${variable%match}* and *${variable%%match}* where the % and %% are similar to the # and ## except the % and %% says anchor the match at the end of the variable's value. So you can remove stuff from the end of the value.

Another bit of useful magic that goes with this is that the *match* string can also be or contain another variable


new=${old##${my_match_1}}
new=${old%${my_match_2}.mov}
new=${old%%${my_match_3}.[0-9]*}

Reply