replace text in multiple files - pathnames with spaces etc.

Question

Level 1

4 points

replace text in multiple files - pathnames with spaces etc.

I’m trying to replace all instances of text in multiple (>300) files with another bit of text. All files share a common extension (.SCRIVX).

lt’s been 30 years since I’ve written shell scripts, but it I would think that should be relatively easy.

But everything I’ve tried blows up (welcome back to Unix 😁). The (first) problem seems to be with “find” passing filenames with spaces to sed or perl. I can’t get past the spaces and other special characters in the file names - I obviously don’t know enough about how to do this.

If someone who knows about this could let me know how to fix the command(s) I would really appreciate it.

I need to replace some fairly lengthy strings, which also have funky characters in them (these are in XML files that hold references to external documents, and the referenced file structure is changing). Do I need to escape the “slash” and % characters in these strings or can I just quote them like “$STR1” - or would it just be best to type the whole thing into the command and not bother with variables? I think I’m likely to have some quoting issues here, as well as file name issues.

Here’s an example of the text I need to replace (bold needs to be replaced):

Here’s the basics of what I’ve tried:

1) Basic find to sed attempt

#!bin/bash

# find-replace-test.sh

#

STR1='newbookworld/bookworld%20folder/All%20My%20Stuff/WORLD'

STR2='TotallyTidaniko/UNIVERSE%20CONVERSION%20TRIAL/A%20POTENT,%20LUSH,%20REVELA TORY%20EXPERIENCE%20OF%20THE%20DIVINE/WORLD%204-1-17'

find . -type f —name *\.scrivx exec sed -i ‘’ ’s/‘“$STR1”’/‘“$STR2”’/g’ {} +

2) Basic find to perl attempt

#!/bin/bash

# testperlreplace.sh path_to_directory

STR1="newbookworld/bookworld%20folder/All%20My%20Stuff/WORLD"

STR2="TotallyTidaniko/UNIVERSE%20CONVERSION%20TRIAL/A%20POTENT,%20LUSH,%20REVELA TORY%20EXPERIENCE%20OF%20THE%20DIVINE/WORLD%204-1-17"

perl -e 's/"$STR1”/“$STR2"/g;' -pi.save $(find $1 -type f -name "*\.scrivx")

this blows up with :

Can't open ./GREAT: No such file or directory.

Can't open FILES: No such file or directory.

Can't open FROM: No such file or directory.

Can't open PAST.scriv/GREAT: No such file or directory.

Can't open FILES: No such file or directory.

Can't open FROM: No such file or directory.

Can't open PAST.scrivx: No such file or directory.

In this instance, the file I’m testing is called “GREAT FILES FROM THE PAST.SCRIV” which is a package in MacOS, so a directory in Unix - the file I’m wanting to replace text in is called “GREAT FILES FROM THE PAST.SCRIVX”

Posted on Apr 21, 2017 9:49 AM

Reply

Answer 1

Tony T1

Level 6

10,247 points

Apr 21, 2017 1:03 PM in response to DrZ

This works with sed:

#!/bin/bash

STR1='newbookworld/bookworld%20folder/All%20My%20Stuff/WORLD'

STR2='TotallyTidaniko/UNIVERSE%20CONVERSION%20TRIAL/A%20POTENT,%20LUSH,%20REVELA TORY%20EXPERIENCE%20OF%20THE%20DIVINE/WORLD%204-1-17'

find . -type f -name *\.scrivx -exec sed -i "" "s|$STR1|$STR2|g" {} \;

Note that I changed 's/$STR1/$STR2/g' to "s|$STR1|$STR2|g"

/ to | will avoid escaping the / in the variables, and also double quotes are needed when using variables in sed.

I also needed to fix some other quoting in sed, and didn't end find with +, but should work with it

Reply

Answer 2

Apr 22, 2017 7:23 AM in response to DrZ

I need to replace some fairly lengthy strings, which also have funky characters in them (these are in XML files that hold references to external documents, and the referenced file structure is changing). Do I need to escape the “slash” and % characters in these strings or can I just quote them like “$STR1”

You should be aware that STR1 is treated as a regular expression. So, yes, escaping certain characters is necessary.

You could have escaped the forward slash and then use the forward slash as the delimeter in sed (bash and zsh). Other character that may be problematic are: single and double quotes, square and curly brackets, parenthesis, the dollar sign and caret, etc. On the replacement end, in STR2 the backwards slash and the ampersand have to be treated special.

In your find statement the period doesn't need quoting but the glob should be quoted in single or double quotes.

find . -name '*.scrivx' .........

Your issue with your perl statement is that the command substitution is presenting the filenames to perl as- file1 with spaces file2 with spaces -and the shell is performing word splitting at the white spaces.

find can handle the files with spaces in their name as could shell globbing. You could use globstar if you have the latest version of bash, or use zsh or ksh.

sed -i '.save' 's@'"$STR1"'@'"$STR2"'@g' **/*.scrivx

I suggest that if you are going to edit "inplace" save the original file then delete it if you get the desired results.

Reply

Answer 3

rccharles

Level 6

13,037 points

Apr 23, 2017 11:48 AM in response to Tony T1

sed is really a line editor. You cannot have line end characters in your input stream.

R

Reply