Need help with sed to fix migrated web site links

Question

Level 1

106 points

Need help with sed to fix migrated web site links

I'm moving my web files from one host to another, and have backed up my directory called WEB into my user downloads folder. The site is dead simple, mostly links within the site, pages primarily text with some links to my photos posted on flickr, and a few outside links. I want to avoid opening/replacing/saving a hundred or more files to fix the links from "http://www.old_domain_name/user/my_username" to "http://my_new_domain_name"

A friend suggested the following scripts to do this

mkdir fix_urls

cd fix_urls

cp -r <wherever>/WEB .

cd WEB

xargs -0 egrep -i old_domain_name $(find * -name "*.html" -print0) > ../to_fix

cd ..

less to_fix

to see what needs changing, but after I've created and moved to WEB, the xargs command results in a zero kb file called to_fix appearing in WEB, and nothing else happens....and the prompt in terminal (myusername$) doesn't reappear.

I tried his part II anyway:

mkdir new_web

while read filename; do

sed 's/\/www.old_domain_name/user/my_username/\/my_new_domain_name/' < WEB/$filename > new_web/$filename

done

diff -r WEB new_web > what_i_did

less what_i_did

and after the 'done', I get again....nothing. No new cursor gray rectangle, no new prompt, haven't even gotten to the last bit of diff to review the changes.

Any tips on what fundamental bone-headed thing I'm doing wrong?

Posted on Jul 1, 2012 12:45 AM

Reply

Answer 1

Best reply

X423424X

Level 6

14,265 points

Jul 1, 2012 1:23 AM in response to Donot Haveone

I want to avoid opening/replacing/saving a hundred or more files to fix the links from "http://www.old_domain_name/user/my_username" to "http://my_new_domain_name"

If that's all you want to do then here's the pertinent sed command to do it:

sed -e 's,www.old_domain_name/user/my_username,my_new_domain_name,'

Syntax is a bit simpler when you don't use slashes as the sed substitute delimiters. You can use anything. Here I used commas.

Reply

Answer 2

Donot Haveone Author

Level 1

106 points

Jul 1, 2012 1:31 AM in response to X423424X

Actually, it's not fixed. I got excited too quickly after the first file I clicked looked correct--the first file in the subdirectory.

I see no indication in terminal that the process is going or ending--no sense of when I can close terminal and move on.

Checking the directory in finder shows no updates to 'date modified' for any of the files.

So how do I know if it is in progress, or done working, aside from opening random files to see if they're 'done'?

Reply

Answer 3

X423424X

Level 6

14,265 points

Jul 1, 2012 1:54 AM in response to Donot Haveone

Put it in a text file and trace it (and I don't recommend using textedit, use a real text editor like TextWrangler or BBEdit).

Say you create a text file called my-script. In terminal

chmod +x my_script

Now it's an executable script. You can type my_script from the terminal and execute it.

Of course that's the basic scheme. I leave it to you to get the pathnames corrrect.

All this is leading to the fact that if you place a

set -x

at the beginning of the script you will see each command as it is executed. I would do this on a small subset of the data until you fix the problems.

Alternatively it's only about 8 command lines so just type them on the terminal to see what happens that way.

I only gave you the central sed to edit the strings you presented in your original post. I didn't check your script and assumed you could figure out the appropriate placement.

Reply

Answer 4

X423424X

Level 6

14,265 points

Jul 1, 2012 2:12 AM in response to Donot Haveone

I may have left out a few steps but maybe what I hav below is what you need:

while read f; do

echo "$1/$f"

sed -e 's,www.old_domain_name/user/my_username,my_new_domain_name,' < "$1/$f" >"$2/$f"

done< <(ls "$1")

This is a script as I described previously. It takes two arguments. The first is the path to the input directory and the second is the path to the output directory. So if my-script was the name of this script you would execute something like:

my-script WEB new-web

Hope this helps.

(and again it is up to you to specify the pathnames to the script and the input/output directory pathnames correctly)

---

Note the cryptic while loop is done to allow for the possibility that the filenames in the source directory having special characters (like blanks in them).

Reply