Skip navigation

Need help with sed to fix migrated web site links

545 Views 8 Replies Latest reply: Jul 1, 2012 2:34 PM by Donot Haveone RSS
Donot Haveone Level 1 Level 1 (15 points)
Currently Being Moderated
Jul 1, 2012 12:45 AM

I'm moving my web files from one host to another, and have backed up my directory called WEB into my user downloads folder.  The site is dead simple, mostly links within the site, pages primarily text with some links to my photos posted on flickr, and a few outside links.  I want to avoid opening/replacing/saving a hundred or more files to fix the links from "http://www.old_domain_name/user/my_username" to "http://my_new_domain_name"


A friend suggested the following scripts to do this


  mkdir fix_urls

  cd fix_urls

  cp -r <wherever>/WEB .

  cd WEB

  xargs -0 egrep -i  old_domain_name $(find * -name "*.html" -print0) > ../to_fix

  cd ..

  less to_fix


to see what needs changing, but after I've created and moved to WEB, the xargs command results in a zero kb file called to_fix appearing in WEB, and nothing else happens....and the prompt in terminal (myusername$) doesn't reappear.


I tried his part II anyway:


  mkdir new_web

  while read filename; do

      sed 's/\/www.old_domain_name/user/my_username/\/my_new_domain_name/' < WEB/$filename > new_web/$filename


  diff -r WEB new_web > what_i_did

  less what_i_did


and after the 'done', I get again....nothing.  No new cursor gray rectangle, no new prompt, haven't even gotten to the last bit of diff to review the changes.


Any tips on what fundamental bone-headed thing I'm doing wrong?

  • X423424X Level 6 Level 6 (14,190 points)

    I want to avoid opening/replacing/saving a hundred or more files to fix the links from "http://www.old_domain_name/user/my_username" to "http://my_new_domain_name"


    If that's all you want to do then here's the pertinent sed command to do it:


    sed -e 's,www.old_domain_name/user/my_username,my_new_domain_name,'


    Syntax is a bit simpler when you don't use slashes as the sed substitute delimiters.  You can use anything.  Here I used commas.

  • X423424X Level 6 Level 6 (14,190 points)

    Put it in a text file and trace it (and I don't recommend using textedit, use a real text editor like TextWrangler or BBEdit).


    Say you create a text file called my-script.  In terminal


    chmod +x my_script


    Now it's an executable script.  You can type my_script from the terminal and execute it.


    Of course that's the basic scheme.  I leave it to you to get the pathnames corrrect.


    All this is leading to the fact that if you place a


    set -x


    at the beginning of the script you will see each command as it is executed.  I would do this on a small subset of the data until you fix the problems.


    Alternatively it's only about 8 command lines so just type them on the terminal to see what happens that way.


    I only gave you the central sed to edit the strings you presented in your original post.  I didn't check your script and assumed you could figure out the appropriate placement.

  • X423424X Level 6 Level 6 (14,190 points)

    I may have left out a few steps but maybe what I hav below is what you need:


    while read f; do

        echo "$1/$f"

        sed -e 's,www.old_domain_name/user/my_username,my_new_domain_name,' < "$1/$f" >"$2/$f"

    done< <(ls "$1")


    This is a script as I described previously.  It takes two arguments.  The first is the path to the input directory and the second is the path to the output directory.  So if my-script was the name of this script you would execute something like:


    my-script WEB new-web


    Hope this helps.


    (and again it is up to you to specify the pathnames to the script and the input/output directory pathnames correctly)




    Note the cryptic while loop is done to allow for the possibility that the filenames in the source directory having special characters (like blanks in them).

  • BobHarris Level 6 Level 6 (12,505 points)

    Download the free TextWrangler utility and use it multi-file search and replace capabilities.


    It will be a little more tedious than a script, but more comfortable as it is a GUI text editor.


More Like This

  • Retrieving data ...

Bookmarked By (0)


  • This solved my question - 10 points
  • This helped me - 5 points
This site contains user submitted content, comments and opinions and is for informational purposes only. Apple disclaims any and all liability for the acts, omissions and conduct of any third parties in connection with or related to your use of the site. All postings and use of the content on this site are subject to the Apple Support Communities Terms of Use.