"scp""hdiutil""rsync""Broken Pipe"Errors

Question#1: Is there any way I can use “Pipe” to combine two commands in one?

I've done things using tar

cd /where/I/want/to/start
tar cf - ./ | ssh username@remote.system.addr "bzip2 -c >/where/to/store/backup20090209.tar.bz2"

It is also possible to use the find command to collect the list of files and then pass that list to *tar --files-from=list.of.files ...*

Question#2: How to exclude folders for making a disk image?

With rsync it would be the --exclude option. In find it would be *! -regex*. In tar it would --exclude. It all depends on what tools you are using. Then again, tar may not transfer all the Mac OS X attributes. The Mac OS X rsync command includes the --extended-attributes option that can make sure you get everything.

Question#3: What caused the “rsync””Broken Pipe” errors?

This is very likely a broken network connection. That is to say the network connection was broken. You should check your /var/log files on BOTH the client and the server systems to see if there is anything that would indicate what went wrong that cause the connection to break. For example if the rsync on the server side died, it would drop its side of the network connection, and you would receive EPIPE error (Broken Pipe).

Dear Bob:
Nice posting!
Question for you:
#1
You mentioned:
cd /where/I/want/to/start
tar cf - ./ | ssh username@remote.system.addr "bzip2 -c >/where/to/store/backup20090209.tar.bz2"

Do “tar””bzip2” commands preserve the attributes of decade-old Adobe Type1 PS Fonts well?

#2
Max mentioned:
I am thinking about using “hdiutil” and “scp”=>
hdiutil create –srcdir /Users/user1 /Users/user1.dmg
scp –pv /Users/user1.dmg admin@FileServer:/Volumes/UserHomeFolderBackUp/

It seems to me that “hdiutil” is designed to run locally. How to combine “hdiutil””scp” in one, so as to copy the finished disk images to remote volumes automatically.

#3
Is there any way to use “find” or ”grep” or shell script or awk, or all the above, to exclude unwanted files, before executing “hdiutil” and/or “scp” commands?

My suggestions on fixing “rsync””Broken Pie”:
Instead of starting SSH session from the client, with the source folders, start SSH session from the Server, with the destination folders, and log onto the client, so as to initiate the “rsync” session to synchronize the source and the destination folders.

Merlin

Do “tar””bzip2” commands preserve the attributes of decade-old Adobe Type1 PS Fonts well?

Good question. If they are special Mac attributes or resource forks, I'm not sure that tar would do this (I'm not sure it would not either - I do not know).

Then again, you could try using "rsync -e ssh --extended-attributes" (man rsync) over an ssh connection.

You could also use "ditto --extattr --rsrc" (man ditto) specifying "-" as the src_archive or dst_archive

It seems to me that “hdiutil” is designed to run locally. How to combine “hdiutil””scp” in one, so as to copy the finished disk images to remote volumes automatically.

I would look to ditto or rsync for doing this over the network.

Is there any way to use “find” or ”grep” or shell script or awk, or all the above, to exclude unwanted files, before executing “hdiutil” and/or “scp” commands?

I do not know about hdiutil. rsync can use --include-from or --exclude-from, and ditto can use a --bom file.

My suggestions on fixing “rsync””Broken Pie”:
Instead of starting SSH session from the client, with the source folders, start SSH session from the Server, with the destination folders, and log onto the client, so as to initiate the “rsync” session to synchronize the source and the destination folders.

If you are having network connection problems, it does not matter where the connection is initiated. You need to figure out why you are getting broken connections.

If your server is going to automatically reach out and pull files from the client, then that is fine. I actually us a utility called rsnapshot from a server to make backups for my Mom's iMac (300 miles away) over an ssh connection.

Note: Some of this stuff, I'm getting by reading the man pages for ditto and rsync. It would most likely take me a lot of experiments to figure out how to get exactly what I want. In this case, you get to experiment 🙂