compress data through a pipe

Hi all,

I have a command which is going to spew a whole lot of output (>1 G). Rather than writing that all to disk and then compressing it with tar, is there a way to compress straight thru a pipe?

I'd like to be able to do something like:

command | tar -cvzf -

but of course that doesn't work because tar is looking for a file name, not a stream of data.

Any ideas?

Thanks!

iMac, Mac OS X (10.5.4)

Posted on Aug 24, 2008 4:26 PM

Reply
6 replies

Aug 24, 2008 5:11 PM in response to etresoft

Thanks! I got gzip to work. Strange, I had tried it before but was doing something wrong I guess.

But I'm not so sure about tar only working with streams. If that's so, then what's wrong with any of this?

+mbp:~ me$ ls -al | tar -cvz > tared8+
+tar: Cowardly refusing to create an empty archive+
+Try `tar --help' or `tar --usage' for more information.+

+mbp:~ me$ ls -al | tar -cvzf tared8+
+tar: Cowardly refusing to create an empty archive+
+Try `tar --help' or `tar --usage' for more information.+

+mbp:~ me$ ls -al | tar -cvzf tared8 -+
+tar: -: Cannot stat: No such file or directory+
+tar: Error exit delayed from previous errors+

Aug 24, 2008 5:31 PM in response to Gorgan J

I should have said that tar's input (for archiving) or output (for unarchiving) is a stream by default. You would need different options with both "ls" and "tar" to get your examples to work. Try: ls -1 | tar -cvzf tare8 -T -

Tar can be very powerful with used with pipes. You can tar entire directories, selecting certain files, and pipe the output into another tar that will unarchive those files in a new location, for example.

Aug 24, 2008 6:11 PM in response to etresoft

Sorry, I think I've confused you. I used ls -al just for an example of a command generating output. I don't want to tar up the whole directory. I want to tar up the output of a command without having to write it to a file first.

I'm working with a proprietary package that gives me a command to dump out log information. The log information will be > 1 G. Rather than redirecting the command output to a file and then taring the file, I'd like to just tar the data directly through a pipe.

gzip works just fine:
command | gzip -c file1.Z

But tar won't work because it expects to work with a stream of file names, not streams of data.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

compress data through a pipe

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.