Can I copy files – but with certain restrictions?

I have just finished a large project, the archives of which involves about 5000 "base" files, stored on about 80 CDs and 50 DVDs, involving about 50,000 files in total. Each of the "base" files may have had up to 30 incremental versions. i.e. a certain text file may have undergone revision 23 times, and each revision was saved and archived to (probably) a different disk, with a different suffix – a, b, c and so on. But sometimes the suffix didn't change even though the file was edited. I might have done a bit more dust removal on an image and just overwrote the old file (already archived), and so the new one was archived on a different disk.

I now have 130 disks from which I would like to extract all the files and collapse them to one large archive that will probably span about 20 disks by the time I delete some files not needed. That way I can easily search for all versions of, say, GB097, by going to the particular DVD that has the "G" files on it. Up would come:

GB097
GB097a
GB097b
GB097b-1
GB097b-2
GB097c
... and so on.

This is what I would like to do:

1. Grab the first archive disk, open every folder, and copy all the files to the one folder on a hard drive.

2. Open the second disk and repeat step (1), but with these two provisos.

(a) If a file is identical to a previously copied file (maybe I archived it twice), the file isn't copied. However...

(b) If a file has the same name as a previously copied file, but the data within that file is different (i.e. I removed some dust from an image file, but left the name unchanged), I'd like that file to be copied with a numbered suffix, the same way that Trash treats identically named files.

Any suggestions how I could do this?

G5 iSight, Mac OS X (10.4.11)

Posted on May 18, 2010 3:27 AM

Oct 13, 2010 3:06 PM in response to rccharles

Hello again, sorry for the delay - I've been swamped with work. {quote:title=Guy Burns wrote:}> How can two files identical in every respect occupy the same folder? But there it is, I've just had a look. In my Trash is: ... C22 C22 C22 17-27-48 ... The last file, named C22, I sent to Trash from my Test folder to see what happened. It got renamed.{quote} Because when you look in the Trash can via the GUI, it is displaying items from your user Trash folder as well as items from your trash folder on each different volume. In other words, the two files with the same names are not actually in the same folder. (Get Info on them and look at the Where field.) You can simulate this by creating a new folder on each of two hard drives, and using the same name for both folders. Then trash them each and look in the Trash to find two folders with the exact same name. I did all my testing of the script with the data folder on the boot volume so I never noticed a problem 'mv'ing items to ~/.Trash since the item was still on the same volume. However, the main problem with how I originally implemented the trash routine is that if the data folder is on a different volume then the item is copied to the boot volume as trash and then deleted from the data folder (by mv.) Sooo, perhaps the script was stopping because the boot volume filled up, not the data volume? Anyway, instead of mv'ing the item to the Trash (and to which .Trash), or rm'ing the item completely, we could avoid the whole issue and let Finder deal with the problem (in which case it will handle the correct location for .Trash and also handle duplicate name issues.) Replace this: mv "${FilePath[${j}]}" ~/.Trash 2>/dev/null || mv "${FilePath[${j}]}" ~/.Trash/$RANDOM With this, which will decrease overall speed of the script when files are to be trashed and will very likely have problems with filenames that begin with a . (period.): osascript -e "tell app \"Finder\" to delete POSIX file \"${${FilePath[${j}]}}\"" In practice, it would probably be best to ensure that the trash is empty first anyway - since the point of moving the items to trash rather than deleting them is to allow for recovery of the files being trashed by the script in the event something goes horribly wrong. If the trash is already full of items, it might be difficult to determine which might be from the data folder. {quote:title=rccharles wrote:}if you put your data at /mydata the trash would be in your path.{quote} I see what you mean, if the script is given a volume as the folder to process then the hidden .Trash folder on that volume would indeed be within the path. Easiest way to prevent that would be to add a line after these two: [ ! -d "${MainFolder}" ] && echo "Folder not found." && exit 3 [ "${MainFolder}" = "/" ] && echo "You must be crazy." && exit 4 such as: [ $(dirname "${MainFolder}" ) = "/Volumes" ] && echo "Please select a subfolder not a volume." && exit 5 {quote:title=Guy Burns wrote:}I'm no programmer, just a tester. I used to be able to program. I remember the quaint process of punching holes in 80-column punch cards for feeding into a PDP-8 (a DEC minicomputer). It's amazing what you can do with a foot-high stack of punch cards. I've still got some of the printouts. They're historic. I was pretty good at FORTRAN and BASIC, but these new languages... My goodness. What's a sane person supposed to make of (for instance): mv "${filePath${j}}" ~/.Trash 2>/dev/null. Too many incomprehensible brackets and squiggles for my liking.{quote} The main system at my first job was a DEC 1145 🙂 (With gigantic 10 MB Winchester drives instead of punch cards.) -- The double-quotes protect spaces and special characters within the file and folder names of the path while still allowing variables and certain wild-cards characters to be used. Single quotes would protect the path but not allow variables and wild-cards to be used. -- The {} are used to indicate that everything inside them is part of the variable being used. In some cases, bash can't differentiate between what pertains to a variable and what is merely static text... for instance: $ test=hello $ test[1]=goodbye $ echo $test hello $ echo $test[1] hello[1] $ echo ${test[1]} goodbye The curved braces (or curved brackets) made it clear to bash that we wanted it to echo/display the value stored in the array named "test" at cell #1 and not the default value (at cell 0) of the array test followed by the text "[1]". {quote:title=Bruce Bathurst wrote:}This is the most difficult of the Unix (or Linux) shell languages. It's used by programmers writing operating system files that won't change, and need to run fast.{quote} Bash is the default shell on OS X since 10.3 (prior to that it was sh / tcsh.) I tend to use bash for examples because it -should- be something that Mac OS X users become familiar with (in my opinion, since it is the default) and it requires nothing more than Terminal and a text editor. No need for a user to install Xcode/Developer Tools or other IDE, no need to compile etc. Plus, much of the knowledge that an OS X user might learn from using the command line or bash scripts may be helpful to the user in general (again, because it's the default shell on Macs.) Glad to see the development continuing and I hope the script worked for Guy Burns as desired. 🙂