Sorting jpeg files

Question

Sorting jpeg files

My Mac savvy neighbour helped me retrieve a bunch of pictures I accidentally deleted during an external hard drive back up using Data Rescue. The original library had about 17,000 pictures which was imported into my iPhoto library. During the retrieval, it somehow generated over 130,000 jpg files and in looking at them in Coverflow, I've discovered that some of these are just thumbnails of weird clips (like iTunes cover albums or thumbnails of clip art) and some are actually the pictures. I've tried sorting them by file size because my original images are usually about 1 - 3mb but in randomly clicking on several of the 130,000+ jpgs, I actually do have images that are as large as 900mb to as small as 40kb. Is there a program which could scan this salvaged jpg folder and identify ONLY the original files (e.g.: by using the camera's meta tag data or something?) Right now, going through them one by one is going to take me forever.

Any advice would be greatly appreciated.

MacBook Pro, Mac OS X (10.5.5), PC

Posted on Mar 1, 2009 1:59 PM

Reply

Answer 1

Mar 1, 2009 3:03 PM in response to karentam6600000

Hi, me again. A couple of other details I should probably include here. During the retrieval process in Data Rescue, it created a new set of file names starting with 1.jpg, 2.jpg, etc. all the way to over 131,0000. I've manually culled through some of the weird thumbnail jpgs that aren't images from my iPhoto library and now I am mainly seeing very small files (e.g:48kb) if images which I am assuming are just the thumbnails of the original files. Is there a way to isolate them to delete or would it be safe to assume that all files that are that small must have a larger equivalent that is the original file? I don't want to randomly delete in the event Data Rescue wasn't able to salvage some files and only was able to salvage a very low res of it.

Any suggestions on how best to automate this entire process of cleaning up my photos would be greatly appreciated it.

Reply

Answer 2

Yer_Man

Level 10

162,408 points

Mar 1, 2009 3:22 PM in response to karentam6600000

In list view in the Finder you can sort on Size.

This will quickly aloow you to rid yourself of the thumbnails and icons. Frankly, keeping anything less that 150kb or so is pretty pointless.

Regards

TD

Reply

Answer 3

Mar 1, 2009 5:10 PM in response to Yer_Man

Thanks for the tip Terrance. I can go through this process but with still over 120,000 files, it is pretty tedious. Is there another way to do this more quickly than simply looking at the list in Finder sorted by file size? I've already deleted about 5,000 files this way but it took a while because my machine was chugging slowly. Also, if I use the search function to search based on Canon EOS (the camera most of these were shot with), how exactly do I do that? I got as far as creating a folder called Searching"This mac".savedSearch. And finally, if I go through and get rid of everything that is 150kb and smaller, how do I figure out what to do with the rest? Can duplicate Annilator detect duplicates not within the iPhoto library? I need something to scan the folder with these 120,000+ files in an external drive. Will a duplicate detection software be able to identify various sizes of the same image? I think that may be what I'm dealing with - 17,000 originals multiplied exponentally as various file sizes.

Thanks, K

Reply

Answer 4

Yer_Man

Level 10

162,408 points

Mar 2, 2009 1:03 AM in response to karentam6600000

You can try this with Graphic Coverter.

I would strongly advise that you break the folder of 130k images down to more manageable sizes - say 500 or 1000 images. Otherwise you may choke the app with the sheer volume of mages.

The use Graphic Coverter to browse a Folder - File -> Browse Folder...

Give it a couple of moments to create the thumbnail images.

Then Edit -> Find, Compare, Replace -> Find File

This gives you a dialogue

From the DropDown menu at the Bottom select

Exif Metadata -> contains -> Canon

This will find all the images in the folder with Canon in the Exif - drag those images to a new Folder and trash the other one.

Now you can use the Find Duplicate command on the search result.

Regards

TD

Regards

TD

Reply

Answer 5

Mar 2, 2009 8:16 AM in response to Yer_Man

Thanks for the tip. I will download this and give it a shot. About half of the 130,000 images made it into the iPhoto library so I've been manually deleting them but your suggestion seems much more efficient. Once I edit the folder, should I do another iPhoto library import and will it only import those not already imported?

Reply

Answer 6

Yer_Man

Level 10

162,408 points

Mar 2, 2009 8:29 AM in response to karentam6600000

I don’t think you should go anywhere near iPhoto until you have found the required files.

Regards

TD

Reply

Answer 7

Mar 2, 2009 9:45 AM in response to Yer_Man

Thanks for all your help so far. I'm in the process of breaking down the 130,000 files into folders containing 500-1000 images so I do not choke Graphic Converter. The way I'm grouping the images is by grabbing them in sequential order (Data Rescue generated a new numbering system so there appears to be various ways in which these files are named now). When I go through the Graphic Converter process to compare images by each folder, will I run the risk that a duplicate image is now on a different folder in which case I may still have many duplicates? If so, any suggestions on how to handle this?

Thanks, Karen

Reply

Answer 8

Yer_Man

Level 10

162,408 points

Mar 2, 2009 10:56 AM in response to karentam6600000

The process I described above will find the Canon made photos in the Mess.

They should be a lot less than 130,000.

When you have that done - and it will take a while - see how many you have left, and using the GC Help, I’m sure you can find a way to search the Canon files for duplicates, and how to include nested folders in the search.

Regards

TD

Reply

Answer 9

Mar 2, 2009 11:23 AM in response to karentam6600000

Perhaps I am missing something, but why don't you just search your 130,000 files using the advanced search criteria function in Finder? In Finder, you can search for "All Images". Then, click on the action button (the little cog), which should give you the option "Show Search Criteria". Click on this, and it brings up three more toggle switches: "Kind", "Images" and "All". If you change "Kind" to "Other", Finder will generate a list of 50-100 attributes that you can search for, including Device Make, Device Model, Exposure Time, Created Date, etc. I think if you play around with the search criteria, you should be able to find some combination which separates the real files from the copies that have been made.

Reply

Answer 10

Yer_Man

Level 10

162,408 points

Mar 2, 2009 11:45 AM in response to Greg Coleman

Greg

Dead right! Spotlight will find Canon on Device Make in that folder. I missed as I was looking for Exif headers.

Regards

TD

Reply

Answer 11

Mar 2, 2009 12:14 PM in response to Yer_Man

Thanks both. I will try this instead. Do I still need to break down the 130,000 files into smaller folders then to do so?

Also, will the copies and thumbnails just not have any meta data in them? I actually owned 2 cameras, a Fujifilm one at the beginning of my digital photography days and then upgraded to a Canon SLR about 3 years ago, so I'll do the same with the Fuji pics.

Reply

Answer 12

Old Toad

Level 10

207,751 points

Mar 2, 2009 12:33 PM in response to karentam6600000

K:

You could try using the demo version of Media Expression. If the EXIF data is intact EM can sort and find photos based on the camera used as well as many other criteria.

TIP: For insurance against the iPhoto database corruption that many users have experienced I recommend making a backup copy of the Library6.iPhoto (iPhoto.Library for iPhoto 5 and earlier versions) database file and keep it current. If problems crop up where iPhoto suddenly can't see any photos or thinks there are no photos in the library, replacing the working Library6.iPhoto file with the backup will often get the library back. By keeping it current I mean backup after each import and/or any serious editing or work on books, slideshows, calendars, cards, etc. That insures that if a problem pops up and you do need to replace the database file, you'll retain all those efforts. It doesn't take long to make the backup and it's good insurance.
I've created an Automator workflow application (requires Tiger or later), iPhoto dB File Backup, that will copy the selected Library6.iPhoto file from your iPhoto Library folder to the Pictures folder, replacing any previous version of it. There are versions that are compatible with iPhoto 5, 6, 7 and 8 libraries and Tiger and Leopard. Just put the application in the Dock and click on it whenever you want to backup the dB file. iPhoto does not have to be closed to run the application, just idle. You can download it at Toad's Cellar. Be sure to read the Read Me pdf file.

NOTE: iPhoto 8's new option in it's rebuild library window, "Rebuild the iPhoto Library Database from automatic backup" may make this tip obsolete. We'll know when users have occasion to use it and see if that's the case.

Reply

Answer 13

Mar 3, 2009 9:36 AM in response to karentam6600000

I've started to use Graphic Converter to find duplicates and that's working great. However, the process of breaking down the 130,000 files is really tedious and time consuming (I'm creating a new folder, selecting 1000 files and then dragging them into the folder). How do I use Apple Script to automate this process? I've been naming the folder names by the first and last file name (e.g.: 1-999, 1000 - 1999, etc.).

tx

Reply

Answer 14

Mar 4, 2009 9:37 PM in response to karentam6600000

Per the earlier post, I've started the process of breaking up the 130,000+ files into folders as there are some corrupted files now that I've been taking a closer look at the thumbnails. The files are in numeric order so I'm wondering if there is a way to automate the process of grouping the files by file name (e.g.: 1-999.jpg, 1000-1999.jpg, etc.) so that it isn't such a tedious task. My computer chokes each time it has to try and load the thumbnails of 130,000+ files so looking at them in smaller groups of 1000 is much better.

Any advice?

Thanks,
Karen

Reply