Listing File, path, sizes and dates

Question

Level 2

216 points

Listing File, path, sizes and dates

Hi all
Case: I have 6 machines on one network (some with 2 drives), another on another network (connected by vpn) and 3 external drives.
These all have accumulated stuff, some repeated as I've upgraded and re-purposed machines but left directories behind as a safeguard.
I need to rationalise the space and develop a more systematic approach to backup and archiving.
My first step is do an inventory of what's where and my natural approach is work with a data base of file name, path, size, created date and last modified to allow me to do some maths on archiving to dvd.
Using find as follows
find [Start Somewhere Directory] -print > [workspace path]/TestOutput.txt
gives me a nice list of file name I can parse out in a database
But I don't get size and dates.
Adding -ls produces that info but creates header lines for parent directories and adds permissions, node and other info I don't need.
I got to here
find [Start Somewhere Directory] -type df -exec ls -lshk {} \; -print > [workspace path]/TestOutput.txt
but still has the hierarchical output rather than flat paths.
Am I doing this all the hard way ? Is there a tool that returns just what I'm looking for ?
Or what command will allow me to take just the relevant columns form ls to the print parameter ?
Or can I extend find to add the size and date info to the output ?

Kind Regards
Eric

G5 at last :-), Mac OS X (10.4.6)

Posted on Aug 18, 2007 8:38 PM

Reply

Answer 1

Jun T.

Level 4

2,191 points

Aug 19, 2007 1:19 AM in response to Eric Wright

Hi Eric,

The -type option accepts only a single letter; "-type df" is equivalnet to "-type d" so only the directories are passed to "ls". What you need are not directories but files, so "-type f" will give you all the files with full path name.

But the problem is "ls -l" gives only the last modification date. If you also want the creation date, I guess you need to use Mac-specific commands like "mdls" or "GetFileInfo" (the latter is a part of Xcode Tools).

mdls -name kMDItemFSCreationDate flename
/Developer/Tools/GetFileInfo -d filename

See man pages of each command for more detail. But anyway you need to process the ouptputs by using "sed", "awk", "perl" etc., whichever you like.

Reply

Answer 2

Aug 19, 2007 8:17 AM in response to Eric Wright

Eric

I just so happened to have done something similar before! 🙂

It relies on mdls so isn't exactly speedy, but produces a full path, size, modification date, modification time, creation date and creation time as a comma separated list. mdls is not exactly predictable as to which order you get its output, so basically you have to try first without any editing.

Anyway, here it is:

sudo find ~/testfolder -type f \! -name ".*" -exec echo 'mdls -name kMDItemFSSize -name kMDItemFSCreationDate -name kMDItemFSContentChangeDate "{}" | tr "\n" "," | sed "s%^$/.*$ .*ChangeDate = $....-..-..$ $.*$ .CreationDate.= $....-..-..$ $.*$ ...00,.FSSize.= $.*$,$%\"\1\",\6,\2,\3,\4,\5%"' \; | sh

I'm sure it could be improved!

You could also do it with AppleScript, since that can access the creation date easily.

Reply

Answer 3

Aug 19, 2007 8:31 AM in response to Michael Conniff

I was going to suggest using find, but there is a lot more you can do with this command, like using -newercc and -newermm to compare files to a reference file by creation or modification date. find can also delete files that match the search criteria. It's a really powerful command.

Reply

Answer 4

b3and88

Level 1

70 points

Aug 19, 2007 9:19 PM in response to Eric Wright

Try ...

bash# find . -type df -exec stat {} \; > oufile

... and see if that's more to your liking.

This is a version of the stat command I've never seen before and it looks pretty good. There are a plethora of ways to format output.

Reply

Answer 5

Eric Wright Author

Level 2

216 points

Aug 20, 2007 1:01 AM in response to b3and88

Hi Jun et al 🙂
Thank you for the ideas.
I'll try a couple of variations tonight and tomorrow.
First thoughts are that stat is much quicker but doesn't natively delimit the output.
mdls is slow. I ran your're string Michael but with have to look at it to remove the labeling of dates etc. It delimits though which is good.
Jun you are right (but you know that) about df. When I think about it the aim is to get the files so f will do that and bring the path (directory names) with it.
Getting close.
Thanks all. I'll post when I get closer.
Kind Regards
Eric

Reply

Answer 6

b3and88

Level 1

70 points

Aug 20, 2007 1:41 AM in response to Eric Wright

The best way to do this is probably to write a simple perl script. The find2perl program will translate a find command into a perl script that you can easily modify to print the output in any format you want. Just type ...

bash# find2perl . -type df > myscript

... then edit myscript to print in whatever format you want. The generated perl script is very close to what you'll need. You'll need to change the return of lstat to give you more of the stat struct. This line ...

(($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&

... needs to become ...

(($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime) = lstat($_)) &&

... and then just modify the vanilla print statement ...

print("$name\n");

... to be something like ...

print("$name:$ctime:$mtime:$mode\n");

... or whatever you like. When you're done modifying the script ....

bash# chmod 755 myscript
bash# myscript > outfile

Using perl's find will be much faster than the shell version.

Reply

Answer 7

Jun T.

Level 4

2,191 points

Aug 20, 2007 3:32 AM in response to b3and88

The "ctime" returned by stat is the time when the inode was last changed, not the createion time of the file. AFAIK, the only commands which give the creation time are mdls and GetFileInfo.

Reply