1 2 Previous Next 26 Replies Latest reply: Apr 27, 2006 11:19 AM by Michael Conniff Go to original post
  • 15. Re: Ditto command wont copy permissions
    biovizier Level 5 Level 5 (7,925 points)
    Maybe I missed it, but I haven't seen mention of what protocol is being used to mount the shares. From the description though, I suspect this all has to do with the convoluted "translation" that afp (server together with client) perform to give the person on the client side access to files on the server side regardless of their respective 'uid' at either end.

    Without going into details, some of the variables might be the whole issue of "translation" (I don't know what the official word for this process is) of ownership, "translation" of permissions, differences in 10.4 vs 10.3 with respect to how items owned by user "unknown" (ie "uid:gid" of "99:99") are handled, and differences between how a regular user and "root" see items owned by "99:99"... for starters. This can result in apparent oddities such as, for example, a file on the server with "070" permissions appearing to be "700" to the client, depending on the respective group memberships. So if the user on the server whose credentials are used to connect to the server has permissions to an item, the client side user who used those credentials will one way or another get permissions to access the item, but the actual mode used might be different.

    So I suspect that 'ditto' actually is preserving the permissions, but as it sees them from the client side.

    But given all of the variables, I'm not sure if it will be possible to do this migration over afp. I don't know - 'scp' might work but I don't know how to get it to preserve ownership; firewire target disk mode might be another possibility.
  • 16. Re: Ditto command wont copy permissions
    hyphen Level 6 Level 6 (15,345 points)
    biovizier, Michael

    Wouldn't cp -p noe that it can handle resource forks?

    To be honest, I never paid any attention to resource forks; I never had any use for them.
  • 17. Re: Ditto command wont copy permissions
    biovizier Level 5 Level 5 (7,925 points)
    I think 'cp -p' would work to preserve permissions for a volume that is actually mounted directly by the local computer, like a USB or FireWire drive, or even a mounted ".dmg". Shares mounted over afp on the other hand are mounted by the server computer then shared out to the client (with all of the contortions to "translate" ownership and permissions mentioned above) so I have a feeling that 'cp' would have the same problems as 'ditto' in that it would try to preserve the ownership it thinks it sees from the client side rather than the "true" permissions (assuming that that's what the problem with 'ditto' is).

    In addition, I'm not sure if 'cp' preserves ownership when copying across volumes - 'ditto' appears to but I can never keep track of these differences between various tools...

    Looking at the 'man' page, it looks like 'ditto' might still be useful if used to create an archive first as "root" on the old machine. It should then be possible to move the archive over via afp, after which it can be expanded, as "root" on the new machine. There's probably an easier way using pipes or something but I don't know how off hand - and it might require connecting remotely as "root" which doesn't seem ideal. If the archive is transferred, it would only require 'sudo' at each end. There are apparently other unix tools that might be applicable ('pax', 'rsync' maybe?) but their relative pros and cons and behaviours with respect to 10.3 and 10.4 are beyond me. So actually suggesting a strategy is something I'll leave to someone else - my main purpose posting earlier was to point out the idiosyncrasies of afp when it comes to permissions and ownership.
  • 18. Re: Ditto command wont copy permissions
    baltwo Level 9 Level 9 (60,115 points)
    Carbon Copy Cloner uses the ditto command to do its copying. CCC doesn't work over networks because there are too many difficulties in getting the proper set of privileges to make this possible. That's probably why you're seeing the problem.
  • 19. Re: Ditto command wont copy permissions
    hyphen Level 6 Level 6 (15,345 points)
    Thanks biovizier
  • 20. Re: Ditto command wont copy permissions
    Michael Conniff Level 7 Level 7 (33,120 points)
    biovizier

    Thanks for the contribution!
    the convoluted "translation" that afp (server together with client) perform
    Oops! I hope it isn't that, but it could well be the explanation.
    I'm not sure if 'cp' preserves ownership
    Well, it tries to if you use '-p', but doesn't warn you if it is unable to do so.
    firewire target disk mode might be another possibility.
    Yes, that should avoid these difficulties, particularly in the light of baltwo's comment.

    Let's wait for the OP to reply.
  • 21. Re: Ditto command wont copy permissions
    bbauer Level 1 Level 1 (0 points)
    Hi Michael, Here's what you asked for...

    New server starts here:
    BSICS1:~ admin$ sudo -u root ls -al /Volumes/
    total 8
    drwxrwxrwt 7 root admin 238 Apr 26 08:02 .
    drwxrwxr-t 33 root admin 1224 Apr 19 07:00 ..
    drwxrwxrwx 13 admin staff 544 Apr 26 13:37 G5 SubSystem
    drwxr-xr-x 3 admin admin 102 Apr 24 15:11 G5 SusSystem
    drwx------ 11 admin unknown 476 Sep 28 2005 Retrospect 6.1
    lrwxr-xr-x 1 root admin 1 Apr 14 08:00 Server HD -> /
    drwxr-xr-x 37 admin unknown 1214 Apr 20 15:09 Users

    BSICS1:~ admin$ sudo -u root nireport . /users uid gid realname home | awk '$1>500 {print $0}'
    Password:
    501 20 Administrator /Users/admin
    511 511 angel /Users/angel
    505 505 april /Users/april
    1033 20 brian /Users/brian
    509 509 brian2 /Users/brian2
    1029 20 buddy /Users/buddy
    503 503 chelsie /Users/chelsie
    1042 20 chuck /Users/chuck
    1039 20 danielle /Groups/danielle
    510 510 dean /Users/dean
    1035 20 doug /Users/doug
    1041 20 internmac #NoValue#
    1031 20 jacque /Users/jacque
    512 512 jamie /Users/jamie
    508 508 joe /Users/joe
    1040 20 julie /Users/julie
    1030 20 justin /Users/justin
    1043 20 kim /Users/kim
    1038 20 meech /Users/meech
    1032 20 michelle /Users/michelle
    507 507 michelleb /Users/michelleb
    1028 20 mike /Users/mike
    506 506 mindy /Users/mindy
    1034 20 nate /Users/nate
    1037 20 nicole /Users/nicole
    1027 20 pete /Users/pete
    1025 20 scanner two /Users/scannertwo
    504 504 shannon /Users/shannon
    1026 20 tanya /Users/tanya
    1079 20 Jennifer #NoValue#

    BSICS1:~ admin$ sudo -u root diskutil info /
    Device Node: /dev/disk0s3
    Device Identifier: disk0s3
    Mount Point: /
    Volume Name: Server HD

    File System: Journaled HFS+
    Journal size 8192 k at offset 0x6401000
    Owners: Enabled
    Partition Type: Apple_HFS
    Bootable: Is bootable
    Media Type: Generic
    Protocol: ATA
    SMART Status: Verified
    UUID: 3795C11B-5570-33AA-BDCD-6C020C377A33

    Total Size: 76.6 GB
    Free Space: 70.3 GB

    Read Only: No
    Ejectable: No
    Device Location: "Bay 1"

    - - - - Old server starts here:- - - - - - - - - - -

    DesignServicesServer:~ server$ nireport . /users uid gid name realname home | awk '$1>500 {print $0}'
    501 501 server Server /Users/server
    1027 20 pete pete /Users/pete
    1028 20 mike mike /Users/mike
    1029 20 buddy buddy /Users/buddy
    1030 20 justin justin /Users/justin
    1031 20 jacque jacque /Users/jacque
    1032 20 michelle michelle /Users/michelle
    1033 20 brian brian /Users/brian
    1034 20 nate nate /Users/nate
    1035 20 doug doug /Users/doug
    1037 20 nicole nicole /Users/nicole
    1038 20 meech meech /Users/meech
    1039 20 danielle danielle /Groups/danielle
    503 503 chelsie chelsie /Users/chelsie
    1026 20 tanya tanya /Users/tanya
    504 504 shannon shannon /Users/shannon
    505 505 april april /Users/april
    506 506 mindy mindy /Users/mindy
    507 507 michelleb michelleb /Users/michelleb
    1025 20 scannertwo scanner two /Users/scannertwo
    502 502 jennifer jennifer /Users/jennifer
    1041 20 internmac internmac #NoValue#
    508 508 joe joe /Users/joe
    509 509 brian2 brian2 /Users/brian2
    510 510 dean dean /Users/dean
    511 511 angel angel /Users/angel
    512 512 jamie jamie /Users/jamie
    1040 20 julie julie /Users/julie
    1042 20 chuck chuck /Users/chuck
    1043 20 kim kim /Users/kim

    DesignServicesServer:~ server$ diskutil info /
    Device Node: /dev/disk0s3
    Device Identifier: disk0s3
    Mount Point: /
    Volume Name: Server HD

    File System: Journaled HFS+
    Journal size 8192 k at offset 0x6401000
    Permissions: Enabled
    Partition Type: Apple_HFS
    Bootable: Is bootable
    Media Type: Generic
    Protocol: ATA

    Total Size: 233.6 GB
    Free Space: 16.6 GB

    Read Only: No
    Ejectable: No
    Device Location: Bay 1

    I don't know if this matters or not but, I thought that I would mention it... The location were I'm copying to on the new server is within a volume that is on an "XServe RAID" system. Not sure if that might be causing any problems..??? Total free space is 700GB. The volume is called "G5 SubSystem" which you probably already know.

    I did try transferring the data using Retrospect but I don't have the correct version in order to be able to backup another server. I might just purchase what they suggest and transfer the data that way. They say that Retrospect can retain the permissions as well, have you heard that?
    I hope all this helps..!!!! Thanks...Bob
  • 22. Re: Ditto command wont copy permissions
    Michael Conniff Level 7 Level 7 (33,120 points)
    Bob

    Much to my surprise, your two lists of users are not that different:

    1c1,2
    < 501 20 Administrator /Users/admin
    ---

    501 501 server /Users/server
    502 502 jennifer /Users/jennifer

    12c13
    < 1025 20 scanner two /Users/scannertwo
    ---
    1025 20 scannertwo two

    30d30
    < 1079 20 Jennifer #NoValue#</p>I wonder if you thought of using Migration Assistant to handle this? You wouldn't have had any differences then.

    There are some other odd things – why "G5 SubSystem" and "G5 SusSystem"? And why a "/Users" folder inside "Volumes"? There are also too many "unknown" groups for my liking.
    The location were I'm copying to on the new server is within a volume that is on an "XServe RAID" system.
    Did you check in the Xserve RAID forum? I don't know much about this implementation of RAID. Nor do I know much about Retrospect these days: it must be 10 years since I used it.

    biovizier and baltwo raised some interesting questions. It may be worth experimenting by copying just one file (or a small folder) and seeing what happens to ownership/permissions when you do this in different ways.
  • 23. Re: Ditto command wont copy permissions
    biovizier Level 5 Level 5 (7,925 points)
    If the output of:
    $ sudo -u root ls -al /Volumes/
    on the new machine included this:
    drwxr-xr-x 37 admin unknown 1214 Apr 20 15:09 Users

    and "Users" is the volume mounted from the old machine, that would be normal for an afp mount. There may be exceptions but in general, at the afp client end the volume seems to be mounted giving ownership to the user that did the mounting, but "group" retains the characteristic "99" which is a part of the "translation" voodoo I was referring to earlier (items owned by "99" always appear to be owned by the owner of the process attempting to access them - except to "root" who can see through the subterfuge). So to user "admin", the ownership appears to be "admin:admin", but to "root", it appears as "admin:99". Assuming the "Users" folder contains folders owned by different users, if you were to compare 'sudo ls -al /Volumes/Users' on the new computer to the output from the equivalent command for the directory executed on the old computer, the difference would probably jump out at you - everything will appear to be owned by "admin:unknown" on the client side, whereas on the server end, there would presumably be different users and groups, reflecting the real ownership.

    I've been fiddling a bit and using 'ditto', and doing everything as "root" might work some of the time, but not reliably. The "allowRootLogin" setting in afp still seems flakey - maybe it's better in OS X Server, but with the client, it seems to do something different every time a connection is made, especially with how it behaves with respect to access to nested folders. Leaving "root" out of it and using 'sudo ditto -c --rsrc' on the 10.3 end to create an archive, transferring the archive to the new computer as any user over afp, and 'sudo ditto -x' on the 10.4 machine to expand the archive is clunky, but seems at least to get everything over with the correct ownership / permissions, though there is probably an easier way...
  • 24. Re: Ditto command wont copy permissions
    bbauer Level 1 Level 1 (0 points)
    To all:
    Looks like I've been approved to purchase an upgrade to our Retrospect software (to server version 6.1, we currently have the workgroup version). I think I'm more familiar with that then the Terminal command line stuff. So hopefully it does the trick. I'm still going to continue to work with the ditto command to see how far I can get. It's bugging me that it's not working when Apple says it should be. They want $695.00 for one incident and between $6k and $7k for a year contract....WOW. Youe get 90 days free support but that doesn't included command line support, go figure. I know Microsoft is $245 per incident. That just blew my mind when I heard their price of $695. Also, I wish could use the "archive" option but, I'm really low on disk space on the old server. I guess I could do it in pieces but I'm also running out of time to get this all done. The old server is leased and it's time for it to go back. I wonder how the Migration Assistant would have went..?? I tried to find that and I can't, it's not under Applications or Utilities, somehow it's missing. I wasn't the one that first turned on the server so maybe that's why I can't find it? Thanks...Bob
  • 25. Re: Ditto command wont copy permissions
    biovizier Level 5 Level 5 (7,925 points)
    That's good news about the Retrospect, but shocking about Apple's support rates, especially compared to MS!

    Regarding your other comments:
    I've never used it, but "Migration Assistant.app" does appear to be in "/Applications" > "Utlitites" on my OS X client machine, but maybe the server has something different.

    As for the archive function of 'ditto', from tinkering, it does seem to allow the creation of the archive over afp, as long as the destination is world writable and its full path is world accessible, such as "/Users/Shared" (but any folder will do as long as it meets the accessibility requirement). Since the archive is being created over afp, space on the old machine shouldn't be an issue (and hopefully the new system doesn't have a space issue yet).

    For example, on the old machine, connect as an "admin" to the new machine and mount the boot volume (in this example) of the new machine - using "Finder" is fine for this part. Then make the archive with this command (entered on the old machine):<pre>sudo ditto -c --rsrc /path/to/Users /Volumes/ShareName/Users/Shared/archivename</pre>

    Then expand the archive with 'sudo', entering the command on the new machine, eg:<pre>sudo ditto -x /Users/Shared/archivename /Volumes/G5\ SubSystem/Usersditto</pre>

    This does seem to preserve ownership, permissions and resource forks at least one folder deep. However, it isn't a command I normally use and I haven't tested it thoroughly, so it would be best to consider this only as a starting point for your own testing...

    Edit: Somewhat belatedly, it occurs to me that you might get some better answers asking in the "server" section of this site...
    http://discussions.apple.com/category.jspa?categoryID=96
  • 26. Re: Ditto command wont copy permissions
    Michael Conniff Level 7 Level 7 (33,120 points)
    Bob
    I wonder how the Migration Assistant would have went..?? I tried to find that and I can't,
    It should be in /Applications/Utilities. You could always download Pacifist and use that to extract it from your Mac OS X Install DVD (it comes with good instructions).

    Otherwise good luck with the new Retrospect!
1 2 Previous Next