9 Replies Latest reply: Sep 4, 2012 6:55 PM by roam
iMac-newbie Level 1 Level 1 (0 points)

I have huge high definition video files on a 2TB external hard drive (and its clone).  The external hard drive is maxed out.  I would like to move many of the video files to a new 3TB external hard drive (G-drive, and a clone) and leave a sub-group of video files (1+ TB) on the original external hard drive (and its clone).  

 

I am copying files from original external drive ("ext drive A") to new external drive ("ext drive B") via Carbon Copy Cloner (selecting iMovie event by event that I want to transfer). Just a note: I do not know how to partition or make bootable drives, I see suggestions with these steps in them.

 

My questions:

 

1.)  I assume this transfer of files will create extreme fragmentation on drive A.  Should I reformat/re-initialize ext drive A after moving the files I want?  If so, how best to do this?  Do I use "Erase" within Disk Utilities?  Do I need to do anything else before transfering files back onto ext drive A from its clone?

 

2.) Do I also need to defrag if I reformat ext drive A? Do I defrag instead of or in addition to reformating?  If so, how to do this? I've read on these forums so many warnings and heard too many stories of this going awry.  Which 3rd party software to use? 

 

Thank you in advance for any suggestions, tips, advice.  This whole process makes me SO nervous.


iMac, Mac OS X (10.6.8)
  • 1. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    BDAqua Level 10 Level 10 (116,470 points)

    A clone should defrag the files.

     

    Apps that can defrag...

     

    Drive Genius…

     

    http://www.prosofteng.com/products/drive_genius.php

     

    Intech SpeedTools Utilities 3…

     

    http://eshop.macsales.com/item/Intech%20Software/STUPROU/

     

     

    And iDefrag which I havent tried.

  • 2. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    roam Level 6 Level 6 (13,545 points)

    Hi Mac newbie,

    It sounds like you are doing OK, just on defragmentation, it is a bit of an obsession with ex-Windows users that causes little concern with Mac users. It just doesn't excite most Mac users.

     

    Having said that I'll try and answer some of your queries.

     

    I do not know how to partition or make bootable drives

     

    A drive can be partitioned (sliced up into separate volumes that appear as different drives) in Disk utility

    A bootable drive is one that has an operating system installed on it. When you start your computer you are booting into that volume. An external drive that  only used to store media etc, is just a storage disk.

     

    Should I reformat/re-initialize ext drive A after moving the files I want?

    No don't do that, because reformatting will destroy all the data that remains on drive A. so losing everything remaining on there.

    If after the transfer of files there is nothing at all left on there, then reformatting is pointless, because it is already empty.

     

    Basically you do not need to defragment because Mac OS X is quite good at avoiding fragmentation with it HFS+ file system. 

     

    Check this link, it is pretty old but is still relevant.

    http://www.brighthub.com/computing/mac-platform/articles/42616.aspx

     

    Message was edited by: roam

  • 3. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    iMac-newbie Level 1 Level 1 (0 points)

    Roam, thanks for this thorough reply.  I thought I should reformat drive A after removing ~1/2 the files to the new external drive, and then copy back the other 1/2 of the files to drive A from the clone drive after reformatting. I do understand defrag is not nearly an issue with the Mac like it is with Windows, but since I am removing huge video files, I thought this might be the time it would be a good idea to start fresh and not leave gaps throughout...or are you saying it still really doesn't matter?  Will reformatting the drive make defrag moot anyway?

     

    BDAqua:  does cloning defrag the source or the destination drive? And if so, then no need to use 3rd party defrag software?  Thank you.

  • 4. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    roam Level 6 Level 6 (13,545 points)

    Reformat is the same as Erase. If you copy half the files from A to B, both drives will have data on them that you do not want to erase.

     

    If you had a scattering of files on A and an empty drive called B, and you randomly copied from A to B, B would not be fragmented. 'A' will still be fragmented.

    If you copy all the data off A and leave it empty, when you copy it back it will in effect be defragged.  If you don't have the space to completely empty A, what remains on A will be fragmented. Reformatting it will erase it. Re-arrangement of those remnants on 'A' without erasure requires Defragmentation software. You only need to use such software if you feel you must. I doubt if you will see any performance boost from doing it.

     

    A clone will also produce a defragmented product on the destination drive. So if you cloned A to B and then B back to A , you would be assured A would be not fragmented.

     

    However the whole fragmentation question is largely irrelevant because of modern disk access speeds.

     

    some more study 

    http://support.apple.com/kb/HT1375

  • 5. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    iMac-newbie Level 1 Level 1 (0 points)

    Thank you again, Roam.  Okay, here is my remaining question:

     

    I will delete video files from drive A, leaving ~50% of the 2TB hard drive space remaining, but the files that are deleted are randomly scattered throughout the drive, they are not consecutive.  Is it okay to leave the remaining files on the hard drive AS IS - OR- better to reformat the drive then copy the files back on from the backup drive?  My concern with fragmentation does not have to do with boosting speed, just about optimizing storage of new files on the drive and not having files split into different places unnecessarily.  I don't know if the risk is greater to have a somewhat fragmented drive or all the movement and copying of files back and forth?  Is it a good idea to wipe a drive clean/reformat every couple of years (after taking necessary backup precautions) or is that not necessary?  Thanks in advance for any feedback...

  • 6. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    RRFS Level 5 Level 5 (4,430 points)

    Here is a very good writeup on de-fragging in the OS environment that I borrowed

    From Klaus1:

     

    Defragmentation in OS X:

     

    http://support.apple.com/kb/HT1375  which states:

     

    You probably won't need to optimize at all if you use Mac OS X. Here's why:

    • Hard disk capacity is generally much greater now than a few years ago. With more free space available, the file system doesn't need to fill up every "nook and cranny." Mac OS Extended formatting (HFS Plus) avoids reusing space from deleted files as much as possible, to avoid prematurely filling small areas of recently-freed space.
    • Mac OS X 10.2 and later includes delayed allocation for Mac OS X Extended-formatted volumes. This allows a number of small allocations to be combined into a single large allocation in one area of the disk.
    • Fragmentation was often caused by continually appending data to existing files, especially with resource forks. With faster hard drives and better caching, as well as the new application packaging format, many applications simply rewrite the entire file each time. Mac OS X 10.3 onwards can also automatically defragment such slow-growing files. This process is sometimes known as "Hot-File-Adaptive-Clustering."
    • Aggressive read-ahead and write-behind caching means that minor fragmentation has less effect on perceived system performance.

    Whilst 'defragging' OS X is rarely necessary, Rod Hagen has produced this excellent analysis of the situation which is worth reading:

     

    Most users, as long as they leave plenty of free space available , and don't work regularly in situations where very large files are written and rewritten, are unlikely to notice the effects of fragmentation on either their files or on the drives free space much.

     

    As the drive fills the situations becomes progressively more significant, however.

     

    Some people will tell you that "OSX defrags your files anyway". This is only partly true. It defrags files that are less than 20 MB in size. It doesn't defrag larger files and it doesn't defrag the free space on the drive. In fact the method it uses to defrag the smaller files actually increases the extent of free space fragmentation. Eventually, in fact, once the largest free space fragments are down to less than 20 MB (not uncommon on a drive that has , say only 10% free space left) it begins to give up trying to defrag altogether. Despite this, the system copes very well without defragging as long as you have plenty of room.

     

    Again, this doesn't matter much when the drive is half empty or better, but it does when it gets fullish, and it does especially when it gets fullish if you are regularly dealing with large files , like video or serious audio stuff.

     

    If you look through this discussion board you will see quite a few complaints from people who find that their drive gets "slow". Often you will see that say that "still have 10 or 20 gigs free" or the like. On modern large drives by this stage they are usually in fact down to the point where the internal defragmentation routines can no longer operate , where their drives are working like navvies to keep up with finding space for any larger files, together with room for "scratch files", virtual memory, directories etc etc etc. Such users are operating in a zone where they put a lot more stress on their drives as a result, often start complaining of increased "heat", etc etc. Most obviously, though, the computer slows down to a speed not much better than that of molasses. Eventually the directories and other related files may collapse altogether and they find themselves with a next to unrecoverable disk problems.

     

    By this time, of course, defragging itself has already become just about impossible. The amount of work required to shift the data into contiguous blocks is immense, puts additional stress on the drive, takes forever, etc etc. The extent of fragmentation of free space at this stage can be simply staggering, and any large files you subsequently write are likely to be divided into many , many tens of thousands of fragments scattered across the drive. Not only this, but things like the "extents files", which record where all the bits are located, will begin to grow astronomically as a result, putting even more pressure on your already stressed drive, and increasing the risk of major failures.

     

    Ultimately this adds up to a situation where you can identify maybe three "phases" of mac life when it comes to the need for defragmentation.

     

    In the "first phase" (with your drive less than half full), it doesn't matter much at all - probably not enough to even make it worth doing.

     

    In the "second phase" (between , say 50% free space and 20% free space remaining) it becomes progressively more useful, but , depending on the use you put your computer to you won't see much difference at the higher levels of free space unless you are serious video buff who needs to keep their drives operating as efficiently and fast as possible - chances are they will be using fast external drives over FW800 or eSata to compliment their internal HD anyway.

     

    At the lower end though (when boot drives get down around the 20% mark on , say, a 250 or 500 Gig drive) I certainly begin to see an impact on performance and stability when working with large image files, mapping software, and the like, especially those which rely on the use of their own "scratch" files, and especially in situations where I am using multiple applications simultaneously, if I haven't defragmented the drive for a while. For me, defragmenting (I use iDefrag too - it is the only third party app I trust for this after seeing people with problems using TechToolPro and Drive Genius for such things) gives a substantial performance boost in this sort of situation and improves operational stability. I usually try to get in first these days and defrag more regularly (about once a month) when the drive is down to 30% free space or lower.

     

    Between 20% and 10% free space is a bit of a "doubtful region". Most people will still be able to defrag successfully in this sort of area, though the time taken and the risks associated increase as the free space declines. My own advice to people in this sort of area is that they start choosing their new , bigger HD, because they obviously are going to need one very soon, and try to "clear the decks" so that they maintain that 20% free buffer until they do. Defragging regularly (perhaps even once a fortnight) will actually benefit them substantially during this "phase", but maybe doing so will lull them into a false sense of security and keep them from seriously recognising that they need to be moving to a bigger HD!

     

    Once they are down to that last ten per cent of free space, though, they are treading on glass. Free space fragmentation at least will already be a serious issue on their computers but if they try to defrag with a utility without first making substantially more space available then they may find it runs into problems or is so slow that they give up half way through and do the damage themselves, especially if they are using one of the less "forgiving" utilities!

     

    In this case I think the best way to proceed is to clone the internal drive to a larger external with SuperDuper, replace the internal drive with a larger one and then clone back to it. No-one down to the last ten percent of their drive really has enough room to move. Defragging it will certainly speed it up, and may even save them from major problems briefly, but we all know that before too long they are going to be in the same situation again. Better to deal with the matter properly and replace the drive with something more akin to their real needs once this point is reached. Heck, big HDs are as cheap as chips these days! It is mad to struggle on with sluggish performance, instability, and the possible risk of losing the lot, in such a situation.

  • 7. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    roam Level 6 Level 6 (13,545 points)

    To your remaining question,

    Is ... it better to reformat the drive then copy the files back on from the backup drive? .

     

    I have already answered this.

    If you want to rearrange the remaining files (on A) to a contiguous space, you do not have to reformat, instead, you clone fragmented A to B. That makes the B copy contiguous. OK. Then you clone  the B back to A which is then also contiguous, The result is a defragmented A.

     

    No reformatting is required to effect this, nor is reformatting a maintenance procedure. Maintenance of the directory structure can be performed by Repair Disk in Disk utility.

  • 8. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    iMac-newbie Level 1 Level 1 (0 points)

    Ok, got it.  Will do.  Thank you.

    Repair Disk happens after the cloning and copying processes are completed, yes?

  • 9. Re: How to move huge HD video files between external hard drives and defrag ext drive?
    roam Level 6 Level 6 (13,545 points)

    You don't even have to repair the disk afterwards, I mentioned it to indicate what a maintenance practice is, in order to emphasise that reformatting is not a maintenance practice.

     

    Note: if you want to run Repair Disk, it can only perform the operation on a disk that is not itself, to say, it cannot perform the operation on itself.

    So A can run repair on B but not on itself.

    To run repair on A, it must be from an OS installation elsewhere, such as a fully functioning cloned OS on another drive, or from the booted installation DVD.

     

    However running repair disk is not a frequent requirement, only once a month if you wish.

     

    Here is some more information about it.

    Using Disk Utility to verify or repair disks