Automating backup of Flash drives Creative Commons License

posted in the wee hours by Constantinos. Filed under Code, OS X

This post was originally published in 2007
The tips and techniques explained may be outdated.

I have a lot of my critical work on a few flash drives which I carry around with me constantly, mainly because I work on a number of machines and would like to have a central repository of my work. After losing one of the drives recently though (bound to happen sooner or later), I looked further into automating the backup process (I had lost about 2 days of work). That’s when I stumbled accross this hint on macosxhints.com, but the script didn’t exactly fulfil what I had imagined my automated process to do.

Ideally, the automated backup process would have these features:

  1. Not require installation of any third party app
  2. Have the ability to somewhat customize the backup script for each thumb drive
  3. Have the backup script silently run whenever the thumb drive is mounted
  4. Require little to no set up for each new thumb drive that I want to backup
  5. Support spaces in Volume names
  6. Not try to backup the boot drive under any circumstances (don’t ask…)

Updated 2007-03-10: I updated the backup script to include automated handling of archives, and keeps an arbitrary number (specified in the preferences of the script) of incremental archives using rsync's --link-dest parameter, as recommended by John in the comments. This creates what functionally amounts to FULL backups, but with a fraction of the space needed for an actual tar backup. Also it can be configured to keep at most one backup per day, or with minor editing at most one backup for any period of time.

Updated 2007-03-18: I've created a second script which can be used as a drag-n-drop solution. Simply save the script as an application bundle, save it somewhere handy, and drag it to your dock. Now you can drag your thumb drive on the script which will perform the backup, and then eject your drive!

Updated 2008-03-10: Folder action modified to work with Leopard. Well, more like hacked at this point, but it seems to work.


Now to satisfy the first parameter, using Do Something When was out of the question, even though it seems like a good piece of software. That’s when I turned to Fodler Actions, and found this hint again on macosxhints.com. I then proceeded to read the Apple-published Folder Action for automating backups (which, again, didn’t do quite what I wanted), so I came up with this:

A folder action that scans the root folder of the mounted volume, and looks for (one of) a specified file name.

property backup_script : "rsync.app" (* add/change this string to match the name of your script *)
 
on adding folder items to this_folder after receiving these_volumes
    tell application "Finder"
        try
            set the volume_list to every file of this_folder
 
            (* go through all entries in /Volumes/ *)
            repeat with i from 1 to number of items in volume_list
                set this_item to the original item of item i of volume_list
                if the kind of this_item is "Volume" then
                    set this_disk to (this_item as alias)
 
                    (* is this item the newly mounted disk? *)
                    if this_disk is in these_volumes then
 
                        (* iterate through all files in the root of disk *)
                        set searchCmd to "ls -d " & quoted form of POSIX path of this_disk & backup_script
                        (* check to see if a backup script is available *)
                        set searchResult to ""
                        try
                            set searchResult to do shell script searchCmd
                        end try
                        (* run the backup script *)
                        if (searchResult starts with "/Volumes") then
                            set backupFile to POSIX file searchResult as alias
                            open backupFile
                        end if
 
                    end if
                end if
            end repeat
        on error error_message number error_number
            if the error_number is not -128 then
                display dialog error_message buttons {"OK"} default button 1
            end if
        end try
    end tell
end adding folder items to

Download this script

If you already know how to apply folder actions, just save the script above and apply it as a folder action to the /Volumes/ folder. If you don’t know (or don’t remember) how to do that, re-iterating the instructions from this Apple page, follow these steps to do it:

  1. Copy the above code in Script Editor
  2. Save it as a script in /Library/Scripts/Folder Action Scripts/, and give it a descriptive name
  3. In Finder, click Go -> Go to Folder (or just press cmd-shift-G), and type /Volumes/.
  4. Launch the Folder Actions Setup utility (probably located in /Applications/Apple Script/ folder)
  5. Click the Add Folder Action (round plus) button at the bottom left
  6. Drag the small folder icon from the Finder window title bar into the Choose Folder sheet dialog, and click ok.
  7. From the dialog that comes up, chose the script you just saved.
  8. You’re done.

Alternatively (or additionally), you can save this script as an application bundle, save it somewhere handy, and drag it to your dock. When you drag a thumb drive on this application the backup process will start, and as soon as it's done the drive will eject.

property backup_script : "rsync" (* add/change this to match the name of your script *)
 
on open dropped_item
    set this_vol_alias to (dropped_item as alias)
    try
        tell application "Finder"
            if the kind of this_vol_alias is "Volume" then
                set the file_list to every file of this_vol_alias
                repeat with j from 1 to number of items in file_list
                    (* check to see if a backup script is available *)
                    set this_file to name of item j of file_list
                    if this_file is equal to (backup_script & ".app") then
                        open item j of file_list (* run the backup script *)
                    end if
                end repeat
            else
                display alert "This script can only be executed on volumes"
                return
            end if
        end tell
 
        delay 2 (* give some time for the backup script to actually launch *)
        tell application "Finder"
            try
                set bakapp to creator type of process backup_script
                repeat while bakapp is not equal to ""
                    delay 1 (* the backup is still running, we can check again in a second *)
                    set bakapp to creator type of process backup_script
                end repeat
            on error
                if (ejectable of this_vol_alias is true) then
                    (* display alert POSIX path of dropped_item *)
                    eject this_vol_alias
                end if
            end try
        end tell
 
    on error error_message number error_number
        if the error_number is not -128 then
            display dialog error_message buttons {"OK"} default button 1
        end if
    end try
end open

Download this script

Now, for the modified backup script. The properties at the top can be modified on a per copy basis to customize the function of the script.

property archive_backup : true
property number_of_archives : 3 (* 0 for unlimited *)
property with_administrator_privileges : false
property one_archive_per_day : true
 
property display_notification : false
property backup_target : "Backups/" (* MUST be a folder location with trailing slash! Always relative to home folder *)
 
property rsync_params : "-aEz --delete-excluded"
(* END OF PREFERENCES *)
 
set WhereImRunningFrom to path to me
tell application "Finder"
    (* can't run from the script editor *)
    set AppCreator to creator type of WhereImRunningFrom
    if AppCreator is "ToyS" then
        activate of me
        beep
        display alert "This script cannot run directly from ScriptEditor"
        return
    end if
 
    (* NEVER run from the hard drive! *)
    set bootVolume to name of disk of home (* safety feature! *)
    set NameOfDisk to name of disk of WhereImRunningFrom
    if NameOfDisk is bootVolume then
        beep
        display alert "Should not run this script from the boot volume!"
        return
    end if
 
    (* setup backup dir *)
    set homeDir to path to home folder from user domain
    set backup_folder to POSIX path of homeDir & backup_target
    try
        (* quick and dirty check to see if folder exists - must be a good way to do this in applescript? *)
        do shell script "cd " & backup_folder
    on error
        try
            do shell script "mkdir " & backup_folder
        on error
            display alert "There was an error creating the backup folder " & backup_folder
            return
        end try
    end try
 
    (* find source and target *)
    set backupBase to NameOfDisk & "-Backup"
    set targetDir to backup_folder & quoted form of backupBase
    set sourceDir to quoted form of ("/Volumes/" & NameOfDisk & "/")
 
    (* set up archive filename *)
    if archive_backup then
        set thedate to current date
        set theday to day of thedate
        set themonth to month of thedate
        set theyear to year of thedate
        set thetime to time of thedate
 
        set thedate to "-" & theyear & "-" & shortMonth(themonth) of me & "-" & theday as string
        if not one_archive_per_day then set thedate to thedate & "-" & thetime as string
 
        set backupFolder to quoted form of (backup_folder & backupBase & thedate  & "/")
    end if
end tell
 
(* do the backup *)
(* try *)
if with_administrator_privileges then
    set excluded to " "
else
    set excluded to " --exclude='.Trash*' --exclude='.Spotlight*'  "
end if
 
if archive_backup then
    set posixDir to POSIX file backup_folder
    set fileList to list folder posixDir
    set archiveList to {}
    tell application "Finder"
        (* Search for existing backups *)
        repeat with i from 1 to number of items in fileList
            set thisFile to item i of fileList
            set filePath to path to resource thisFile in bundle posixDir
            if thisFile starts with backupBase then
                set archiveList to archiveList & POSIX path of filePath
            end if
        end repeat
    end tell
 
    (* List existing backups in order of last modified date *)
    set latestArchive to ""
    set linkDest to ""
    set listString to ""
    repeat with x from 1 to number of items in archiveList
        set listString to listString & " " & quoted form of item x of archiveList
    end repeat
    set newString to do shell script "ls -dt " & listString (* sort files by modified date *)
    set theList to (paragraphs of newString)
 
    (* Clear out outdated backups *)
    if (number of items in theList is greater than number_of_archives) and (number_of_archives is greater than 0) then
        repeat with y from (number_of_archives + 1) to (number of items in theList)
            do shell script "rm -Rf " & quoted form of item y of theList
        end repeat
    end if
    if number of items in theList is greater than 0 then
        set latestArchive to quoted form of item 1 of theList
        if latestArchive is equal to backupFolder then
            (* Preserve same target folder *)
            if number of items in theList is greater than 1 then
                set latestArchive to quoted form of item 2 of theList
            else
                set latestArchive to ""
            end if
        end if
    end if
 
    if latestArchive is not equal to "" then
        set linkDest to " --link-dest=" & latestArchive
    end if
    set theScript to "rsync " & rsync_params & linkDest & excluded & sourceDir & " " & backupFolder & "; touch " & backupFolder
else
    set theScript to "rsync " & rsync_params & excluded & sourceDir & " " & targetDir & "; touch " & targetDir
end if
 
if with_administrator_privileges then
    do shell script theScript with administrator privileges
else
    do shell script theScript
end if
 
activate of me
beep
if display_notification then display alert NameOfDisk & " drive backed up"
(*
on error
    display alert "There was an error backing up " & NameOfDisk
end try
*)
 
 
to shortMonth(themonth)
    set MonthList to {January, February, March, April, May, June, July, August, September, October, November, December}
    set mm to 1
    repeat until item mm of MonthList = themonth
        copy mm + 1 to mm
    end repeat
    if mm < 10 then copy "0" & mm to mm
    return mm
end shortMonth

Download this script

Copy the above code in Script Editor, and save it as an Application bundle giving it the name "rsync". If you wish to give it a different name, or maintain a number of copies each with a different name, make sure to modify the first line of the Folder Action script appropriately.

Finally, I can just drop a copy of the rsync application bundle in any thumb drive I want to keep a backup of, and apply the Folder Action to any computer I want to maintain backups on. Since the backup script is using rsync for the backup, I can have multiple backups on any number of computers and they will all mirror the contents of the thumb drive whenever the thumb drive is mounted on them. Additionally, a tar-gzipped copy of the thumb drive can optionally be created with a date stamp for archiving. Deletion of old archives must be managed manually.

If you are having permission issues with some of the files on one of your thumb drives, you can enable the with_administrator_privileges property in the backup script. This will cause the script to ask you for a password every time the thumb drive is mounted before performing the backup, but will perform the backup as an administrator eliminating any permission issues.

I feel that this script can still be improved, but for now it works just fine for my needs. If you have any suggestions for improving this process, feel free to add them in the comments.

78 Responses to “Automating backup of Flash drives”

Pages: [2] 1 » Show All

  1. 78. Comment by skororu
    on 2013-11-26 @ 4.30 am · Quote #206702 ·

    Update for OSX Mavericks

    I made a quick hack to the folder action script that attaches to the /Volumes directory so it works for OSX Mavericks. The script that sits on the memory stick itself works fine without any modification.

    http://pastebin.com/d77rwsrP

  2. 77. Comment by Nichole
    on 2011-06-20 @ 7.57 am · Quote #33108 ·

    Where do I go to access my backedup files?

    The 6th property in the actual backup script reads


    property backup_target : "Backups/" (* MUST be a folder location with trailing slash! Always relative to home folder *)

    If you did not change it, then your backups would be in ~/Backups/, where ~/ is your Home directory. So Open up the finder, click on your home dir, and look for a folder called Backups.

    Thank you for your reply… I went to finder, searched for backups and nothing comes up. I followed the instructions as per above, and my flashdrive seems to get to business as soon as its mounted. Also, when I save something the screen briefly flashes (as if it’s backing up – just my imagination?)

    I’m having a heck of a time with this… Nothing seems to be saved anywhere on my computer. I def. did not change the script. Any help would be greatly appreciated. I recently lost everything on a different flash drive and don’t want to go through that again.

  3. 76. Comment by Constantinos
    on 2011-06-13 @ 7.41 pm · Quote #32771 ·

    Where do I go to access my backedup files?

    The 6th property in the actual backup script reads


    property backup_target : "Backups/" (* MUST be a folder location with trailing slash! Always relative to home folder *)

    If you did not change it, then your backups would be in ~/Backups/, where ~/ is your Home directory. So Open up the finder, click on your home dir, and look for a folder called Backups.

  4. 75. Comment by Nichole
    on 2011-06-12 @ 11.58 pm · Quote #32730 ·

    Where do I go to access my backedup files?

  5. 74. Comment by Guapo
    on 2011-01-21 @ 3.23 am · Quote #24562 ·

    Mine’s not working like that. I get the total of all folders when I go up one level.

    Could it be that I need to change some of the preferences/properties of the rsync.app? Referring to my concern above.

  6. 73. Comment by Guapo
    on 2011-01-19 @ 2.57 am · Quote #24446 ·

    Try this: Go in your backups folder, click on one specific backup, and hit cmd-I to see it’s info. This will tell you the total size of that folder, which should be around 2gb (as it’s a whole copy of your memory stick). Now go one level up, to the Backups folder, and do the same. You’ll notice that the ENTIRE Backups folder, which contains all 5 backup copies of your memory stick, is in fact just over 2Gb and NOT 10Gb in size.

    Mine’s not working like that. I get the total of all folders when I go up one level.

  7. 72. Comment by El Guapo
    on 2011-01-18 @ 9.21 pm · Quote #24424 ·

    I am using Leopard, and am wondering if I am doing something wrong. I have a Backups folder that was created and the backup appears to have worked. But instead of using rsync to do incremental copies, it seems to just create an entire folder with the date appended.

    Thus, if I have an almost full 2GB flash drive, every day I plug it this script uses almost 2GB per day.

    That’s how rsync works. It copies everything using hard links. Therefore even though it looks like you have 5 different copies of all of your files, in fact you only have one copy that is pointed to from 5 different memory locations.

    Try this: Go in your backups folder, click on one specific backup, and hit cmd-I to see it’s info. This will tell you the total size of that folder, which should be around 2gb (as it’s a whole copy of your memory stick). Now go one level up, to the Backups folder, and do the same. You’ll notice that the ENTIRE Backups folder, which contains all 5 backup copies of your memory stick, is in fact just over 2Gb and NOT 10Gb in size.

    The ‘incremental’ aspect of the backup is automatically handled by rsync, where if a file hasn’t changed since the last copy, instead of copying it over it just creates a hard link to the copy saved by the previous backup.. :)

  8. 71. Comment by Constantinos
    on 2010-12-30 @ 8.54 am · Quote #23313 ·

    I got stuck a the part where you said to launch the Folder Actions Setup utility. This doesn’t seem to be anywhere on my Mac. I know I haven’t deleted it as this is a new MBP (purchased Oct 2010). Please help

    In Snow Leopard, right click on the /Volumes/ folder. If you don’t see a Folder Actions entry in the contextual menu, look for a Services entry at the end – Folder Actions should be in there.

  9. 70. Comment by Guapo
    on 2010-12-04 @ 4.44 am · Quote #21340 ·

    I second what Talcorath said above. I like the idea of what you’re trying to do, but I don’t know enough about AppleScript to know if all this will work under Snow Leopard.

    I got stuck a the part where you said to launch the Folder Actions Setup utility. This doesn’t seem to be anywhere on my Mac. I know I haven’t deleted it as this is a new MBP (purchased Oct 2010). Please help.

  10. 69. Comment by Talcorath
    on 2010-09-01 @ 11.56 pm · Quote #17627 ·

    Does this script and folder action work under Snow Leopard? I’ve been looking for a free solution and yours looks the most valid, just not sure if it still works.

  11. 68. Comment by Xavier
    on 2009-08-23 @ 11.41 pm · Quote #4608 ·

    The following how to might let do what you want in a simple way. If you know what you want to back up and where, you just plug, wait till it does the job, and unplug the key.
    http://mogrifiers.blogspot.com/2009/08/quick-usb-backup-from-mac.html

  12. 67. Comment by Constantinos
    on 2008-08-20 @ 11.35 am · Quote #2525 ·

    I just can’t seem to figure out how to make it work. I’m so lost that I’m not even sure where the files should end up (after being backed up). It also seems like I need to save a file on the flash drive but again, I’m not sure which file.

    You’re probably right in that I could’ve made the explanation for this a bit simpler.

    You only really need two of the three, above scripts to make this work. The first script on this page (auto_backup.scpt), you need to download as-is and save it as a folder action as per the instructions right under it. The last script on this page (rsync.app) you need to download, extract (as the download is compressed), and simply move the extracted file on to the thumb drive you wish to have backed up.

    Those are the two files you need. To modify the options for the backup, you need to open up Script Editor (found under Applications / Utilities), open the rsync.app file with it, modify the settings found at the beginning of the file, and save it.

    Hope this helps you!

  13. 66. Comment by Andy
    on 2008-08-20 @ 3.04 am · Quote #2524 ·

    This seems like a really cool utility and I’d love to figure out how to make it work. I’m fairly new to Macs and while I’ve tried to follow the instructions carefully I just can’t seem to figure out how to make it work. I’m so lost that I’m not even sure where the files should end up (after being backed up). It also seems like I need to save a file on the flash drive but again, I’m not sure which file.

    I know I’m really a novice on this so feel free to ignore me – I’d understand. But man I’d love to get this working. I’m on Leopard by the way.

    Thanks!

  14. 65. Comment by Jacob
    on 2008-07-17 @ 5.15 pm · Quote #2396 ·

    I’m getting an error message: “Finder got an error: Can’t get original item of doc file “File name” of disk “Diskname”

    Anyone else seen this? Have a solution?

    The backup and eject script seems to work OK — this is only on the auto-backup when mounted script.

  15. 64. Comment by Kevin
    on 2008-06-22 @ 12.41 pm · Quote #2330 ·

    I did a cmd-i on the individual folders, not on the Backups folder. Thanks for clearing that all up!

  16. 63. Comment by Constantinos
    on 2008-06-21 @ 10.24 pm · Quote #2329 ·

    I am using Leopard, and am wondering if I am doing something wrong. I have a Backups folder that was created and the backup appears to have worked. But instead of using rsync to do incremental copies, it seems to just create an entire folder with the date appended.

    Thus, if I have an almost full 2GB flash drive, every day I plug it this script uses almost 2GB per day.

    That’s how rsync works. It copies everything using hard links. Therefore even though it looks like you have 5 different copies of all of your files, in fact you only have one copy that is pointed to from 5 different memory locations.

    Try this: Go in your backups folder, click on one specific backup, and hit cmd-I to see it’s info. This will tell you the total size of that folder, which should be around 2gb (as it’s a whole copy of your memory stick). Now go one level up, to the Backups folder, and do the same. You’ll notice that the ENTIRE Backups folder, which contains all 5 backup copies of your memory stick, is in fact just over 2Gb and NOT 10Gb in size.

    The ‘incremental’ aspect of the backup is automatically handled by rsync, where if a file hasn’t changed since the last copy, instead of copying it over it just creates a hard link to the copy saved by the previous backup.. :)

  17. 62. Comment by Kevin
    on 2008-06-21 @ 10.18 pm · Quote #2328 ·

    First, thanks for such a great utility.

    I am using Leopard, and am wondering if I am doing something wrong. I have a Backups folder that was created and the backup appears to have worked. But instead of using rsync to do incremental copies, it seems to just create an entire folder with the date appended.

    Thus, if I have an almost full 2GB flash drive, every day I plug it this script uses almost 2GB per day.

    Am I wrong in expecting it to just create one folder and incrementally back it up? Does anyone know how I could modify the rsync command to do this?

  18. 61. Comment by Mark
    on 2008-03-23 @ 12.39 pm · Quote #2262 ·

    Is there a way to do this in reverse?

    i.e. So that when I mount my flash drive it executes an automatic backup of a specific folder or specific folders from my hard drive onto the flash drive?

    Here is a link to a tutorial on how to do this: http://maclawstudents.com/blog/techniques/automatic-backups-to-usb-flash-drives/

    It basically uses all Constantinos’ code but makes a backup _to_ your flash drive. It suffers from the same Leopard folder-action bug. But it works perfectly in Tiger.

  19. 60. Comment by noxlady
    on 2008-03-17 @ 6.43 am · Quote #2255 ·

    Hey, it works! Well, my rsync script launches, anyhow:) I’m pretty sure that once rsync launches I’m home free, it was just those darn folder actions that were the problem. Thanks so much for the fix!!!!

  20. 59. Comment by Constantinos
    on 2008-03-10 @ 10.36 am · Quote #2252 ·

    Oh yeah, forgot to mention: The modification means that you can no longer have a list of differently named backup scripts, which will launch depending on what thumbdrive you insert. All the thumbdrives you use need to have their backup script named the same. I’m guessing it’s not a real problem for anyone using this, but just a heads up anyway.

  21. 58. Comment by Constantinos
    on 2008-03-10 @ 10.34 am · Quote #2251 ·

    Has anybody had a chance to work out the Leopard bug yet? I”m a teacher, and I keep my grades on a thumb drive, so losing them would be disastrous. The old auto backup script made me feel much safer — I’m eagerly awaiting a Leopard update :)

    Since there are people who are actively using this script (I stopped because I’m done working with my memory stick so there’s nothing for me to back up), I’ve sorta fixed the problem. The new version of the folder action (which is the only thing that changed) seems to work just fine with Leopard, and should also just work with Tiger. I’m saying ‘sorta’ fixed the problem because it’s not a real Applescript solution, it’s piping terminal commands to search for the file, a technique I used for simulating encrypted thumb drives in OS X and appears to work without issues, but it’s not my favourite solution in the world.

    Let me know how it works out. Again, only the folder action changed, which is what actually launches the backup script. The script itself is unchanged, so if it launches automatically it should just work the same way it did before.

  22. 57. Comment by noxlady
    on 2008-03-10 @ 3.12 am · Quote #2250 ·

    Has anybody had a chance to work out the Leopard bug yet? I”m a teacher, and I keep my grades on a thumb drive, so losing them would be disastrous. The old auto backup script made me feel much safer — I’m eagerly awaiting a Leopard update :)

  23. 56. Comment by Michael
    on 2008-03-09 @ 2.33 am · Quote #2249 ·

    The script has a known problem under Leopard. The problem lies with the line set the file_list to every file of item i of volume_list, which returns an empty list in Leopard (I don’t know why). It could be fixed by switching to a terminal command, but I haven’t had time to do it yet.

    Thanks for the effort thus far, Constantinos.

  24. 55. Comment by lemonteh
    on 2008-03-06 @ 4.03 am · Quote #2245 ·

    Cool! Thanks.

  25. 54. Comment by Constantinos
    on 2008-03-04 @ 2.54 am · Quote #2243 ·

    What happen to the hardlinks then since all the unchanged files are actually hard linked to the previous?

    When you delete a file in unix, the file isn’t actually deleted. What is deleted is the link to that file, which usually marks the space on disk as free.

    In the case of hard links, what you have is several entries in the disk’s index that point to the exact same position on the disk. As far as the system is concerned those are separate files that reside in the same memory location. When you delete one of them, the relevant index is removed, but the rest of the indexes remain. The file isn’t actually ‘erased’ (or the space it occupies isn’t marked as free) until ALL the hard links pointing to it are removed from the index.

  26. 53. Comment by lemonteh
    on 2008-03-04 @ 2.45 am · Quote #2242 ·

    I was reading through this applescript. If archive backup is turned on, the oldest archive will be deleted when the number of archive reaches the maximum. What happen to the hardlinks then since all the unchanged files are actually hard linked to the previous?

  27. 52. Comment by Constantinos
    on 2008-02-26 @ 1.51 pm · Quote #2237 ·

    The script has a known problem under Leopard. The problem lies with the line set the file_list to every file of item i of volume_list, which returns an empty list in Leopard (I don’t know why). It could be fixed by switching to a terminal command, but I haven’t had time to do it yet.

  28. 51. Comment by Matt
    on 2008-02-26 @ 11.27 am · Quote #2236 ·

    I’ve noticed that sometimes the script on my USB stick does not run. I’m not sure why this would be as at other times it runs without any problems – It all seems completely random. Is this a known problem with this script or applescripts in general?

Pages: [2] 1 » Show All

Leave a Reply

Click to quote selected text