Dropbox - another way to shoot your foot

Whew, the data loss in the last post actually wasn't that bad because the folder was version controlled anyway (and stored on another server). I thought only using dropbox for working copies would keep me safe.

So much for that idea! In a sleepy state just now I managed to get Dropbox to delete scattered files outside of my Dropbox folder!

This comes back to the symlink semantics, but it can bite you in strange ways. Here's how you can really screw yourself. First, let's say you've got a working copy that stores all your notes and other reference text files that you search often:

   mv foobar ~/dropbox/foobar

Let's further suppose that you are on a Mac or Linux client and foobar contains symbolic links that point to other scattered text files elsewhere in your personal archives that you may want to have at hand (for grepping).

Next, you want to make a major change to foobar, so you back it up, just to be safe:

   cp -a ~/dropbox/foobar ~/dropbox/foobar_bak

A while later you come across this backup (on a different dropbox-linked machine) and delete it:

rm -rf foobar_bak

BAM! That's it, you just screwed yourself.

How? When Dropbox went back to your original Mac/Linux machine to delete foobar_bak, it decided to FOLLOW THE SYMLINKS. That's right, it deleted the original files. Even though the original foobar directory is still there, its links are now broken of course.

The whole point of these links was organizational. They pointed all over the place. Even if you have backups you now have to track them down and restore those scattered files. (Which is what I just spent my time doing.) I guess I'm a synchronization masochist because I seem to ask for this kind of punishment.

Bottom line. If you're a power user, avoid Dropbox or keep your usage very light, e.g. stick an office document in there every once in a while. Or be ready to suffer.

A New Dropbox Nightmare

I was cautiously optimistic and hopeful -- "maybe Dropbox is the synchronization solution I've been waiting for" -- but that sentiment is quickly being replaced with jaded.

First, it is good to recognize that backup and synchronization (mirroring) are orthogonal. Mirroring only insures that mistakes get propagated. Well, I just experienced my second Dropbox spontaneous deletion event, which, due to the miracle of synchronization, was applied to all of my computers and to the cloud.

What can be worse than a silent deletion event for a service like Dropbox? Nothing that I can think of. I can't babysit the 96,000 files in my Dropbox to catch something like this.

Now the details. I came across a directory that had 95% of the files deleted (leaving a random few remaining). Scrolling back through the event history I found that this happened after linking a new client, which happened to be a Windows virtual machine. Below is a snippet of the event audit trail. The entire series of events happening at 10:09 was spurious (including deleting 411 files and 206 folders).

You edited the file .saves-25811-rnmacbook13~.      11:27 AM
You renamed the file intel_aes.c to test038.h.ERR. 10:09 AM
You renamed the file setup to test047.h.ERR.        10:09 AM
You added 686c9fad58174636354 and 78 more files.    10:09 AM
You edited simple_cycle.out and 49 more files.      10:09 AM
You deleted 6c88b2f3aaaa585db12 and 411 more files. 10:09 AM
You added eb and 6 more folders.                    10:09 AM
You deleted intel_aes_lib and 206 more folders.     10:09 AM
You moved 04 and 24 more folders.                   10:09 AM
You added the folder fabl.                          12:42 AM
The computer rn-waspvm was linked to your account   12:24 AM
You edited the file ghc_setup.txt.                  12:24 AM

A couple additional oddities show up in the above:

• it waited over 9 hours after the new client was linked to go crazy
• Files were "'renamed", intel_aes.c to test038.h.ERR Huh? Is that a remove being aliased with an add to create a fake rename? Possibly a secondary bug.

I went back and checked out the state of that Windows client just now. A clue! The only files it successfully downloaded were the ones not deleted in the above nonsense. So Dropbox confused a partially downloaded directory with a new version of the directory (deleting most of the contents). Unison has never done anything like that to me in ten years of heavy use!

It may be corroboration that the Windows client currently thinks it "hasn't connected yet" [Screenshot below]. Oh really?

Well, perhaps this is "fool me twice, shame on me". I should have known better -- my wife tried Dropbox about a year ago, dumped her ~50 gigabyte personal archive in all at once, then moved a folder, and ended up with her data in a never-fully-sychronized, confused state. It still hasn't been completely fixed; there's a backup, but also new stuff mixed into the messed up Dropbox version. I chalked that one up to bad behavior while under load and before the first sync is successful (plus the dangers of syncing being too implicit and automatic). Still, I wanted to give Dropbox another chance; trying it myself this time, thinking that:

• It's surely improving and ...
• I would be more cautious, spoon-feed it, and pay more attention to its syncing state.

But clearly that didn't save me.

Dropbox semantics - oh that there were such a thing

After some recent Dropbox problems I've been having (not the first unfortunately) I added an entry on the Dropbox wiki explaining what happens if you are so impertinent as to add directories containing symbolic links to your Dropbox folder:

http://wiki.dropbox.com/Symlink%20Semantics%20and%20Workarounds

[UPDATE - dropbox took down their wiki, but I reposted the page here.]

One day maybe we'll have something with the correctness and robustness of Unison and the convenience of Dropbox. People will dance in the streets. I think Dropbox can be that solution if they work at it.

UPDATE - these issues are explained at greater length in this post:

http://aurelio.net/articles/dropbox-symlinks.html