When the web breaks

AWS errorsIf you weren’t aware of it already, there was a problem on the internet yesterday for a couple of hours. A little problem in terms of describing it: Amazon Web Services (AWS) was “having problems”. However, it was really a big problem for any app or service relying on the S3 services. One of those services was us. Our downloads were unavailable for a period of time. Slack, Trello, Time Inc., and others were affected. Some even experienced some home automation problems because they relied on the IFFFT website.

This encouraged discussion about the dangers of relying on the cloud so much. Sure the promises sound all well and good, but what if the problem had not been solved in a matter of hours? How would you have gotten to your data? What if your DEVONthink databases only lived on the servers that went down and Amazon couldn’t bring them back? And what if that data was critical to your business, your academic pursuits, or even information vital in saving someone’s life?

This is part of why we don’t support storing everything solely in the cloud and fully advocate and support local resources. DEVONthink databases are stored locally*. If you are indexing files, they should be locally available, not existing only in the cloud. And we strongly suggest a primary backup strategy with local media.

But: “Hard drives are old-fashioned. It’s the 21st century! Everything is in the cloud!” First, no, not everything is in the cloud. And secondly, always keep this question in mind: “What would happen if someone pulled the plug on the cloud right now?” Yesterday proved that it’s a question worth asking.

* Here is our oft-repeated warning: You should never put your databases in any cloud-synced folder, including the Documents or Desktop folders if you are using the disk management feature in macOS 10.12 Sierra. ​Never. If you have, you should ​relocate them or you could irreparably damage them. If you use Sierra’s iCloud features the safest location is a folder in your home directory, like ​~/Databases. If you have them in an unsafe location, quit DEVONthink and move your databases to a new folder in your home directory.

9 Responses to “When the web breaks”

  1. David Parker says:

    Thanks, Jim. Sound advice that we should all take note of.

    David

  2. Stefanie says:

    Thanks Jim for advice.

    For private people it makes therefore sense to have your own cloud running. That is one reason I am very passionate about the own cloud because people are just relying on their own setup and responsibility.

    Regards,

    Steffi

  3. Peter Terlingen says:

    The Cloud has only value when its always accessible.
    There are many regions in the world where the coverage of networks is bad.
    Travelling in the Netherlads, Belgium, France, Spain and Italy I prefer to rely on my harddisk.

  4. Jim Neumann says:

    @David: Thanks. It’s not talked about nearly enough nowadays.

    @Stefanie: That can certainly be a good (and private) alternative!

    @Peter: Agreed! There is a phenomenon I call “the myth of network persistence”, the idea that the cloud and network is ever present no matter where you are. I can travel less than 15 minutes from a city of about 75,000 people (not huge, but not tiny) and be off-the-grid.

  5. Greg says:

    I have my databases in my Documents folder and they are backed-up to the Cloud using Crashplan. Is this what you mean by having the databases in a Cloud-synced folder because I interpret it differently.

  6. Philip says:

    Dear Jim,

    I have my database(s) stored locally on my laptop. As a “backup”, these are synched via WebDAV to my NAS device, which will be backed up – in the near future – to a second NAS. Of course, TimeMachine backups are also regularly performed. If this may appear quite paranoid (or dumb) in your eyes – which of these would you stop doing first? (The reason for the WebDAV “copy” is that I can also add items using DevonThinkToGo…

    Regards

    Philip

  7. Robin says:

    Thanks, Jim, for this brief treatment of a vital topic. Could you perhaps point us all to the relevant sections of DT(PO)’s documentation or even do a mini tutorial series on the various options – there are several best solutions, depending on individual requirements (e.g. do I use DTTG or not)?
    Best Regards
    Robin

  8. Jim Neumann says:

    @Greg: The Documents folder is okay as long as you’re no using the disk management feature of Sierra.

    @Philip: Doesn’t sound paranoid or dumb at all. It sounds like you have a good strategy to me.

    @Robin: Thanks for the comment. A tutorial on what specifically?

  9. Robin says:

    @Jim: Thanks for responding. I guess I’m thinking less of a tutorial and more of an overview discussion of the merits of different solutions for cloud or non-cloud storage and/or syncing of DT – taking the issue of your post into account. That would include the do’s and don’t’s but also point to the relevant sections of the manual/help files to provide the how-to’s as well (hence “tutorial”. It might cover different use-scenarios, such as with or without DT To Go, need to access from multiple (macOS) devices or only from one, and so on, and treat the related but different issues of syncing and backing up. Does that make sense?

Leave a Reply

Note: Blog comments are for commenting on a post. They are unsuitable for proper customer support, honestly. Please consider writing to our support team instead.