Tuesday Tip: Troubleshooting a defective database

Today’s operating and file systems are robust, but, still, they do fail. So, it can happen that also a DEVONthink database becomes defective for whatever reason. So, here are some tips on how to keep your database healthy:

  1. Run Tools > Verify & Repair regularly.
  2. Use Tools > Backup & Optimize from time to time to create a copy of the index files and to clean up internal structures.
  3. Make sure to set your backup options in Preferences > Backup to your needs; this function creates copies of the index files inside the database package in case that the index gets corrupted.
  4. Backup your database to a safe place; use the Scripts > Export scripts to export all your documents into the file system and store it to an external hard disk, DVD-R, or your iDisk.

But what can you do when Verify & Repair reports an error? Follow these steps:

  1. Don’t Panic. Except for text and markup files all documents are stored as regular files inside the database package/folder. So even when DEVONthink completely fails there is a good chance that you can retrieve most of your data.
  2. Close DEVONthink and create a copy of your database package in the Finder just in case something goes wrong with the next steps; if you’re running DEVONthink Personal, make a copy of ~/Library/Application Support/DEVONthink (where ~ is your home directory).
  3. Run Tools > Verify & Repair again.
  4. If it fails again, try to go back to one of the internal backups using Tools > Restore Backup… This is non-destructive and simply swaps the backup index with the current one. Run Tools > Verify & Repair on the backup index and if this also fails, try all the other backups.
  5. Should all internal backups be corrupted, too, rebuild the index from scratch: Tools > Rebuild Database…
  6. Finally, if this also fails, it’s time to find your last backup DVD, get your external hard disk, or open Time Machine and replace your defective database with your last full backup.

If even Verify & Repair crashes (and so also Rebuild Database) and your database is extremely large (more than 50 million words) you may succeed by exporting half of your documents to the file system by using ‘File > Export > as Files and Folders’ and deleting them for the database before trying to repair it again. Mac OS X’s memory management is known to crash applications in tight memory conditions when a lot of memory is requested and moved between RAM and hard disk.

9 Responses to “Tuesday Tip: Troubleshooting a defective database”

  1. avalon4053 says:

    hi, so I have an urgent problem that is not addressed here: yesterday I tried to put a big (music)library into one of my databases, it should have been 540 MB. At the end my start volume was full and DevonThinkPro crashed. I can now open all other databases without problem but this one (my entire life library…) won’t open. That is, the initializing process starts but never finishes. So I can’t apply any of the methods you suggest! What can I do? Apple-I gives me a size of 295 MB, so I’m quite sure my data exists somewhere and just part of that new musiclibrary wasn’t imported correctly.

  2. Eric says:

    avalon4053: Try the following: Open a copy (!) of the database package in the Finder with right-click, then Show Package Contents. Trash the ten *.database files and replace them with the same ten files from one of the Backup.XXX folders. Then, try to open the database again with DEVONthink Pro. Does this help?

  3. avalon4053 says:

    No, unfortunately, or not yet. There’s no Backup folder in there yet, unfortunately, the database must be too new. Instead, there’s a file called DEVONthink.lock – I tried to just trash that one and open, but it doesn’t work, still stalls, and when I check Package Contents afterwards, the DEVONthink.lock file is there again… Big sigh. Any idea how to continue?

  4. Bill DeVille says:

    Running out of hard drive space on your boot drive is a no-no. Apple recommends keeping at a minimum 10% – 15% of your nominal HD capacity free, to provide room for system and application swap files, temporary files and so on. When you run out of free hard drive s[ace, there’s not room to write files and in the worst case the system can start overwriting (erasing or corrupting) existing data.

    The lock in your database is an attempt to keep itself from being damaged. And if you had run out of hard drive room, there’s wouldn’t be enough space to make a copy of the database. CAUTION: don’t make a Finder copy of an open database, as the copy is likely to be incomplete, with errors.

    The odds are that your music files are located in the Files folder, inside the database package file. If you have enough drive space, they can be copied from the Files folder to the Finder, perhaps to an external drive.

  5. Rick says:

    Hello Bill and Eric, I have several support emails in on this issue and you have responded…though the issue remains. Essentially, what has happened is two databases have become corrupt…after being stable for months….something has changed! When this started happening, I would boot up DTPO as usually (my first app in the mornings) and I would see a message saying that the inbox datatbase is open – continue or cancel? I select continue..then the same message for another database…I’d select continue (I have no idea why these would be open…I exited DTPO properly the night before). Well, one of these databases would cause DTPO to crash and send an error report to Apple. A process of elimination helped me determine which was corrupt and on booting up DTPO select to not re-open that one. This allowed DTPO to open…then I would go to the menus to try and repair the data base…but could not since it was not opened…trying to open it would crsah DTPO. So, I would go to Time Machine and restore an older copy of the database and re-open it. This would work. But the same thing has happened to other databases now…and I have had to restore them from Time Machine.

    Let me describe my setup. For months I have had DTPO running on my iMac and on my MacBook Pro (both are fairly new with min 4 GB RAM and 450 GB free on hard disks). I would never use DTPO on both machines simultaneously. When in my office, I would use DTPO on the iMac…and use ChronoSync to sync the database files to the MBP. When I was at the university using DTPO on the MBP, it would work fine and when I get to my office ChronoSync would copy the databases frm the MBP to the iMac. This worked for months without a hiccup. Just recently, the trouble described above started happening (after the most recent update actually). So, I tried moving my databases to DropBox to avoid the use of bidirectional syncing between the iMac and the MBP…but it is still happening. When I start DTPO I get messages saying that all databases are open and sometimes selecting continue opens them all without problems…but eventually one will become corrupt and DTPO will crash. Not sure why.

    If anyone has experienced this problem and solved it, please send some advice.

    Thank you in advance. Rick

  6. Daniel says:

    Hoi,

    one question, in the posting you write: “Except for text and markup files all documents are stored as regular files inside the database package/folder. ” — is this still the case with the current version of DEVONthink?

    I have a database with ideas as .txt files, and if that one would really crash, “most of your data” would not be good enough. I tried out creating a new database, created an .txt and .rtf file, then closed it and opened the contents of the database with “show package contents” and found both the .txt and the .rtf there — so did this change?

    Thanks and best regards

    Daniel

  7. eboehnisch says:

    @ Daniel: This info is a few years old. DEVONthink 2 always stores all documents as separate files inside of its database package.

  8. Vinny says:

    I am considering to buy Devonthink, I downloaded the Devonthink trial version and unable to sync the 2 data base, using cincture option.

    Can you please confirm it is actually possible.

    Database_one – iMAC
    Database_two – MAC Book
    Syncstore – NAS storage

    Created Syncstore using new option, then synced it with Database_one and hopefully all went well. Now the Databse_two has few new files added and it will not transfer or retrieve files form Syncstore.

    Please help?

  9. jneumann says:

    Please start a Support Ticket.

Leave a Reply

Note: Blog comments are for commenting on a post. They are unsuitable for proper customer support, honestly. Please consider writing to our support team instead.