Login | Register
My pages Projects Community openCollabNet

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Catacomb] external storage of document data

All, we've implemented external storage of file contents for Catacomb and I'd like to discuss merging it back into the main Catacomb release. It's fairly straightforward, however there is one (philosophical?) decision to make. I'd like to hear your opinions and see if this activity is already underway by someone else.

Should the names/organization in the filesystem mirror the structure in Catacomb? Another option is files can be stored in a hashed directory structure based on the document id.

Advantages of mirroring the structure in the fs:
* easy to understand
* can be interacted with outside of DAV

* performance of containers with many (thousands) resources and for
  very deep container hierarchies
* hard to divide between filesystems (a useful operation as the DAV
  repository grows)
* requires a filesystem operation for MOVE operations
* how to store multiple versions of a document?
* changes outside of the DAV interface should be mirrored back into DAV
* limited to the character set restrictions of the fs (or need to store
  the filesystem name of the file)

FYI, I used to work on a web-based document management system and we moved from mirroring the structure to using a hash structure to resolve problems that occurred when people did many move operations. Conversely, our Catacomb server we are working with here at NASA mirrors the structure into the filesystem (and I may work on a system to mirror back into Catacomb when changes are performed on the fs.)