Login | Register
My pages Projects Community openCollabNet

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Catacomb] external storage of document data

Bryce Harrington wrote:

Hi Chris,

I'm developing a web-based document management system at OSDL, and we're
going with a directory structure based on the document id, like you
describe.  The lack of that capability in Catacomb has been a stumbling
block for us in being able to incorporate DAV support into it, so I'd
*love* to see it supported by Catacomb.

Specifically, I'm using a repository structure like this:

Actually, what about having a function in dbms.c that looks like:

int generate_filepath(char **path, dav_repos_resource *r) // note, should run apr_file_mkdirs

Which you could then come up with your own conventions. I'd go even further with:

/[doc_id % 100]/[doc_id / 100]/[doc_id]/[doc_rev]...

This "stripes" the resources across a set of 100 top-level and 10,000 second-level directories (100 per top-level.) Want to move ~45% of the total space to another drive? Move the top level directories 00 through 44 (and install symbolic links or alter the generate_filepath?)

Otherwise, if you have a million resources in your server, you'd have a million directories all under one parent.

'course, with a generic fn to generate the path, you could take this even further to handle millions/billions of documents without too much difficulty. (However, your inode count will be very high in this scheme...You might want to ensure that your inode/block ratio is higher than normal.)