Re: [Catacomb] dbms.c proposed changes

[Chris Knight <cknight@mail.arc.nasa.gov>]

Jim Whitehead wrote:

> I'm in favor of making the database code more secure -- you certainly
> identify a very significant security hole!
>  
One that I've overlooked on other projects (luckily, a test suite found 
this hole, not a hacker!)
> The database independent layer sounds good to me, though I will note that
> discussions I've had with people who have developed document management
> systems indicate that for performance reasons they quickly get to the point
> where the optimize heavily for specific databases, including making use of
> functions executing in the DB.  Since we're  focusing on just getting things
> working, we don't have to worry so much about performance. But, the limited
> performance profiling that was done last Winter indicates that Catacomb's
> performance is really slow, orders of magnitude off what commercial doc.
> mgmt. systems achieve. For Catacomb to be taken seriously, we'll eventually
> need to address performance.
>  
Hmm, that's curious.  Do you have information you can point me to 
regarding the profile tests and the results?  (If not, don't worry 'bout 
it...I'd definitely aim for features first, then performance tuning.)
> So, what I want is a database-independent layer that leaves the door open
> for database-specific optimization. I'm not sure what this would look like.
>  
True enough. Given that there are very few cases where even the SQL 
between two databases is 100% compatible, I'd say that dbms.c would have 
to be customized for each RDBMS anyways. (Although as we abstract away 
the semantics of initializing the db and preparing standard queries, we 
do reduce the amount of code that would have to be re-written in dbms.c.)
As for performance, this will add one more layer of function calls, a 
minimal impact. I am concerned about the performance of memory 
allocation (although the pools should help quite a bit) and string 
escaping, but it's a necessary evil.