Login | Register
My pages Projects Community openCollabNet

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Catacomb] RE: storing large files in blob...

thank you all for your responses.
there where more after these first few, but
they got into a design debate hinted at by dana below.

at this point, i am using the catacomb application and 
am not involved with the group doing development/enhancements.

i have posted the original message to the catacomb mailing list
but haven't gotten any response yet.  i will post this response
there. maybe one of the principal developers there could address
how catacomb stores the data.????

so,  if i follow all this, the simple answer to my question:
can i store files larger than 16M in a long blob?
is yes and no.

yes, if the application "chains" the data together in the row=inode
method described below.

no, if the application "jams all the data into 1 row using hugeblob"
(i assume that is _long blob_)

is this the long and short of it?


lloyd knight

-----Original Message-----
From: colbey@dreamwerx.net [mailto:colbey@dreamwerx.net]
Sent: Friday, May 23, 2003 12:32 PM
To: Dana Diederich
Cc: 'walt@nea-fast.com'; Knight, Lloyd; 'mysql@lists.mysql.com'
Subject: RE: storing large files in blob...

I love the use of storing files/data/etc in mysql and have never had any
problems...  The main thing I do not like about how most
systems/code/concepts approach this; they they try to jam all the data
into 1 row using hugeblob..

I disagree with this approach.. Instead I use the row=inode idea storing
64K chunks in the database.. So an individual file say 400MB in size would
require approximately 6250 rows to store it.

This has several advantages.. realtime streamin in and out of database
with larger files with low memory overhead.. your not running huge query,
but many more smaller ones..

There is an article on it/concept at:

This method completely bypasses all the max query/packet size

On Fri, 23 May 2003, Dana Diederich wrote:

> We have, hopefully, agreed to disagree about how appropriate it is to store
> large blobs in the database.  Since this action is a supported feature of
> the database, we should try to help people use the supported feature.
> I am assuming that storing >16MB blobs in MySQL is considered a supported
> feature.  Am I not correct in that?
> Cheers,
> -Dana
> -----Original Message-----
> From: walt [mailto:kernel@nea-fast.com <mailto:kernel@nea-fast.com> ]
> Sent: Friday, May 23, 2003 11:26 AM
> To: Knight, Lloyd
> Cc: 'mysql@lists.mysql.com'
> Subject: Re: storing large files in blob...
> "Knight, Lloyd" wrote:
> >
> > i am using an application (catacomb) which stores file contents in a large
> > blob.
> > i can only successfully store a file of ~16M, anything bigger, corrupts
> the
> > database
> > beyond repair.
> > i am running mysql4.0.12 on solaris2.8
> >
> > i have the max_allowed packet=100M on mysqld startup.
> >
> > i have searched archives and have found no definitive answer.
> >
> > can this be done? if so, what configuration for mysql is needed?
> >
> > thanks
> > Lloyd J. Knight
> > Senior Software Engineer
> > BIA Engineering Systems
> > Tel: 952.403.8473
> > Cell: 612.418.2628
> >
> > lloyd_knight@adc.com
> > Learn about ADC - The Broadband Company at www.adc.com <www.adc.com>
> >
> >   ------------------------------------------------------------------------
> Llyod,
> Can you change the app to store the files on the file system? There have
> been several discussions on the list about the disadvantages of storing
> files (blobs) inside the database.
> walt
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> <http://lists.mysql.com/mysql>
> To unsubscribe:
> http://lists.mysql.com/mysql?unsub=Dana.Diederich@wal-mart.com
> <http://lists.mysql.com/mysql?unsub=Dana.Diederich@wal-mart.com>
> **********************************************************************
> This email and any files transmitted with it are confidential
> and intended solely for the individual or entity to
> whom they are addressed.  If you have received this email
> in error destroy it immediately.
> **********************************************************************
>              Wal-Mart Stores, Inc. Confidential
> **********************************************************************