Login | Register
My pages Projects Community openCollabNet

HOWTO Use Catacomb with Apache

DLR-SISTEC http://www.dlr.de/sc

0.03, 2008-08-05


This is a brief explanation about Catacomb. This explains what is Catacomb, howto install it and how to use it. The enables the SEARCH method which is specified in the DASL protocol. The Catacomb is a module for Apache and repository interface of mod_dav.

1. Introduction

There are many web servers. Suppose you have one web server which has millions of documents. How would you find find particular documents whose author is Jim and which contain a key word such as java. A Catacomb is a perfect solution for that.

The Web-DAV protocol is an extension of HTTP and it enables the user to write to the HTTP server, give properties for resources and other functions. DASL is extension of Web-DAV and it enables the user to search resources in the Web server. Catacomb is a DASL implementation for Apache and mod_dav.

This document explains how to install the Catacomb module and use it.

1.1 Copyright

/* Copyright 2008 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. (DLR)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*     http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

2. What is Catacomb?

Briefly, it is :

  • DASL implementation for mod_dav and Apache.
  • DBMS Back-ended storage system.
DASL specifies the SEARCH method for searching and locating resources. In other words, it enables the user to search resources in the web server using the DASL protocol. Catacomb is the implementation of DASL which allows web server to support a search function.

To implement the search function, Catacomb uses mod_dbd from the apache project to acces a relational database. That means if you are saving some resources using PUT or MKCOL which are Web-DAV methods, all of the resources and their information are saved in not a file system, but in DBMS. In fact, not really all resources have to be saved in the database, you could for example set a size-limit on which the resources are save in the local server filesystem but their metadata are saved in the database.

3. Installation

3.1 What do I need?

You need :

  • Linux or Unix system which can be installed Apache.
  • Apache 2.2.6 or higher version.
  • mod_dbd support of apache and a supported database.
  • Catacomb latest version.

3.2 Quick Tour

We assume that, if you use for example MySQL, it is installed in the /usr directory (default in ubuntu and debian linux).

<shell> tar xvfz  httpd-2.2.6.tar.gz
<shell> cd http-2.2.6
<shell> ./configure --enable-so --enable-dav --enable-dav-fs --enable-dbd --with-mysql=/usr 
<shell> make; make install
<shell> cd ..
<shell> tar xvfz catacomb-x.x.x.tar.gz 
<shell> cd catacomb-x.x.x
<shell> ./configure --with-apache=/usr/local/apache2
<shell> make; make install
<shell> add this in /usr/local/apache2/conf/httpd.conf
____________________________________________________________________
|# load the module                                                  |
|LoadModule dav_repos_module   modules/libmod_dav_repos.so          |
|                                                                   |
|# This is server configure now                                     |
|DavDBMSTmpDir /tmp/                                                |
|DavDBMSFileDir /usr/local/apache2/htdocs/repos/                    |
|DavDMBSMaxFileSize 102400                                          |
|                                                                   |
|<Location /repos>                                                  |
|DAV repos                                                          |
|<//Location>                                                       |
|                                                                   |
|DBDriver mysql                                                     |
|DBDParams "host=localhost,user=user,pass=pass,dbname=catacomb_dbd" |
--------------------------------------------------------------------

<shell> mysqladmin create repos
<shell> mysql repos < table.sql
<shell> mysql repos < data.sql
<shell> /usr/local/apache2/bin/apachectl start
Enjoy DASLed Apache server!

3.3 Installation

You need MySQL, and you can download it from the MySQL Web Site. I assume that MySQL is already installed on your machine. There is no special MySQL option for mod_dav_dasl, so you don't need to reinstall MySQL for Catacomb, if MySQL is already installed. If you have some difficulties to install MySQL, please visit the MySQL homepage. The default MySQL installation directory is /usr/local.

You need Apache 2.0.36 or later version for Catacomb. You can download the latest Apache server from the Apache Web Site. When you install Apache, we need two options, --enable-dav and --enable-so. Catacomb is a mod_dav interface, so we need to enable mod_dav. The Catacomb object is a shared style object, so your Apache must support shared objects. The default Apache installation directory is /usr/local/apache2. You can enable a few option by configuring with this command:

<shell> ./configure --enable-so --enable-dav --enable-dav-fs --enable-dbd --with-mysql=/usr 

Now, it is time to install Catacomb. You can download the latest version from the Catacomb project Web Site. Untar the source. When you configure the Catacomb, you can select two options, --with-apache and --with-debug. Using --with-debug option, you can enable Catacomb to log everything to apache logfile, and using --with-apache, you will specify the apache directory. The default directory is the /usr/local/apache2 directory for Apache.

After finishing the configuration, you can do make and make install.

3.4 Configuration

We need two configurations such as:

  • Apache configuration.
  • DBMS initialization.

You want to add these lines into the /usr/local/apache2/conf/httpd.conf file.

____________________________________________________________________
|# load the module                                                  |
|LoadModule dav_repos_module   modules/libmod_dav_repos.so          |
|                                                                   |
|# This is server configure now                                     |
|DavDBMSTmpDir /tmp/                                                |
|DavDBMSFileDir /usr/local/apache2/htdocs/repos/                    |
|DavDMBSMaxFileSize 102400                                          |
|                                                                   |
|<Location /repos>                                                  |
|DAV repos                                                          |
|<//Location>                                                       |
|                                                                   |
|DBDriver mysql                                                     |
|DBDParams "host=localhost,user=user,pass=pass,dbname=catacomb_dbd" |
--------------------------------------------------------------------
Here are the directive explanations:
  • DavDBMSTmpDir: Catacomb needs a temporary directory to manipulate the files. Make sure your HTTP daemon has writing permission in the directory.
  • Location specifies the URI information. You need to specify DAV provider as "repos".

Second configuration job is creating DB and tables for mod_dav_dasl. DB which is already exist for other usage is reusable for Catacomb, but creating a new DB for Catacomb is strongly recommend. You can create new DB using the mysqladmin command:

<shell>mysqladmin create repos
This command will create DB named repos. There are several tables used by Catacomb. You can create tables by importing table.sql in the Catacomb source directory using this command:
<shell>mysql repos < table.sql
You might need -u and -p options for MySQL, if you specified a database user for the DB. Finally, you need to import data for your root information of the Catacomb enabled directory. In our example, the directory name is /repos. If you specified different directory name, you need to edit data.sql in the Catacomb source directory. Also, if you have more than two directory directives for Catacomb, you need to import data.sql several times with different directory data. This is the example of data.sql.
INSERT INTO dasl_resource VALUES 
(1,'','/repos',651165539,'(null)','(null)',0,'httpd/unix-directory',
'(null)',1014759103979654,'(null)',1,0,'(null)',1,2,NULL,'1234567890');
Make sure the third arguments in the example (/repos) is same as your Location in httpd.conf. You want to import the data.sql using this command:
<shell>mysql repos < data.sql

3.5 Fire your Apache.

<shell>/usr/local/apache2/bin/apachectl start

4. DASL Clients

If you want to test your DASLed server very quickly, the simplest and best way is using telnet. Do telnet to your server and the port. Let's suppose our DASLed server is ocean.cse.ucsc.edu and port is 7777. We can test by typing the OPTION method:

ocean 10> telnet ocean.cse.ucsc.edu 7777
Trying 128.114.51.104...
Connected to ocean.cse.ucsc.edu.
Escape character is '^]'.
OPTIONS /dasl/ HTTP/1.1
Host: ocean.cse.ucsc.edu

HTTP/1.1 200 OK
Date: Wed, 08 May 2002 09:16:13 GMT
Server: Apache/2.2.6 (Unix) DAV/2 DASL/0.1
DAV: 1
DAV: <http://apache.org/dav/propset/fs/1>
MS-Author-Via: DAV
Allow: OPTIONS,GET,HEAD,POST,DELETE,TRACE,PROPFIND,PROPPATCH,COPY,MOVE,SEARCH
DASL: <DAV:basicsearch>
Content-Length: 0
Content-Type: text/plain; charset=ISO-8859-1
If you can see SEARCH method in Allow field in the response, your DASLed server is working properly.

4.1 Read/Write resources

Unfortunately, you can not copy resources or make directory directly like usual Apache file system directories, since Catacomb use MySQL to save all resources and their data. So you need to use Web-DAV client to read/write your resources in the DASLed server. There are many WebDAV client programs. You can use one of these client and read/write your resources. If you want to only read your resources, a web browser are a good tool. For more information about WebDAV software, visit the WebDAV resource site.

4.2 Search

So far, there is not many DASL clients. When we write this HOWTO, cadaver DASL patch is on developing. The cadaver client will support DASL, when you read this HOWTO. Hopefully there will be lots of DASL clients very soon.

5. Trouble shooting.

If your DASLed server doesn't work properly, reading error_log in the Apache's log directory is good idea. Here are simple check lists:

  • Did you configure Apache with --enable-so, --enable-dav and --enable-dbd option?
  • MySQL demon is running before Apache server is running?
  • Did you specify the right DB information in httpd.conf?
  • Temporary directory which is specified httpd.conf is readable and writable?
  • ...

6. Conclusion

Catacomb project is just started, so we have lots of thing to do and to improve though. I hope Catacomb and this HOWTO could help your Internet environment.

6.1 Call for participation

Catacomb is open source project under Apache License. So anybody is welcome to the project. Please content to dev[at]catacomb.tigris.org, if you have interest. To join Click Here.