loader - Example Loader Program

SYNOPSIS

loader [-ddatabase] [-ttable] file ...

DESCRIPTION
This example loader program shows how to put large amounts of data into a database without the overhead and limitations of a client/server architecture and the SQL language.

The example used is to build a database of patent information from the file mmex2.dat, which is shipped with Texis. The actual parsing of the file is intentionally left very simple, as the object of this example is to show how to load data into a database, not how to parse a data file.

Making and Running the Program

To make the program copy makefile and loader.c from the /usr/local/morph3/api directory to a working directory, then type make loader in that directory. To run the loader you should invoke it with ./loader /usr/local/morph3/api/mmex2.dat This will make a database in /usr/local/morph3/texis/testdb that contains a table patent which contains the patent information from the file mmex2.dat.

Options

-ddatabse

Use database instead of the default.

-ttable

Create table instead of the default.

To query the database you can either use the enclosed example client program netex3 or else use tsql, the interactive interface to Texis.

Program Internals

This section describes how the program is structured so that a programmer can create a similar program for their own data.

The basic structure of the program is:

1. Open database.

2. If failed then create database.

3. Open the table.

4. If failed then create table.

5. Get pointers to the fields.

6. Read a record from the data file.

7. Stuff the data in to the fields

8. Write fields to database.

9. Repeat from step 6 until file empty.

10. Close table.

11. Close database.

Calls used (in order of appearance)

DDIC *ddopen(char *dbname);

Open the database dbname. Returns NULL if the database can not be opened.

void createdb(char *dbname);

Create the database dbname. This creates the directory if needed, and also creates the system tables.

DBTBL *opendbtbl(DDIC *ddic, char *tbname);

Opens table tbname in the database ddic. Returns NULL if the table does not exist or could not be opened.

DD *opendd(void);

Open a Data Definition structure.

int putdd(DD *dd, char *name, char *type, int n, int nonnull);

Add a field to the data definition structure dd. The field will be called name and have the named type.

DBTBL *createdbtbl(DDIC *ddic, DD *dd, char *filename, char *tablename, char *comment, char type);

Create a new table in the data dictionary. Adds the table specified by dd to ddic. The filename to use on disk will be filename (with the possibility of an added suffix). The table will be know as tablename to the database. Comment is limited to 80 chars, and is stored in the system table. Nothing else is done with comment. Type should be 'T' for most tables. The use of other values should be avoided.

DD *closedd(DD *dd);

Close a data definition structure, and free the associated memory.

DDIC *ddclose(DDIC *ddic);

Close a database, and free memory.

FLD *dbnametofld(DBTBL *tbl, char *name);

Get a field from a table. This will return a pointer to the field called name.

void getindexes(DBTBL *tbl);

This function looks up all the indices associated with a table so that when inserts are made to the table the indices will be kept up to date. If you fail to use this call then the indices will not reflect the tables.

void putfld(FLD *fld, void *buf, size_t n);

Put data into a field. The pointer buf is stored into the field fld. N specifies how many elements of the datatype are present. The data is not copied, so buf must remain valid until the field is no longer needed.

RECID *putdbtblrow(DBTBL *tbl, RECID *loc);

Tries to write the current data in tbl to location loc. If the record at loc is too small, or loc is NULL then a new location will be found. The return value is the location that the record was stored at, or NULL if an error occurred.

DBTBL *closedbtbl(DBTBL *tbl);

Close the table, and free associated memory.