A directory in which to store the index files. The
default is the empty string, which means use the database directory.
This can be used to put the indexes onto another disk to balance load
or for space reasons. If indexspace is set to a non-default
value when a Metamorph index is being updated, the new index will be
stored in the new location.
indexblock
When a Metamorph index is created on an indirect field,
the indirect files are read in blocks. This property allows the size
of the block used to be redefined.
indexmem
When indexes are created Texis will use memory to speed up the
process. This setting allows the amount of memory used to be
adjusted. The default is to use 40% of physical memory, if it
can be determined, and to use 16MB if not. If the value set is less
than 100 then it is treated as a percentage of physical memory. It
the number is greater than 100 then it is treated as the number of
bytes of memory to use. Setting this value too high can cause
excessive swapping, while setting it too low causes unneeded extra
merges to disk.
indexmeter
Whether to print a progress meter during index creation/update.
The default is 0 or 'none', which suppresses the meter.
A value of 1 or 'simple' prints a simple hash-mark meter
(with no tty control codes; suitable for redirection to a file and
reading by other processes). A value of 2 or 'percent' or
'pct' prints a hash-mark meter with a more detailed percentage
value (suitable for large indexes). Added in
version 4.00.998688241 Aug 24 2001.
meter
A semicolon-separated list of processes to print a progress meter
for. Syntax:
{process[=type]}|type [; ...]
A process is one of index, compact, or the catch-all
alias all. A type is a progress meter type, one of
none, simple, percent, on (same as
simple) or off (same as none). The default
type if not given is on. E.g. to show a progress meter for
all meterable processes, simply set meter to on.
Added in version 6.00.1290500000 20101123.
addexp
An additional REX expression to match words to be
indexed in a Metamorph index. This is useful if there are
non-English words to be searched for, such as part numbers. When an
index is first created, the expressions used are stored with it so
they will be updated properly. The default expression is
\alnum{2,99}. Note: Only the expressions set when the
index is initially created (i.e. the first CREATE METAMORPH
... statement - later statements are index updates) are saved.
Expressions set during an update (issuance of "create
metamorph [inverted] index" on an existent index) will not
be added.
delexp
This removes an index word expression from the list.
Expressions can be removed either by number (starting with 0) or by
expression.
lstexp
Lists the current index word expressions. The
value specified is ignored (but required syntactically).
addindextmp
Add a directory to the list of directories to use
for temporary files while creating the index. If temporary files are
needed while creating a Metamorph index they will be created in one of
these directories, the one with the most space at the time of
creation. If no addindextmp dirs are specified, the default
list is the index's destination dir (e.g. database or indexspace),
and the environment variables TMP and TMPDIR.
delindextmp
Remove a directory from the list of directories to
use for temporary files while creating a Metamorph index.
lstindextmp
List the directories used for temporary files while
creating Metamorph indices. Aka listindextmp.
indexvalues
Controls how a regular (B-tree) index stores table values. If set
to splitstrlst (the default), then strlst-type fields
are split, i.e. a separate (item,recid) tuple is stored for
each (varchar) item in the strlst, rather than just
one for the whole (strlst,recid) tuple. This allows the index to be
used for some set-like operators that look at individual items in a
strlst, such as most IN, SUBSET
(here) and INTERSECT
(here) queries.
If indexvalues is set to all - or the index is not on
a strlst field, or is on multiple fields - such splitting
does not occur, and the index can generally not be used for set-like
queries (with some exceptions; see here for
details).
Note that if index values are split (i.e. splitstrlst set and
index is one field which is strlst), table rows with an empty
(zero-items) strlst value will not be stored in the index.
This means that queries that require searching for or listing
empty-strlst table values cannot use such an index. For
example, a subset query with a non-empty parameter on the right side
and a strlst table column on the left side will not be able
to return empty-strlst rows when using an index, even though
they match. Also, subset queries with an empty-strlst or
empty-varchar parameter (left or right side) must use an
indexvalues=all index instead. Thus if empty-strlst
subset query parameters are a possibility, both types of index
(splitstrlst and all) should be created.
As with stringcomparemode, only the creation-time
indexvalues value is ever used by an index, not the current
value, and the optimizer will attempt to choose the best index at
search time. The indexvalues setting was added in Texis
version 7; previous versions effectively had indexvalues set
to splitstrlst. Caveat: A version 6 Texis will issue
an error when encountering an indexvalues=all index (as it is
unimplemented in version 6), and will refuse to modify the index or
the table it is on. A version 5 or earlier Texis, however, may
silently corrupt an indexvalues=all index during table
modifications.
btreethreshold
This sets a limit as to how much of an index should
be used. If a particular portion of the query matches more than the given
percent of the rows the index will not be used. It is often more efficient
to try and find another index rather than use an index for a very frequent
term. The default is set to 50, so if more than half the records match,
the index will not be used. This only applies to ordinary indices.
btreelog
Whether to log operations on a particular B-tree, for
debugging. Generally enabled only at the request of tech support.
The value syntax is:
[on=|off=][/dir/]file[.btr]
Prefixing on= or off= turns logging on or off,
respectively; the default (if no prefix) is on. Logging applies to
the named B-tree file; if a relative path is given, logging applies to
the named B-tree in any database accessed.
The logging status is also saved in the B-tree file itself, if the
index is opened for writing (e.g. at create or update). This means
that once logging is enabled and saved, every process that
accesses the B-tree will log operations, not just ones that have
btreelog explicitly set. This is critical for debugging, as
every operation must be logged. Thus, btreelog can just be set
once (e.g. at index create), without having to modify (and track down)
every script that might use the B-tree. Logging can be disabled
later, by setting "off=file" and accessing the index for an
update.
Operations are logged to a text file with the same name as the
B-tree, but ending in ".log" instead of ".btr".
The columns in the log file are as follows; most are for tech
support analysis, and note that they may change in a future Texis release:
Date Date
Time Time (including microseconds)
Script and line Vortex script and line number, if known
PID Process ID
DBTBL handleDBTBL handle
Read locks Number of read locks (DBTBL.nireadl)
Write locks Number of write locks (DBTBL.niwrite)
B-tree handleBTREE handle
Action What action was taken:
open B-tree open: Recid is root page offset
create B-tree create
close B-tree close
RDroot Read root page
dump B-tree dump
WRhdr Write B-tree header: Recid is root page offset
WRdd Write data dictionary: Recid is DD offset.
(Read DD at open is not logged.)
delete Delete key: Recid is for the key
append Append key
insert Insert key
search Search for key
RDpage Read page: Recid is for the page
WRpage Write page
CRpage Create page
FRpage Free page
FRdbf Free DBF block
Result Result of action:
ok Success
fail Failure
dup Duplicate (e.g. duplicate insert into unique B-tree)
hit Search found the key
miss Search did not find the key
Search mode Search mode:
B Find before
F Find
A Find after
Index guaranteeDBTBL.indguar flag (1 if no
post-process needed)
Index type Index type:
NDBIDX_NATIVE (bubble-up)
MDBIDX_MEMORY (RAM B-tree)
CDBIDX_CACHE (RAM cache)
Recid Record id; see notes for Action column
Key size Key size (in bytes)
Key flags Flags for each key value, separated by commas:
DOF_DESCENDING
IOF_IGN_CASE
XOF_DONT_CARE
EOF_PREFER_END
SOF_PREFER_START
Key Key, i.e. value being inserted, deleted etc.;
multiple values separated with commas
Unavailable or not-applicable fields are logged with a dash. Note
that enabling logging can produce a large log file quickly; free disk
space should be monitored. The btreelog setting was added in
version 5.01.1134028000 20051208.
btreedump
Dump B-tree indexes, for debugging. Generally enabled
only at the request of tech support. The value is an integer whose
bits are defined as follows:
Bits 0-15 define what to dump. Files are created that are named after
the B-tree, with a different extension:
Bit
0: Issue a putmsg about where dump file(s) are
Bit
1: .btree file: Copy of in-mem BTREE struct
Bit
2: .btrcopy file: Copy of .btr file
Bit
3: .cache file: Page cache from BCACHE, BPAGE
Bit
4: .his file: History from BTRL
Bit
5: .core file: fork() and dump core
Bits 16+ define when to dump:
Bit
16: At "Cannot insert value" messages
Bit
17: At "Cannot delete value" messages
Bit
18: At "Trying to insert duplicate value" messages
The files are for tech support analysis. Formats and bits subject to
change in future Texis releases. The btreedump setting was added
in version 5.01.1131587000 20051109.
maxlinearrows
This set the maximum number of records that should
be searched linearly. If using the indices to date yield a result set
larger than maxlinearrows then the program will try to find more
indices to use. Once the result set is smaller than maxlinearrows,
or all possible indices are exhausted, the records will be processed. The
default is 1000.
likerrows
How many rows a single term can appear in, and still be
returned by liker. When searching for multiple terms with liker
and likep one does not always want documents only containing a
very frequent term to be displayed. This sets the limit of what is
considered frequent. The default is 1000.
indexaccess
If this option is turned on then data from an index
can be selected as if it were a table. When selecting from an
ordinary (B-tree) index, the fields that the index was created on will
be listed. When selecting from a Metamorph index a list of words
(Word column`), count of rows containing each word
(RowCount), and - for Metamorph inverted indexes - count of
all hits in all rows (OccurrenceCount) for each word will be
returned.
indexchunk
In versions of Texis after October 1998, the
indexchunk setting is deprecated and unused. In prior releases,
when creating a Metamorph index temporary files are
used which in the worst case can grow to twice the size of the data
being indexed. This process can be broken into stages, such that
after indexing a certain amount of data the temporary files are
processed, to generate a partial index, and then the process repeats
for the rest of the data. By default the amount of free disk space is
checked on startup, and used to calculate when it will need to perform
the processing step. If the system does not report free disk space
accurately, or to free more disk space, this value can be changed.
The default is 0, which automatically calculates a value. Otherwise
it is set to the number of bytes of data to index before processing
the temporary files. Lower values conserve disk space, at the expense
of more time to process intermediate files.
cleanupwait
Windows/NT specific After updating a
Metamorph index the database will wait this long before trying to
remove the old copy of the index. This is to allow any other process
currently using the index time to stop using the index, so it can be
removed. The default is twenty seconds. If a whole batch of
Metamorph indices are being updated right after another, it may be
useful to set this to 0 for all but the last index, as an attempt will
be made to remove all old indices after every index update.
dbcleanupverbose
Integer whose bit flags control some tracing
messages about database cleanup housekeeping (e.g. removal of unneeded
temporary or deleted indexes and tables). A bit-wise OR of the
following values:
0x01: Report successful removal of temporary/deleted
indexes/tables.
0x02: Report failed removal of such indexes/tables.
0x04: Report on in-use checks of temporary indexes/tables.
The default is 0 (i.e. no messages). Note that these cleanup actions
may also be handled by the Database Monitor; see also the
[Monitor] DB Cleanup Verbose setting in conf/texis.ini. Added
in version 6.00.1339712000 20120614.
indextrace
For debugging: trace index usage, especially during
searches, issuing informational putmsgs. Greater values
produce more messages. Note that the meaning of values, as well as
the messages printed, are subject to change without notice.
Aka traceindex, traceidx. Added
in version 3.00.942186316 19991109.
tracerecid
For debugging: trace index usage for this particular
recid. Added in version 3.01.945660772 19991219.
indexdump
For debugging: dump index recids during search/usage.
Value is a bitwise OR of the following flags:
Bit 0
for new list
Bit 1
for delete list
Bit 2
for token file
Bit 3
for overall counts too
The default is 0.
indexmmap
Whether to use memory-mapping to access Metamorph
index files, instead of read(). The value is a bitwise OR
of the following flags:
Bit 0
for token file
Bit 1
for .dat file
The default is 1 (i.e. for token file only). Note that memory-mapping
may not be supported on all platforms.
indexreadbufsz
Read buffer size, when reading (not
memory-mapping) Metamorh index .tok and .dat files. The
default is 64KB; suffixes like "KB" are respected. During
search, actual read block size could be less (if predicted) or more
(if blocks merged). Also used during index create/update. Decreasing
this size when creating large indexes can save memory (due to the
large number of intermediate files), at the potential expense of time.
Aka indexreadbufsize. Added in version 4.00.1006398833
20011121.
indexwritebufsz
Write buffer size for creating Metamorph
indexes. The default is 128KB; suffixes like "KB" are
respected. Aka indexwritebufsize. Added in version
4.00.1007509154 20011204.
indexmmapbufsz
Memory-map buffer size for Metamorph indexes.
During search, it is used for the .dat file, if it is
memory-mapped (see indexmmap); it is ignored for the
.tok file since the latter is heavily used and thus fully
mapped (if indexmmap permits it). During index update,
indexmmapbufsz is used for the .dat file, if it is
memory-mapped; the .tok file will be entirely memory-mapped if
it is smaller than this size, else it is read. Aka
indexmmapbufsize. The default is 0, which uses 25% of RAM.
Added in version 3.01.959984092 20000602. In version 4.00.1007509154
20011204 and later, "KB" etc. suffixes are allowed.
indexslurp
Whether to enable index "slurp" optimization
during Metamorph index create/update, where possible. Optimization is
always possible for index create; during index update, it is possible
if the new insert/update recids all occur after the original recids
(e.g. the table is insert-only, or all updates created a new block).
Optimization saves about 20% of index create/update time by merging
piles an entire word at a time, instead of word/token at a time. The
default is 1 (enabled); set to 0 to disable. Added in version
4.00.1004391616 20011029.
indexappend
Whether to enable index "append" optimization
during Metamorph index update, where possible. Optimization is
possible if the new insert recids all occur after the original recids,
and there were no deletes/updates (e.g. the table is insert-only); it
is irrelevant during index create. Optimization saves index build
time by avoiding original token translation if not needed. The
default is 1 (enabled); set to 0 to disable. Added in version
4.00.1006312820 20011120.
indexwritesplit
Whether to enable index "write-split"
optimization during Metamorph index create/update. Optimization saves
memory by splitting the writes for (potentially large) .dat
blocks into multiple calls, thus needing less buffer space. The
default is 1 (enabled); set to 0 to disable. Added in version
4.00.1015532186 20020307.
indexbtreeexclusive
Whether to optimize access to certain
index B-trees during exclusive access. The optimization may reduce
seeks and reads, which may lead to increased index creation speed on
platforms with slow large-file lseek behavior. The default is
1 (enabled); set to 0 to disable. Added in version 5.01.1177548533
20070425.
mergeflush
Whether to enable index "merge-flush" optimization
during Metamorph index create/update. Optimization saves time by
flushing in-memory index piles to disk just before final merge;
generally saves time where indexslurp is not possible. The
default is 1 (enabled); set to 0 to disable. Added in version
4.00.1011143988 20020115.
indexversion
indexversion
Which version of Metamorph index to produce or
update, when creating or updating Metamorph indexes. The supported
values are 0 through 3; the default is 2. Setting version 0 sets the
default index version for that Texis release. Note that old versions
of Texis may not support version 3 indexes. Version 3 indexes may use
less disk space than version 2, but are considered experimental.
Added in version 3.00.954374722 20000329.
indexmaxsingle
For Metamorph indexes; the maximum number of
locations that a single-recid dictionary word may have and still be
stored solely in the .btr B-tree file (without needing a
.dat entry). Single-recid-occurence words usually have their
data stored solely in the B-tree to save a .dat access at
search time. However, if the word occurs many times in that single
recid, the data (for a Metamorph inverted index) may be large enough
to bloat the B-tree and thus negate the savings, so if the
single-recid word occurs more than indexmaxsingle times, it is
stored in the .dat. The default is 8.
uniqnewlist
Whether/how to unique the new list during
Metamorph index searches. Works around a potential bug in old
versions of Texis; not generally set. The possible values are:
0
: do not unique at all
1
: unique auxillary/compound index new list only
2
: unique all new lists
3
: unique all new lists and report first few duplicates
The default is 0.
tablereadbufsz
Size of read buffer for tables, used when it is
possible to buffer table reads (e.g. during some index creations). The
default is 16KB. When setting, suffixes such as "KB" etc.
are supported. Set to 0 to disable read buffering. Added in version
5.01.1177700467 20070427. Aka tablereadbufsize.