Webinator WWW Site Indexer Version 2.5

Thunderstone Software

• Contents
• Overview
  • Features
  • Obtaining Webinator
  • Technical Support
• Installation
  • Download and Install
  • Filesystem Layout
  • Customizing Webinator's Appearance
• Operation
  • Database Usage
  • Indexing
  • Robots.txt page exclusion
  • Command Line Reference
    • Database Tables and Fields
  • Command Line Option syntax
  • Help and Configuration
    • Show version (-version)
    • Show help (-h)
    • Use option file (-m)
    • Save settings (-save)
    • Recall settings (-recall)
    • Don't save settings (-O)
    • Set verbosity (-v)
    • Increase verbosity (-v)
  • Database and File Usage
    • Set database name (-d)
    • Log file name (-l)
    • Set index document name (-I)
  • Database Management
    • Create database (-create)
    • Wipe database (-wipe)
    • Wipe todo list (-wipetodo)
    • Update search indices (-index or -i)
    • Don't automatically update search indices (-noindex)
    • Delete search indices (-unindex)
    • Delete all indices (-dropindex)
    • Rewalk (-rewalk)
    • Rewalk on a schedule (-rewalk)
    • Set word expression (-k)
    • Execute arbitrary SQL (-s)
    • Enable DNS caching (-dnscache)
    • Disable DNS caching (-nodnscache)
    • Enable extra page duplication prevention (-unique)
    • Disable extra page duplication prevention (-nounique)
  • Page Fetching Behavior
    • Grab off-site pages (-o)
    • Don't add to todo (-a)
    • Allow an entire domain (-domain)
    • Allow an entire network (-network)
    • Control allowed extensions (-f and -F)
    • Define a plugin (-n)
    • Allow cgi-bin paths (-C)
    • Allow all file types (-y)
    • Add allowed MIME type (-T)
    • Delete allowed MIME type (-S)
    • Define prefix exclusions (-x)
    • Define path pattern exclusions (-x/)
    • Define query pattern exclusions (-exquery)
    • Define URL prefix restrictions (-j)
    • Ignore robots.txt (-r)
    • Set login user name (-U)
    • Set login password (-P)
    • Set proxy to use (-proxy)
    • Don't lookup every hostname (-L)
    • Don't report as Mozilla (-M)
    • Set reported Mozilla version number (-M)
    • Set reported User-Agent (-M/)
    • Index meta data (-meta)
    • Strip query strings (-Q)
    • Don't include alt text (-N)
    • Don't store refs (-R)
    • Force page insertion (-Force)
  • Time and Size Limits
    • Set delay between pages (-w)
    • Set page timeout (-t)
    • Limit depth of walk (-D)
    • Walk breadth first (-b)
    • Limit run time (-q)
    • Limit retrieved pages (-p)
    • Limit retrieved page size (-z)
  • Mode Control
    • Get a single page (-g)
    • Reload pages (-e)
    • Delete missing pages (-X)
    • Only download modified pages (-V)
    • Copy web site (-c)
    • Don't reauthor during copy (-A)
    • Reauthor to specified path during copy (-A)
  • Other Arguments
    • URL to walk
    • SQL statement to execute
• Procedures and Examples
  • Indexing your Site
  • Searching your Index
  • Page Exclusion and Robots.txt
  • Indexing Other Sites
  • Indexing Individual Pages
  • Reindexing on a schedule
  • Checking for WEB Server Errors
  • Removing Pages from the Database
  • Erasing the Entire Database
  • Using Multiple Databases
  • Using Option Files