Syntax: one or more file extensions separated by space
A list of the URL extensions that the crawler will accept. The defaults are
.html
.htm
.txt
.pdf
To search MS-Word documents, use .doc
. For Shockwave/Flash use
.swf
. For WordPerfect documents specify whatever extension you
use and ensure that the web server returns the MIME type
application/wordperfect
as there is no consistent extension for
WordPerfect documents. Any extensions not listed here will not be
searched or walked.
A few other extensions you may find useful are
.asp
.cfm
.jsp
.shtml
.jhtml
.phtml