Syntax: zero or more REX expressions, separated by space of line break
Allow walk to fetch URLs matching any of the specified REX expressions anywhere in the URL (hostname, path, or query). Most commonly used in conjunction with a hostname to only fetch matching URLs on that host. Links still need to be found to those pages for them to be indexed.
Available from version 4.3.9
.
REX | Matches |
>>http://library/=!math*math= | Urls on the server library containing "math" |
See also Extra Domains
4.5.9.