Texis Web Script source code for:
https:///texisdocs.thunderstone.com
Note: Click on links to view the documentation for HTML tags which are special to the Vortex compiler.
The Source Viewer is also a Code Example.
Note: Click on links to view the documentation for HTML tags which are special to the Vortex compiler.
<!--======================================================================== This script will login to the Search Appliance or Webinator web admin interface and request various status pages and summarize information from them. Look below for "CUSTOMIZATION AREA" for variables that may need to be changed based on your installation as well as default values for: profile - Default profile name to query loginid - Default login name loginpass - Password for loginid server - Default hostname of the machine with the admin interface to query If you set the default for any of those to "" it will require that they be specified on the command line. Specify them on the command line with this syntax: texis profile="yourprofile" loginid="login" loginpass="password" server="hostname" getinfo Once those few variables are set this script will work as-is. But you will almost certainly want to customize the reporting functions to adjust what data is reported and in what format. ========================================================================--> <script language=vortex> <!--========================================================================--> <!-- 62 seconds is enough for 2x30 second fetches to timeout and not kill us --> <timeout=62> The server is not responding or is responding too slowly. </timeout> <entryfunc=init><!-- function to execute before any other on every run --> <!--========================================================================--> <!-- setup vars and login cookie --> <a name=init> <if '' neq $REMOTE_ADDR>This is not a web app.<exit 1></if> <if 'y' eq $didinit><return></if><!-- prevent multiple inits --> <$didinit=y> <!-- ****************************************************************** --> <!-- *********************** CUSTOMIZATION AREA *********************** --> <!-- ****************************************************************** --> <!-- Adjust values for your server or set them on the texis command line to override these settings --> <if '' eq $profile ><$profile="YOUR_PROFILE_NAME_HERE"></if> <if '' eq $loginid ><$loginid="YOUR_LOGIN_ID_HERE"></if> <if '' eq $loginpass><$loginpass="YOUR_PASSWORD_HERE"></if> <if '' eq $server ><$server="YOUR_SERVER_NAME_HERE"></if> <!-- dowalk - the full web path to the dowalk script on $server some possible values: Webinator - "cgi-bin/texis/webinator/dowalk" Webinator - "scripts/texis.exe/webinator/dowalk" Webinator - "texis/webinator/dowalk" Appliance - "texis/dowalk" --> <$dowalk="texis/dowalk"><!-- appliance --> <!-- ****************************************************************** --> <!-- ******************* END OF CUSTOMIZATION AREA ******************** --> <!-- ****************************************************************** --> <vxcp htmlmode off><!-- prevent any HTML escapement of the output --> <login><!-- login to the admin interface --> </a> <!--========================================================================--> <!-- login to the web interface --> <a name=login> <if 'y' eq $didlogin><return></if><!-- prevent multiple logins --> <$didlogin=y> <strfmt "http://%s/%U/login.html" $server $dowalk> <submit method=post url=$ret iname=$loginid ipass=$loginpass> <$page=$ret> <urlinfo errnum> <if $ret gt 0><!-- ensure no network or http errors --> Error logging into the server: <urlinfo errstr>$ret <exit 1> <else> <rex "Existing profiles" $page /><!-- ensure we get the expected page --> <if $ret eq ''> Login to server failed. Expected profile listing, got this: $page <exit 1> <else> Got login ok. </if> </if> </a> <!--========================================================================--> <!-- download the query log tab separated data --> <a name=querylog public> <if '' eq $profile><return></if> <strfmt "http://%s/%U/report.xls?r=SSc_queryreport&wr=excel&profile=%U" $server $dowalk $profile> <fetch urls=$ret /> <$page=$ret> <urlinfo errnum> <if $ret gt 0><!-- ensure no network or http errors --> Error getting server profile status: <urlinfo errstr>$ret <else> Writing querylog.xls <write "querylog.xls"><fmt "%s" $page></write> <!-- you could also use <timport> to parse the data into this script and do summary reports from here. Here is a sample --> <$nsearches=0> <$nclicks=0> <$nfails=0> <timport "csv \x09 field Date date 1 field IP varchar 2 field What varchar 3 field Query varchar 4 field IndexCount int 5 field Url varchar 6 field RankOrder int 7 field RankProx int 8 field RankDFreq int 9 field RankWFreq int 10 field Lead int 11 field Proximity varchar 12 field Suffixes int 13 " $page> <switch $What> <case "Click"> <!-- Clickover: $Date,$Query,$Url --> <$nclicks=($nclicks+1)><!-- count click overs --> <case "Query"> <!-- Search: $Date,$Query,$IndexCount --> <$nsearches=($nsearches+1)><!-- count searches --> <if $IndexCount eq 0><$nfails=($nfails+1)></if><!-- count no answer queries --> <xtree insert $Query queries /><!-- record all unique queries --> <rex "[^\space,\x22]+" $Query><!-- record all unique query terms --> <xtree insert $ret qwords /> </rex> <default> Uknown query log record type: $What for $Date </switch> <xtree insert $IP users /><!-- record all unique client IPs --> </timport> Click-over ratio: $nclicks/$nsearches=<$ret=(convert($nclicks, 'double' )/convert($nsearches, 'double' ))><fmt "%.2f\n" $ret> Bad search ratio: $nfails/$nsearches=<$ret=(convert($nfails, 'double' )/convert($nsearches, 'double' ))><fmt "%.2f\n" $ret> Top queries: <xtree dump queries /><$queries=$ret> <xtree count queries /><$counts=$ret> <sort $counts num desc $queries> <loop max=10 $queries $counts> <fmt "%5d %s\n" $counts $queries> </loop> Top query words: <xtree dump qwords /><$queries=$ret> <xtree count qwords /><$counts=$ret> <sort $counts num desc $queries> <loop max=10 $queries $counts> <fmt "%5d %s\n" $counts $queries> </loop> Top client IPs: <xtree dump users /><$queries=$ret> <xtree count users /><$counts=$ret> <sort $counts num desc $queries> <loop max=10 $queries $counts> <fmt "%5d %s\n" $counts $queries> </loop> </if> </a> <!--========================================================================--> <!-- indicate the current walk rate and last walk time --> <!-- then dump the entire walk status page text --> <a name=walkstatus public> <local statusurl text line> <if '' eq $profile><return></if> <strfmt "http://%s/%U/report.html?r=SSc_longreport&profile=%U" $server $dowalk $profile> <$statusurl=$ret> <fetch urls=$statusurl /> <$page=$ret> <urlinfo errnum> <if $ret gt 0><!-- ensure no network or http errors --> Error getting server profile status: <urlinfo errstr>$ret <else> Status for profile $profile: <sandr ">><noindex>=!</noindex>+</noindex>" "" $page> <fetch urls=$statusurl downloaddoc=$ret /> <urlinfo text><$text=$ret> <rex "[^\n]+>>pages visited in the last hour=[^\n]+" $text /> <$line=$ret> $line <rex ">>=\digit+\F pages" $line /> <$npages=$ret> <if $npages eq 0> NOTICE: No pages fetched in the last hour! </if> <rex ">>\nWalk started at \P=....-..-.. ..:..:..=\F " $text /> INFO: Last walk started at $ret <!-- uncomment these 2 lines to get the full text of the status page Full Status: $text --> </if> </a> <!--========================================================================--> <!-- get system info such as load average and disk usage - warn if high --> <!-- only valid for appliance. Webinator doesn't have a system status page --> <!-- this function does not need to have a profile specified --> <a name=system public> <local infourl text loadstr load diskstr> <strfmt "http://%s/%U/techsupportinfo.html" $server $dowalk> <$infourl=$ret> <fetch urls=$infourl /> <$page=$ret> <urlinfo errnum> <if $ret gt 0><!-- ensure no network or http errors --> Error getting server info: <urlinfo errstr>$ret <else> <rex '<H\digit>Tech Support Info' $page /> <if $ret eq ''> Couldn't get tech support page. Probably not an appliance. <else> <urlinfo text><$text=$ret> <rex "kdbf" $page /> <if $ret ne ''> NOTICE: Database errors detected. </if> <rex "ABEND" $page /> <if $ret ne ''> NOTICE: ABENDs detected. </if> <rex '>>>Load:</TH><TD>=!load average:\P+load average:=!</TD>+\F</TD>' $page /> <$loadstr=$ret> $loadstr <rex "average:=\space\P*[\digit\.]+\F," $loadstr /> <$load=(convert($ret, 'double' ))> <if $load gt 0.90> NOTICE: load seems high ($load) </if> <rex ">>Disk Usage=\n\P+!\n\n+\F\n\n" $text /> <sandr ">>=\space+" "" $ret> <sandr ">>\n\P=\space+" "" $ret> <$diskstr=$ret> $diskstr <timport "csv \x09 field Part varchar 1 field PctUsed int 2 field MAvail int 3 field MUsed int 4 field MSize int 5 " $diskstr> <if $PctUsed gt 90> NOTICE: Disk partion $Part getting full ($PctUsed%) </if> </timport> </if> </if> </a> <!--========================================================================--> <!-- report on all known items --> <a name=main public> <querylog> -------------------------- <walkstatus> -------------------------- <system> -------------------------- </a> <!--========================================================================--> </script>