1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>
ht://Dig: Configuration
</title>
</head>
<body bgcolor="#eef7ff">
<h1>
Configuration
</h1>
<p>
ht://Dig Copyright © 1995-2004 <a href="THANKS.html">The ht://Dig Group</a><br>
Please see the file <a href="COPYING">COPYING</a> for
license information.
</p>
<hr size="4" noshade>
<p>
ht://Dig requires a configuration file and several HTML files
to operate correctly. Fortunately, when ht://Dig is
installed, a very reasonable configuration is created and in
most cases only minor modifications to the files are
necessary.
</p>
<p>
Below, we will use the variables that were set in CONFIG to
designate specific paths.
</p>
<p>
Standard files:
</p>
<ul>
<li>
<a href="#htdig.conf">${CONFIG_DIR}/htdig.conf</a>
</li>
<li>
<a href="#search.html">${SEARCH_DIR}/search.html</a>
</li>
<li>
<a href="#header.html">${COMMON_DIR}/header.html</a>
</li>
<li>
<a href="#footer.html">${COMMON_DIR}/footer.html</a>
</li>
<li>
<a href="#wrapper.html">${COMMON_DIR}/wrapper.html</a>
</li>
<li>
<a href="#nomatch.html">${COMMON_DIR}/nomatch.html</a>
</li>
<li>
<a href="#syntax.html">${COMMON_DIR}/syntax.html</a>
</li>
</ul>
<hr noshade>
<h2>
<a name="#htdig.conf">${CONFIG_DIR}/htdig.conf</a>
</h2>
<p>
This is the main runtime configuration file for all programs
that make up ht://Dig. The file is fully described in the
<a href="confindex.html" target="_top">Configuration file
manual</a>.
</p>
<p>
When ht://Dig is installed, several attributes will be
customized to your particular environment, but for reference,
here is a sample copy of what it can look like:
</p>
<pre>
<font size="-1">
#
# Example config file for ht://Dig.
#
# This configuration file is used by all the programs that make up ht://Dig.
# Please refer to the attribute reference manual for more details on what
# can be put into this file. (http://www.htdig.org/confindex.html)
# Note that most attributes have very reasonable default values so you
# really only have to add attributes here if you want to change the defaults.
#
# What follows are some of the common attributes you might want to change.
#
#
# Specify where the database files need to go. Make sure that there is
# plenty of free disk space available for the databases. They can get
# pretty big.
#
database_dir: /opt/www/htdig/db
#
# This specifies the URL where the robot (htdig) will start. You can specify
# multiple URLs here. Just separate them by some whitespace.
# The example here will cause the ht://Dig homepage and related pages to be
# indexed.
# You could also index all the URLs in a file like so:
# start_url: `${common_dir}/start.url`
#
start_url: http://www.htdig.org/
#
# This attribute limits the scope of the indexing process. The default is to
# set it to the same as the start_url above. This way only pages that are on
# the sites specified in the start_url attribute will be indexed and it will
# reject any URLs that go outside of those sites.
#
# Keep in mind that the value for this attribute is just a list of string
# patterns. As long as URLs contain at least one of the patterns it will be
# seen as part of the scope of the index.
#
limit_urls_to: ${start_url}
#
# If there are particular pages that you definitely do NOT want to index, you
# can use the exclude_urls attribute. The value is a list of string patterns.
# If a URL matches any of the patterns, it will NOT be indexed. This is
# useful to exclude things like virtual web trees or database accesses. By
# default, all CGI URLs will be excluded. (Note that the /cgi-bin/ convention
# may not work on your web server. Check the path prefix used on your web
# server.)
#
exclude_urls: /cgi-bin/ .cgi
#
# Since ht://Dig does not (and cannot) parse every document type, this
# attribute is a list of strings (extensions) that will be ignored during
# indexing. These are *only* checked at the end of a URL, whereas
# exclude_url patterns are matched anywhere.
#
# Also keep in mind that while other attributes allow regex, these must be
# actual strings.
#
bad_extensions: .wav .gz .z .sit .au .zip .tar .hqx .exe .com .gif \
.jpg .jpeg .aiff .class .map .ram .tgz .bin .rpm .mpg .mov .avi .css
#
# The string htdig will send in every request to identify the robot. Change
# this to your email address.
#
maintainer: unconfigured@htdig.searchengine.maintainer
#
# The excerpts that are displayed in long results rely on stored information
# in the index databases. The compiled default only stores 512 characters of
# text from each document (this excludes any HTML markup...) If you plan on
# using the excerpts you probably want to make this larger. The only concern
# here is that more disk space is going to be needed to store the additional
# information. Since disk space is cheap (! :-)) you might want to set this
# to a value so that a large percentage of the documents that you are going
# to be indexing are stored completely in the database. At SDSU we found
# that by setting this value to about 50k the index would get 97% of all
# documents completely and only 3% was cut off at 50k. You probably want to
# experiment with this value.
# Note that if you want to set this value low, you probably want to set the
# excerpt_show_top attribute to false so that the top excerpt_length characters
# of the document are always shown.
#
max_head_length: 10000
#
# To limit network connections, ht://Dig will only pull up to a certain limit
# of bytes. This prevents the indexing from dying because the server keeps
# sending information. However, several FAQs happen because people have files
# bigger than the default limit of 100KB. This sets the default a bit higher.
# (see <http://www.htdig.org/FAQ.html> for more)
#
max_doc_size: 200000
#
# Most people expect some sort of excerpt in results. By default, if the
# search words aren't found in context in the stored excerpt, htsearch shows
# the text defined in the no_excerpt_text attribute:
# (None of the search words were found in the top of this document.)
# This attribute instead will show the top of the excerpt.
#
no_excerpt_show_top: true
#
# Depending on your needs, you might want to enable some of the fuzzy search
# algorithms. There are several to choose from and you can use them in any
# combination you feel comfortable with. Each algorithm will get a weight
# assigned to it so that in combinations of algorithms, certain algorithms get
# preference over others. Note that the weights only affect the ranking of
# the results, not the actual searching.
# The available algorithms are:
# accents
# exact
# endings
# metaphone
# prefix
# regex
# soundex
# speling [sic]
# substring
# synonyms
# By default only the "exact" algorithm is used with weight 1.
# Note that if you are going to use the endings, metaphone, soundex, accents,
# or synonyms algorithms, you will need to run htfuzzy to generate
# the databases they use.
#
search_algorithm: exact:1 synonyms:0.5 endings:0.1
#
# The following are the templates used in the builtin search results
# The default is to use compiled versions of these files, which produces
# slightly faster results. However, uncommenting these lines makes it
# very easy to change the format of search results.
# See <http://www.htdig.org/hts_templates.html> for more details.
#
# template_map: Long long ${common_dir}/long.html \
# Short short ${common_dir}/short.html
# template_name: long
#
# The following are used to change the text for the page index.
# The defaults are just boring text numbers. These images spice
# up the result pages quite a bit. (Feel free to do whatever, though)
#
next_page_text: <img src="/htdig/buttonr.gif" border="0" align="middle" width="30" height="30" alt="next">
no_next_page_text:
prev_page_text: <img src="/htdig/buttonl.gif" border="0" align="middle" width="30" height="30" alt="prev">
no_prev_page_text:
page_number_text: '<img src="/htdig/button1.gif" border="0" align="middle" width="30" height="30" alt="1">' \
'<img src="/htdig/button2.gif" border="0" align="middle" width="30" height="30" alt="2">' \
'<img src="/htdig/button3.gif" border="0" align="middle" width="30" height="30" alt="3">' \
'<img src="/htdig/button4.gif" border="0" align="middle" width="30" height="30" alt="4">' \
'<img src="/htdig/button5.gif" border="0" align="middle" width="30" height="30" alt="5">' \
'<img src="/htdig/button6.gif" border="0" align="middle" width="30" height="30" alt="6">' \
'<img src="/htdig/button7.gif" border="0" align="middle" width="30" height="30" alt="7">' \
'<img src="/htdig/button8.gif" border="0" align="middle" width="30" height="30" alt="8">' \
'<img src="/htdig/button9.gif" border="0" align="middle" width="30" height="30" alt="9">' \
'<img src="/htdig/button10.gif" border="0" align="middle" width="30" height="30" alt="10">'
#
# To make the current page stand out, we will put a border around the
# image for that page.
#
no_page_number_text: '<img src="/htdig/button1.gif" border="2" align="middle" width="30" height="30" alt="1">' \
'<img src="/htdig/button2.gif" border="2" align="middle" width="30" height="30" alt="2">' \
'<img src="/htdig/button3.gif" border="2" align="middle" width="30" height="30" alt="3">' \
'<img src="/htdig/button4.gif" border="2" align="middle" width="30" height="30" alt="4">' \
'<img src="/htdig/button5.gif" border="2" align="middle" width="30" height="30" alt="5">' \
'<img src="/htdig/button6.gif" border="2" align="middle" width="30" height="30" alt="6">' \
'<img src="/htdig/button7.gif" border="2" align="middle" width="30" height="30" alt="7">' \
'<img src="/htdig/button8.gif" border="2" align="middle" width="30" height="30" alt="8">' \
'<img src="/htdig/button9.gif" border="2" align="middle" width="30" height="30" alt="9">' \
'<img src="/htdig/button10.gif" border="2" align="middle" width="30" height="30" alt="10">'
</font>
</pre>
<hr noshade>
<h2>
<a name="search.html">${SEARCH_DIR}/search.html</a>
</h2>
<p>
This is the default search form. It is an example interface to
the search engine, htsearch. The file contains a form with as
its action a call to htsearch. There are several form variables
which htsearch will use. More about those can be found in the
<a href="htsearch.html" target="_top">htsearch
documentation</a>.
</p>
<p>
An example file can be as follows:
</p>
<pre>
<font size="-1">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>ht://Dig WWW Search</title>
</head>
<body bgcolor="#eef7ff">
<h1>
<a href="http://www.htdig.org"><IMG SRC="/htdig/htdig.gif" align="bottom" alt="ht://Dig" border="0"></a>
WWW Site Search</h1>
<hr noshade size="4">
This search will allow you to search the contents of
all the publicly available WWW documents at this site.
<br>
<p>
<form method="post" action="/cgi-bin/htsearch">
<font size="-1">
Match: <select name="method">
<option value="and">All
<option value="or">Any
<option value="boolean">Boolean
</select>
Format: <select name="format">
<option value="builtin-long">Long
<option value="builtin-short">Short
</select>
Sort by: <select name="sort">
<option value="score">Score
<option value="time">Time
<option value="title">Title
<option value="revscore">Reverse Score
<option value="revtime">Reverse Time
<option value="revtitle">Reverse Title
</select>
</font>
<input type="hidden" name="config" value="htdig">
<input type="hidden" name="restrict" value="">
<input type="hidden" name="exclude" value="">
<br>
Search:
<input type="text" size="30" name="words" value="">
<input type="submit" value="Search">
</form>
<hr noshade size="4">
</body>
</html>
</font>
</pre>
<hr noshade>
<h2>
<a name="header.html">${COMMON_DIR}/header.html</a>
</h2>
<p>
This file is the file that is output before any of the search
results are produced in a search. This file can be customized
to reflect your particular web look-and-feel, for example. Take
note that this file is only the top part of the full HTML
document that is produced when search results are displayed.
This means that it should start with the proper HTML
introductory tags and title.
</p>
<p>
This file will not just simply be copied. Instead, the search
engine will look for special variables inside the file. These
variables will be replaced with the appropriate values for
the particular search it is used for. For more details of the
use of these variables, consult the
<a href="hts_templates.html">htsearch templates</a> documentation.
</p>
<p>
Below is the default header.html file that gets installed.
Note that it contains a form to allow the user to refine the
search.
</p>
<pre>
<font size="-1">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html><head><title>Search results for '$&(WORDS)'</title></head>
<body bgcolor="#eef7ff">
<h2><img src="/htdig/htdig.gif" alt="ht://Dig">
Search results for '$&(LOGICAL_WORDS)'</h2>
<hr noshade size="4">
<form method="get" action="$(CGI)">
<font size="-1">
<input type="hidden" name="config" value="$&(CONFIG)">
<input type="hidden" name="restrict" value="$&(RESTRICT)">
<input type="hidden" name="exclude" value="$&(EXCLUDE)">
Match: $(METHOD)
Format: $(FORMAT)
Sort by: $(SORT)
<br>
Refine search:
<input type="text" size="30" name="words" value="$&(WORDS)">
<input type="submit" value="Search">
</font>
</form>
<hr noshade size="1">
<strong>Documents $(FIRSTDISPLAYED) - $(LASTDISPLAYED) of $(MATCHES) matches.
More <img src="/htdig/star.gif" alt="*">'s indicate a better match.
</strong>
<hr noshade size="1">
</font>
</pre>
<hr noshade>
<h2>
<a name="footer.html">${COMMON_DIR}/footer.html</a>
</h2>
<p>
This file is output after all the search results have been
displayed. All the same header.html rules apply to this file,
except that it is supposed to contain all the ending HTML tags.
</p>
<p>
Below is the default footer.html file that gets installed.
Note that it contains the page navigation stuff.
</p>
<pre>
<font size="-1">
$(PAGEHEADER)
$(PREVPAGE) $(PAGELIST) $(NEXTPAGE)
<hr noshade size="4">
<a href="http://www.htdig.org/">
<img src="/htdig/htdig.gif" border="0" alt="ht://Dig">ht://Dig $(VERSION)</a>
</body></html>
</font>
</pre>
<hr noshade>
<h2>
<a name="wrapper.html">${COMMON_DIR}/wrapper.html</a>
</h2>
<p>
This file may be used in place of the header.html and footer.html
files above. It is simply the concatenation of these two files,
with the pseudo-variable <strong>$(HTSEARCH_RESULTS)</strong> as
a separator for the header and footer sections.
All the same header.html and footer.html rules apply to this file.
To make this file override the header and footer files above, you
must define the <a href="attrs.html#search_results_wrapper">
search_results_wrapper</a> attribute.
</p>
<hr noshade>
<h2>
<a name="nomatch.html">${COMMON_DIR}/nomatch.html</a>
</h2>
<p>
If a search produces no matches, this file is displayed. All
the relevant variables will be replaced as in the header.html
and footer.html files. The default nomatch.html is little more
than header.html and footer.html appended:
</p>
<pre>
<font size="-1">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html><head><title>No match for '$&(LOGICAL_WORDS)'</title></head>
<body bgcolor="#eef7ff">
<h1><img src="/htdig/htdig.gif" alt="ht://Dig">
Search results</h1>
<hr noshade size="4">
<h2>No matches were found for '$&(LOGICAL_WORDS)'</h2>
<p>
Check the spelling of the search word(s) you used.
If the spelling is correct and you only used one word,
try using one or more similar search words with "<strong>Any</strong>."
</p><p>
If the spelling is correct and you used more than one
word with "<strong>Any</strong>," try using one or more similar search
words with "<strong>Any</strong>."</p><p>
If the spelling is correct and you used more than one
word with "<strong>All</strong>," try using one or more of the same words
with "<strong>Any</strong>."</p>
<hr noshade size="4">
<form method="get" action="$(CGI)">
<font size="-1">
<input type="hidden" name="config" value="$&(CONFIG)">
<input type="hidden" name="restrict" value="$&(RESTRICT)">
<input type="hidden" name="exclude" value="$&(EXCLUDE)">
Match: $(METHOD)
Format: $(FORMAT)
Sort by: $(SORT)
<br>
Refine search:
<input type="text" size="30" name="words" value="$&(WORDS)">
<input type="submit" value="Search">
</font>
</form>
<hr noshade size="4">
<a href="http://www.htdig.org/">
<img src="/htdig/htdig.gif" border="0" alt="ht://Dig">ht://Dig $(VERSION)</a>
</body></html>
</font>
</pre>
<hr noshade>
<h2>
<a name="syntax.html">${COMMON_DIR}/syntax.html</a>
</h2>
<p>
If a boolean expression search causes a syntax error, this file
will be displayed.
</p>
<pre>
<font size="-1">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html><head><title>Error in Boolean search for '$&(WORDS)'</title></head>
<body bgcolor="#eef7ff">
<h1><img src="/htdig/htdig.gif" alt="ht://Dig">
Error in Boolean search for '$&(LOGICAL_WORDS)'</h1>
<hr noshade size="4">
Boolean expressions need to be 'correct' in order for the search
system to use them.
The expression you entered has errors in it.<p>
Examples of correct expressions are: <strong>cat and dog</strong>, <strong>cat
not dog</strong>, <strong>cat or (dog not nose)</strong>.<br>Note that
the operator <strong>not</strong> has the meaning of 'without'.
<blockquote><strong>
$(SYNTAXERROR)
</strong></blockquote>
<hr noshade size="4">
<form method="get" action="$(CGI)">
<font size="-1">
<input type="hidden" name="config" value="$&(CONFIG)">
<input type="hidden" name="restrict" value="$&(RESTRICT)">
<input type="hidden" name="exclude" value="$&(EXCLUDE)">
Match: $(METHOD)
Format: $(FORMAT)
Sort: $(SORT)
<br>
Refine search:
<input type="text" size="30" name="words" value="$&(WORDS)">
<input type="submit" value="Search">
</font>
</form>
<hr noshade size="4">
<a href="http://www.htdig.org/">
<img src="/htdig/htdig.gif" border="0" alt="ht://Dig">ht://Dig $(VERSION)</a>
</body></html>
</font>
</pre>
<hr size="4" noshade>
Last modified: $Date: 2004/05/28 13:15:18 $
</body>
</html>
|
