Generating the search index

The search system is based on the ht://dig search engine. It uses a search index that is generated by following all the links on a website, downloading each page, scanning for words, and following any links on that page to other pages. A configuration file controls the extent of link following so only documentation files on the server system are indexed.

The search index is not generated during the installation of DocView. The indexing process can take considerable time and system resources, depending on how much doc is installed and how fast the machine is. It is best to run the indexing process when the machine is relatively inactive.

The search index can be 80 megabytes and larger. During the sorting process, it generates temporary files that can double that. Be sure you have sufficient space available under /var/lib/docview where the index files are stored. If you have enough disk space for the index but not enough for the temporary files, you can set the TMPDIR environment variable to a temporary directory on another larger filesystem. That directory should already exist.

The basic command for generating the search index is:

   /usr/lib/docview/conf/rundig

• The DocView server must be running for indexing to take place.

• You can try running the command as a cron job. If that fails, run it directly from the command line. The cron environment may limit resources that prevent the indexing process from completing properly for large doc collections.

• The first time rundig is run, it will generate a configuration file tailored for the system, and then run the collection process. That can take several hours, depending on the speed and load on the machine. When it completes, you should be able to use the Search system in DocView.

• The rundig command has a few options:

     /usr/lib/docview/conf/rundig [-i] [-a] [-s] [-v|-d]
  

  
  -i

  Build a complete new index, not an update. This takes longer, but ensures correct results. It is necessary when documents are removed from the system, because an incremental indexing process will not remove documents.

  
  -a

  Keep old files while indexing. This options lets you keep using the old search index while the new one is generated. It requires lots of disk space.

  
  -s

  Print statistics after completion.

  
  -v or -d

  Verbose output (-d more verbose). These options display information about each document being indexed. They are useful for debugging purposes. One word of caution: the -d option can generate hundreds of megabytes of information.

• If you reindex often and have sufficient disk space, you can speed up reindexing by commenting out a line in the rundig script that removes the db.wordlist file upon completion. That large file takes a lot of space but makes incremental indexing jobs run faster. If you want to keep that file, use an ordinary text editor on the rundig script file to locate and comment out the rm command.

• DocView generates one search index that includes documentation in all languages to which it has access on the system.