Description of the Structure of the Data needed by MyThes
--------------------------------------------------------

MyThes is very simple.  Almost all of the "smarts" are really
in the thesaurus data file itself.

The format for this file is at follows:

- no binary data 

- line ending is a newline '\n' and not carriage return/linefeeds

- Line 1 is a character string that describes the encoding
used for the file.  It is up to the calling program to convert
to and from this encoding if necessary.

     ISO8859-1 is used by the th_en_US_new.dat file.

     Strings currently recognized by OpenOffice.org are:

     UTF-8
     ISO8859-1
     ISO8859-2
     ISO8859-3
     ISO8859-4
     ISO8859-5
     ISO8859-6
     ISO8859-7
     ISO8859-8
     ISO8859-9
     ISO8859-10
     KOI8-R
     CP-1251
     ISO8859-14
     ISCII-DEVANAGARI


- All of the remaning lines of the file follow this structure

entry|num_mean
pos|syn1_mean|syn2|...
.
.
.
pos|mean_syn1|syn2|...


where:

   entry      - all lowercase version of the word or phrase being described
   num_mean   - number of meanings for this entry

   There is one meaning per line and each meaning is comprised of

   pos        -  part of speech or other meaning specific description
   syn1_mean  -  synonym 1 also used to describe the meaning itself 
   syn2       - synonym 2 for that meaning etc.


To make this even more clearer, here is actual data for the
entry "simple".

simple|9
(adj)|simple |elemental|ultimate|oversimplified|simplistic|simplex|simplified|unanalyzable|
undecomposable|uncomplicated|unsophisticated|easy|plain|unsubdivided
(adj)|elementary|uncomplicated|unproblematic|easy
(adj)|bare|mere|plain
(adj)|childlike|wide-eyed|dewy-eyed|naive |naif
(adj)|dim-witted|half-witted|simple-minded|retarded
(adj)|simple |unsubdivided|unlobed|smooth
(adj)|plain
(noun)|herb|herbaceous plant
(noun)|simpleton|person|individual|someone|somebody|mortal|human|soul


It says that "simple" has 9 different meanings and each 
meaning will have its part of speech and at least 1 synonym 
with other if presetn following on the same line.



Once you ahve created your own structured text file you can use
the perl program "th_gen_idx.pl" which can be found in this
directory to create an index file that is used to seek into
your data file by the MyThes code.

The correct way to run the perl program is as follows:

cat th_en_US_new.dat | ./th_gen_idx.pl > th_en_US_new.idx



Then if you head the resulting index file you should see the 
following:

ISO8859-1
142689
'hood|10
's gravenhage|88
'tween|173
'tween decks|196
.22|231
.22 caliber|319
.22 calibre|365
.38 caliber|411
.38 calibre|457
.45 caliber|503
.45 calibre|549
0|595
1|666
1 chronicles|6283
1 esdras|6336


Line 1 is the same encoding string taken from the 
structured thesaurus data file.

Line 2 is a count of the total number of entries
in your thesaurus.

All of the remaining lines are of the form

entry|byte_offset_into_data_file_where_entry_is_found


That's all there is too it.


Kevin
kevin.hendricks@sympatico.ca