Ideas to solve FoonRIP 0.72 problem of concatenated fields (no separate surname,
initials,companu,prefix etc but a single surname field):

1.  Extract initials, surname, company prefix etc from the single surname field 
    with some prog. So basically making the data fit the old FoonRIP <0.71 db
    format. 
    Pro: doesn't just work for MySQl but for other database/plaintext too.
    Con: prolly pretty hard to do this perfectly. Faults can lead to certain
    records not being able to be found.
    
2.  Use MySQL's fulltext search option. This means MySQL creates a searchable 
    index of all words in the surname fields.
    Pro: pretty easy, you let MySQL work for 12 hours and you have the index
    Con: MySQL doesn't index word shorter than 4 characters. So this option 
    also leads to record that can't be found (surname of 3 letters or less for
    instance). MySQL also ignores words from a stop word list. Both that list 
    and the 4 char limit can be changed but only at compile time (it's on 
    MySQL's TODO list to make it changeable at runtime). 
    Also this only works for MySQL on not for other DB/plaintext.
    
3.  Implement our own fulltext search option. Basically make a table of unique
    words that exist in all surname fields combined and make a second list which
    word exists in which surname field. This way we have control over the 
    stopwords (like well known prefix and maybe a list of possible firstnames)
    and over the character limit and we can introduce other specs (like 
    different characters that are word separators).
    Pro: No record will be skipped as you can create the word table on the safe
    side. Same goes for the character limit. Though a low char limit will 
    dramatically increase the amount of entries in the word table.
    Con: Only really works for a database (too much work to effectively do in 
    plaintext format). Takes lots of work/thinking to set up properly.
    Speed, this might be the slowest solution especially in case of a low char
    limit.
    
$Id: fr072-problem-ideas.txt,v 1.1 2002/03/03 01:24:54 M Boerwinkel Exp $