Stuff that needs to be done/implemented in no particular order:
(*=needs to be done, ?=describes something to researched/discussed before implementation, -=done)

?    look at database contents and see if we can make the db searches case sensitive ("WHERE BINARY") as these
    are reportedly faster (also if the db contents is all lowercase already?).=> lots has changed in the foondump
    database, case sensitivity now probably matters as database contains both upper and lower case.

?    Some1 told me PACK_KEYS=1 in table definition could reduce index size and improve search speed (at the cost
    of update speed), we need to look into this.

?    Make the subscriber and/or other tables compressed instead of dynamic(/static). Apparently it
    improves speed and decreases size at the cost of the table being read-only (personally I only change the data 
    every once in a while. This can be done with the myisamchk utility.)

?    Atm some columns are defined as NULL. According to MySQL manual NOT NULL speeds things up and
    uses up less space. Need to check if a. one or more rows actually use NULL b. NULL is used during importing.

-    MySQL 4.0 has new SELECT feature: SQL_CALC_FOUND_ROWS. Might be interesting to see if this can be used to
    see how many rows the search actually finds disregarding LIMIT. Has the drawback off reduced performance if
    you have a really big result set.

?    Atm I've tried to create such indices no search requires a full table scan and therefore a lot of time.
    I've also tried to restrict to type of queries, that is you can't search on only a subset of the row fields and
    also range searches aren't possible. Can't we turn this around, aka isn't there a way to "allow" every query
    but let MySQL automaticly drop any that would allow a full table scan?

?   Idea (the longer I think about it the less sense it makes, but I'll write it down anyways ): replace index on
    single column postalcode with multi-index (postalcode,townid,streetid).
    Does take up lots more space but can't really have negative effect as MySQL can use partial index as long as it's
    a leftmost prefix. Positive effect in case of a search on postalcode is performed but also on TownId/Streetid
    (that last is always done if you JOIN subscriber with town or street table). The same might go for column like
    surname, definately not for phone as phone's entries are unique.
    
?   test speed-increase when switching to static rows instead of dynamic rows. => not tested yet but not much increase visible (yet)
    (add both static and dynamic definitions) => bit overkill, just replace all varchar(x) to char(x) in new-db-def.sql
    
?   think of some sensible way to test speed of static rows => thought up the way (super-smack), but not executed yet :x

?   custom error handling for search.php and extrecord.php -> handling of errors has improved, but not custom handling real PHP errors.    
    
*   update all inline docs for pink search related files

*   solution for "unlimited" record length and limited screen width (especially pink results)

*   check and fix XML-RPC ()

*   optimize foondump database for MPS queries.

*   see if the town and town_street tables are required, could possibly be done with decent indices on subscriber as well.

$Id: TODO 432 2004-11-18 23:25:36Z martijn $
