Stuff that needs to be done/implemented in no particular order:
(*=needs to be done, ?=describes something to researched/discussed before implementation,
-=done (or not if it was a ? ;)) )

?    output some no-cache headers in search.php. On second thought, cache might be interesting as long as
    search.php?surname=a and search.php?surname=b have different cache entries.
    Check cache tutorial for this: http://www.mnot.net/cache_docs/. => added an "Expires:" header, doesn't seem
    to work on browser level caching, haven't yet checked proxy level. EDIT: works now I've updated to mozilla 0.9.8
    I usually use the following code snippet but that doesn't work on my devel server atm (no IF_MODIFIED_SINCE env var):
    <?php
    DEFINE("LAST_MOD_TIME",<last modification time, you can also get this dynamically from db for instance>);
    if ($HTTP_IF_MODIFIED_SINCE == LAST_MOD_TIME) {
            header("HTTP/1.1 304 Not Modified");
            exit;
          }
          /* Cache-Control only really necessary for short lived pages, for longe lived pages caches are usually allowed to
          take some liberties. On hourly changeing pages a minute difference may be important, on half a year not.
          header("Cache-Control: must-revalidate");
          header("Last-Modified: " . LAST_MOD_TIME);
    ?>

-    The results in search.php might needs to be htlmspecialchars()-ed before being printed.

*    partial searches with wildcards (atm only exact searches are supported to keep search time short)

-    parse the postalcode mp_search recieves so it will correspond to the db format (i.e. remove spaces from pc
    and convert to (upper/lower)case

-    search on subscriberid so you can easily search for a single record.

-    some form of ordering system => mp_search class now accepts an extra variable that contains which field
    there should be ordered on. Possible: PHONE,NAME,STREET,POSTALCODE,TOWN

?    look at database contents and see if we can make the db searches case sensitive ("WHERE BINARY") as these
    are reportedly faster (also if the db contents is all lowercase already?).

?    Some1 told me PACK_KEYS=1 in table definition could reduce index size and improve search speed (at the cost
    of update speed), we need to look into this.

?    Make the subscriber and/or other tables compressed instead of dynamic(/static). Apparently it
    improves speed and decreases size at the cost of the table being read-only (personally I only change the data 
    every once in a while. This can be done with the myisamchk utility.

?    Add somesort of info for TownId == 0 and StreetId == 0 into street and town tables so LEFT (OUTER) JOINS
    become obsolete (should result in overall performance increase or at least improve certain queries that atm
    still result in full table scans.

?    Atm some columns still are defined as NULL. According to MySQL manual NOT NULL speeds things up and
    uses up less space.
    FoonRIP compatability: As I import the data from txt files my DB never contains NULL so changing to NOT NULL
    doesn't matter. Does FoonRIP's direct import also never use NULL? If so what are the pros/cons for changing
    that to NOT NULL. This is also important as older MySQL versions don't do indices on NULL columns (side-note:
    Why does my 3.23.48 b0rk on index on NULL columns, MySQL manual says as of 3.23.2 NULL index is supported).

?    handle override fields. How? => on extended record page override fields now take precedence, but how should we
    handle this when searching.

?    MySQL 4.0 has new SELECT feature: SQL_CALC_FOUND_ROWS. Might be interesting to see if this can be used to
    see how many rows the search actually finds disregarding LIMIT. Has the drawback off reduced performance if
    you have a really big result set.

?    Atm I've tried to create such indices no search requires a full table scan and therefore a lot of time.
    I've also tried to restrict to type of queries, that is you can't search on only a subset of the row fields and
    also range searches aren't possible. Can't we turn this around, aka isn't there a way to "allow" every query
    but let MySQL automaticly drop any that would allow a full table scan?

-    Using this script searching with only town = rotterdam as parameter will take a huge amount of time. Why?
    apeldoorn and amsterdam work. => fixed, was a typo in a variable which resulted in LEFT instead of INNER JOIN.

*    part of the code makes assumptions about magic_quotes (I personally always try to turn them off and program
    as if they're not on as SQL injection is too dangerous to take chances with). This needs to be changed.

-    make plans for the clientlib classes API. Atm most of this TODO is about the structural idea/MySQL database.
    Also the API atm is pretty messy. => like I will ever get to that.

-    create an XML-RPC server and accompanying client. Atm I see 2 options: 1. PHP's own API for XML-RPC, it's new
    in version 4.1.0 sos should be stable by the time I get round to this TODO item :). 2. third party interface like
    http://phpxmlrpc.sourceforge.net/. => option 2 implemented.

?    some kind of text export of the search results.

-    Goals for RELEASE_0_1 :
    -    institute GPL license
    -    write README
    -    fix includes/requires, especially the ones for xmlrpc library
    -    Some small coding style changes (mostly following PEAR docs):
            - ident with 4 space instead of tab (not 100% sure but try it anyways) => looks good, now the same look in both ultraedit and gvim (as long as you use a fixed-width font :) )
            - { after function and class definitions on a new line, not for control structs like if, while etc.
            - comments about class definitions and functions in (block) comment above it instead of inside.
    -    see if xmlserver.php can handle zero results in a different way than an error (consider only minor changes for now). => seems to work but dunno if it's still with in XML-RPC spec.
    -    add some sql "scripts" to update the standard FoonRIP mysql db, to the mp_search version
         (don't forget to change the BIGINTs in subscriber).
         => new change 1 made to db: SubscriberId in info and subscriber tables changed to int instead of bigint.
            bigint is uselessly large, MAX(SubscriberId) < 7 million. BIGINT only takes up more space.
         => new change 2 made to db: HouseNumber in subscriber table changed to smallint instead of bigint,
            just like SubscriberId, just takes up space.
         => new change 3 made to db: TownId, StreetId changed to smallint instead of bigint.
            Apart from the useless extra space bigint uses bigint would also degrade performance. TownId and StreetId in
            town and street tables are smallint meaning MySQL has to make a translation when comparing the TownId and
            StreetId from the different tables.
         => changes already made:
                INDEX idx_town ON town (Town)
                INDEX idx_townid ON townname (TownId)
                INDEX idx_surname ON subscriber (Surname(25))
                INDEX idx_townstreetid ON subscriber (TownId,StreetId)
                INDEX idx_px ON subscriber (PostalCode)

-   rewrite README so it will actually make sense. => it does make sense (I hope)

?   FoonRIP 0.72 compatability: in case the new db format doesn't change we can always do a FULLTEXT index on surname field and change
    the SQL query accordingly. => see doc/fr072-problem-ideas.txt for possible solutions.

?   Idea (the longer I think about it the less sense it makes, but I'll write it down anyways ): replace index on
    single column postalcode with multi-index (postalcode,townid,streetid).
    Does take up lots more space but can't really have negative effect as MySQL can use partial index as long as it's
    a leftmost prefix. Positive effect in case of a search on postalcode is performed but also on TownId/Streetid
    (that last is always done if you JOIN subscriber with town or street table). The same might go for column like
    surname, definately not for phone as phone's entries are unique.

$Id: TODO,v 1.28 2002/03/14 22:09:03 M Boerwinkel Exp $
