Handling of Robots

Suggestion: If the CGI detects a robot by User-Agent (or a request for robots.txt?) then it should produce a total index of messages of some form with links to the message view page for each image. Allow the robot to spider this index page and each message and it should see all content whilst not really returning the index page to searchers. Some text in this page can be used to lead the user to the real index page.