Telling robots not to touch /config/ - if there's a JunkBuster

proxy between the robot and the web server, then accessing
/config can affect the proxy.

diff --git a/doc/webserver/robots.txt b/doc/webserver/robots.txt
new file mode 100644 (file)
index 0000000..790f3ff
--- /dev/null
+++ b/doc/webserver/robots.txt
@@ -0,0 +1,15 @@
+# This is the Internet Junkbuster web site.
+#
+# If the robot is connecting through a JunkBuster, then the
+# control interface is at /config.   It isn't very useful
+# to index it, and you're likely to break stuff.  So go away!
+#
+# Even if you're not connected through a JunkBuster, the 
+# one "JunkBuster's not working" page there is not very
+# interesting.
+
+User-agent: *
+Disallow: /config/
+
+
+