diff options
| author | Georg Brandl <georg@python.org> | 2007-08-14 20:06:25 +0000 |
|---|---|---|
| committer | Georg Brandl <georg@python.org> | 2007-08-14 20:06:25 +0000 |
| commit | 690ea9f5111fc4c32f6fed590e955b06b9095cc7 (patch) | |
| tree | 95d6cf05af4c56f3c66c5b4488294698be0be0f6 /sphinx/web/robots.py | |
| parent | f61b28a73d488aa2dd9d3aaa93c8c90893dac897 (diff) | |
| download | sphinx-git-690ea9f5111fc4c32f6fed590e955b06b9095cc7.tar.gz | |
Add a robots.py to the web app.
Diffstat (limited to 'sphinx/web/robots.py')
| -rw-r--r-- | sphinx/web/robots.py | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/sphinx/web/robots.py b/sphinx/web/robots.py new file mode 100644 index 000000000..857dc2fc4 --- /dev/null +++ b/sphinx/web/robots.py @@ -0,0 +1,28 @@ +# -*- coding: utf-8 -*- +""" + sphinx.web.robots + ~~~~~~~~~~~~~~~~~ + + robots.txt + + :copyright: 2007 by Georg Brandl. + :license: Python license. +""" + +robots_txt = """\ +User-agent: * +Disallow: /@source/ +Disallow: /@edit/ +Disallow: /@submit/ +Disallow: /@comments/ +Disallow: /@rss/ +Disallow: /@admin + +User-agent: Googlebot +Disallow: /@source/ +Disallow: /@edit/ +Disallow: /@submit/ +Disallow: /@comments/ +Disallow: /@rss/ +Disallow: /@admin +""" |