deblan/journalduhacker
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

block archivers

Browse source
This commit is contained in:
joshua stein 2012-10-24 11:53:21 -05:00
parent 5ee6c227a3
commit dc71bbbd98
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
public/robots.txt
View file

@ -0,0 +1,5 @@
# no caching
User-agent: ia_archiver
Disallow: /
User-agent: archive.org_bot
Disallow: /