summaryrefslogtreecommitdiff
path: root/Lib/urllib/robotparser.py
Commit message (Expand)AuthorAgeFilesLines
* Issue #25400: RobotFileParser now correctly returns default values for crawl_...Berker Peksag2016-09-181-2/+6
* Merge typo fixes from 3.5Martin Panter2016-05-081-2/+37
|\
| * Issue #16099: RobotFileParser now supports Crawl-delay and Request-rateBerker Peksag2015-10-081-2/+37
* | Fix typos in comments, documentation and test method namesMartin Panter2016-05-081-1/+1
|/
* Issue 21469: Minor code modernization (convert and/or expression to an if/el...Raymond Hettinger2014-05-121-1/+1
* Issue 21469: Mitigate risk of false positives with robotparser.Raymond Hettinger2014-05-121-2/+9
* #17403: urllib.parse.robotparser normalizes the urls before adding to ruleline.Senthil Kumaran2013-05-291-0/+1
* Merged revisions 83238 via svnmerge fromGeorg Brandl2010-08-011-2/+4
* Merged revisions 83209 via svnmerge fromSenthil Kumaran2010-07-281-2/+4
* Merged revisions 65209-65216,65225-65226,65233,65239,65246-65247,65255-65256 ...Benjamin Peterson2008-07-311-0/+5
* Bug 3347: robotparser failed because it didn't convert bytes to string.Jeremy Hylton2008-07-181-2/+6
* Make a new urllib package .Jeremy Hylton2008-06-181-0/+191