mirror of
https://github.com/python/cpython.git
synced 2026-06-05 01:10:53 +00:00
* empty lines are always ignored instead of separating groups * the "user-agent" line after a rule starts a new group * groups matching the same user agent are now merged * the rule with the longest match wins instead of the first matching rule * in case of equal matches, the “Allow” rule wins over “Disallow” * special characters “$” and “*” are now supported in rules * prefer full match for user agent |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| error.py | ||
| parse.py | ||
| request.py | ||
| response.py | ||
| robotparser.py | ||