We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent e998eb9 commit d01fd86Copy full SHA for d01fd86
Doc/library/urllib.robotparser.rst
@@ -18,7 +18,7 @@
18
This module provides a single class, :class:`RobotFileParser`, which answers
19
questions about whether or not a particular user agent can fetch a URL on the
20
website that published the :file:`robots.txt` file. For more details on the
21
-structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html.
+structure of :file:`robots.txt` files, see https://datatracker.ietf.org/doc/html/rfc9309.
22
23
24
.. class:: RobotFileParser(url='')
0 commit comments