Skip to content

gh-79638: Treat an unreachable robots.txt as "disallow all" (GH-138555)#149541

Closed
serhiy-storchaka wants to merge 1 commit intopython:mainfrom
serhiy-storchaka:robotparser-unreachable2
Closed

gh-79638: Treat an unreachable robots.txt as "disallow all" (GH-138555)#149541
serhiy-storchaka wants to merge 1 commit intopython:mainfrom
serhiy-storchaka:robotparser-unreachable2

Commits

Commits on May 8, 2026