Show HN: Fetchcheck – send a signed crawler to any site and see if it gets in
By zeppelin_7 · 2026-08-03 · 2 points · 0 comments
https://sitedex.dev/tools/fetchcheck
If you were ever curious whether a site allows signed crawlers, you can use this tool. This tool respects the website's robots.txt directive. You will be surprised how many sites allow all robots, but then block crawlers
Open the full discussion on BetterNews