I have been working on a tool to access the sitemap of a news website to try to create feeds when the site does not provide feeds. The tool starts by reading the robots.txt file, which usually has a list of things that are “disallowed”. On one of the sites, it specifically included “Claude Code” in the “disallow” list. Claude told me that it could not run the tool, since it was on the disallow list, but I could run it, capture the output, then give it to Claude for further processing. This was the first time I have seen any “limit” imposed on Claude by itself – interesting!
Claude
There is one post filed in Claude.