Agent Grade / AI bots by site / jstor.org
Does jstor.org block ChatGPT?
jstor.org blocks AI training but lets AI search in. ChatGPT search is allowed, and OpenAI's training bot is blocked. From its public robots.txt, checked 23 to 24 October 2026. It ranks #1,283 on the Tranco list of top sites.
| Bot | What it does | Home page | Deciding rule |
|---|---|---|---|
| Googlebot | Google Search | Allowed | no rule blocks it |
| Bingbot | Bing / Copilot | Allowed | no rule blocks it |
| OAI-SearchBot | ChatGPT search | Allowed | no rule blocks it |
| ChatGPT-User | ChatGPT (user asks) | Allowed | no rule blocks it |
| GPTBot | OpenAI training | Blocked | User-agent: GPTBot / Disallow: / |
| Claude-SearchBot | Claude search | Allowed | no rule blocks it |
| Claude-User | Claude (user asks) | Allowed | no rule blocks it |
| ClaudeBot | Anthropic training | Blocked | User-agent: ClaudeBot / Disallow: / |
| PerplexityBot | Perplexity search | Allowed | no rule blocks it |
| Perplexity-User | Perplexity (user asks) | Allowed | no rule blocks it |
| Google-Extended | Gemini training | Blocked | User-agent: Google-Extended / Disallow: / |
| Applebot-Extended | Apple AI training | Allowed | no rule blocks it |
| CCBot | Common Crawl | Blocked | User-agent: CCBot / Disallow: / |
| meta-externalagent | Meta AI | Allowed | no rule blocks it |
| Bytespider | ByteDance | Allowed | no rule blocks it |
"Blanket rule": jstor.org never names that bot; a rule for every bot (User-agent: *) applies. llms.txt: No. robots.txt is what a site asks bots to do; some sites also block bots at the network edge.
Questions
Does jstor.org block ChatGPT search?
No. In its robots.txt as checked 23 to 24 October 2026, jstor.org lets OAI-SearchBot, the crawler behind ChatGPT search, read its home page.
Does jstor.org block GPTBot?
Yes. jstor.org blocks GPTBot, OpenAI's training crawler (User-agent: GPTBot / Disallow: /).
Does jstor.org block Claude?
ClaudeBot (Anthropic training): blocked. Claude-SearchBot (Claude search): allowed. Claude-User (fetches for a person): allowed.
Does jstor.org have an llms.txt file?
We did not find an llms.txt file on jstor.org, or its robots.txt did not allow us to fetch it.
What about your site?
Agent Grade checks the same 15 bots against your robots.txt, loads your page as each AI bot, and checks what attack agents look for first. Free, under a minute.
Grade my site