# ClaudeBot

Anthropic crawler, purpose: training.

The ClaudeBot record on Rattlesnakes By Mail is the full published file on ClaudeBot, a crawler from Anthropic. Each fact on the ClaudeBot record is a dated claim with a verbatim quote from Anthropic, a source URL, a verification method and a confidence. Sections run in a fixed order: Identity, Claims, Observations, Changes, Questions, Notes.

## Identity

vendor: Anthropic
kind: crawler
purpose: training
ua_token: ClaudeBot
robots_token: ClaudeBot
ip_list_url: https://claude.com/crawling/bots.json
docs_url: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-content-from-the-web-and-how-can-site-owners-block-the-crawler
status: active

Collects web content that may contribute to Anthropic model training.

## Claims

| Field | Value | Method | Confidence |
| --- | --- | --- | --- |
| observed_fetches_json | yes | observed_here | high |
| observed_fetches_markdown | yes | observed_here | high |
| observed_first_request_path | /sitemap.xml | observed_here | high |
| observed_format_pass_order | html_json_md | observed_here | high |
| anti_circumvention_stance | documented | vendor_doc | high |
| honours_crawl_delay | yes | vendor_doc | high |
| publishes_ip_list | yes | vendor_doc | high |
| purpose_documented | training | vendor_doc | high |
| respects_robots_txt | yes | vendor_doc | high |
| separate_search_and_training_tokens | yes | vendor_doc | high |
| user_agent_string_full | not_documented | vendor_doc | medium |
| verifiable_by_rdns | not_documented | vendor_doc | n/a |

## Observations

requests, last 30 days: 1142
verified requests, last 30 days: 1140
verified share, last 30 days: 100%
daily rows recorded: 10

## Changes

| Date | Claim | Kind | Old value | New value |
| --- | --- | --- | --- | --- |
| 2026-09-17 | observed_fetches_markdown | new |  | yes |
| 2026-09-17 | observed_fetches_json | new |  | yes |
| 2026-09-17 | observed_first_request_path | new |  | /sitemap.xml |
| 2026-09-17 | observed_format_pass_order | new |  | html_json_md |
| 2026-09-13 | respects_robots_txt | new |  | yes |
| 2026-09-13 | honours_crawl_delay | new |  | yes |
| 2026-09-13 | publishes_ip_list | new |  | yes |
| 2026-09-13 | verifiable_by_rdns | new |  | not_documented |
| 2026-09-13 | purpose_documented | new |  | training |
| 2026-09-13 | separate_search_and_training_tokens | new |  | yes |
| 2026-09-13 | anti_circumvention_stance | new |  | documented |
| 2026-09-13 | user_agent_string_full | new |  | not_documented |

## Notes

No published notes for this entity yet.

## Compare with

- ClaudeBot vs Claude-SearchBot
- ClaudeBot vs Claude-User
- ClaudeBot vs Applebot
- ClaudeBot vs Applebot-Extended
- ClaudeBot vs Google-Extended
- ClaudeBot vs GoogleOther
- ClaudeBot vs Googlebot
- ClaudeBot vs Bingbot
- ClaudeBot vs ChatGPT-User
- ClaudeBot vs GPTBot
- ClaudeBot vs OAI-AdsBot
- ClaudeBot vs OAI-SearchBot
- ClaudeBot vs Perplexity-User
- ClaudeBot vs PerplexityBot

Every claimed field for ClaudeBot is documented at /fields.

Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.