GPTBot
OpenAI crawler, purpose: training.
The GPTBot record on Rattlesnakes By Mail is the full published file on GPTBot, a crawler from OpenAI. Each fact on the GPTBot record is a dated claim with a verbatim quote from OpenAI, a source URL, a verification method and a confidence. Sections run in a fixed order: Identity, Claims, Observations, Changes, Questions, Notes.
Identity
- vendor
- OpenAI
- kind
- crawler
- purpose
- training
- ua_token
GPTBot- robots_token
GPTBot- ip_list_url
- https://openai.com/gptbot.json
- docs_url
- https://platform.openai.com/docs/bots
- status
- active
Crawls public web content for OpenAI model training.
Claims
| Field | Value | Method | Confidence |
|---|---|---|---|
| honours_crawl_delay | not_documented | vendor_doc | n/a |
| publishes_ip_list | yes | vendor_doc | high |
| purpose_documented | training | vendor_doc | high |
| respects_robots_txt | yes | vendor_doc | high |
| robots_txt_fetch_marker | yes | vendor_doc | high |
| separate_search_and_training_tokens | yes | vendor_doc | high |
| user_agent_string_full | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot | vendor_doc | high |
| verifiable_by_rdns | not_documented | vendor_doc | n/a |
Observations
- requests, last 30 days
- 1109
- verified requests, last 30 days
- 1105
- verified share, last 30 days
- 100%
- daily rows recorded
- 5
Changes
| Date | Claim | Kind | Old value | New value |
|---|---|---|---|---|
| 2026-09-13 | respects_robots_txt | new | yes | |
| 2026-09-13 | honours_crawl_delay | new | not_documented | |
| 2026-09-13 | publishes_ip_list | new | yes | |
| 2026-09-13 | verifiable_by_rdns | new | not_documented | |
| 2026-09-13 | purpose_documented | new | training | |
| 2026-09-13 | separate_search_and_training_tokens | new | yes | |
| 2026-09-13 | user_agent_string_full | new | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot | |
| 2026-09-13 | robots_txt_fetch_marker | new | yes |
Notes
No published notes for this entity yet.
Compare with
- GPTBot vs Claude-SearchBot
- GPTBot vs Claude-User
- GPTBot vs ClaudeBot
- GPTBot vs Applebot
- GPTBot vs Applebot-Extended
- GPTBot vs Google-Extended
- GPTBot vs GoogleOther
- GPTBot vs Googlebot
- GPTBot vs Bingbot
- GPTBot vs ChatGPT-User
- GPTBot vs OAI-AdsBot
- GPTBot vs OAI-SearchBot
- GPTBot vs Perplexity-User
- GPTBot vs PerplexityBot
Every claimed field for GPTBot is documented at /fields.
Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.