GUIDE / STRAIGHT ANSWER
Can ChatGPT search crawl my website?
Check the crawler, the specific page and the server response. Search access, user-requested visits and model training have different controls.
The short answer
OpenAI documents OAI-SearchBot for ChatGPT search, GPTBot for content that may be used in model training and ChatGPT-User for certain user-initiated visits. Review robots.txt and the site’s actual access controls separately. Allowing search is not the same as opting into training, and neither guarantees an appearance.
A practical Hypertrail guide. Worked examples are not client results unless a case study is explicitly linked. Last updated 23 October 2026.
Why Hypertrail is founder-led →What to check first
Use these checks to identify the problem before buying work or changing the setup.
- ✓The relevant rules in robots.txt
- ✓A successful public page response
- ✓Visible service content without a login
- ✓Hosting or firewall restrictions checked independently
Read the rule for the right purpose
A training preference should not be silently reversed to pursue search visibility. Record the current settings and agree any change. OpenAI’s official crawler documentation is the source of truth for its user agents and published network ranges; do not trust a random list copied into an old blog.
Test a service page, not only the homepage
Check that the URL resolves to the right content with a successful response and a sensible canonical. A robots allowance does not fix a login wall, an error page, an empty response or a host-level block. A header claiming a bot identity also does not prove that the real provider can reach the site from its network.
Keep private areas private
Do not expose account pages, customer records or admin tools to make a site “AI friendly”. Public service descriptions and approved project evidence are enough for the discovery journey. Search accessibility should not become a reason to weaken access controls.
Treat access as one checkpoint
Record what was checked and the limits. A public-response test verifies that request, not every future crawl. Search inclusion, source selection and recommendations remain outside the website owner’s direct control.
Apply this to your business
Tell Graeme what you sell, where customers are and where the journey breaks. The next step is a scoped conversation about ai search visibility and chatgpt discoverability, not an automatic package of unrelated services.
Sources checkedPrimary platform documentation (1)+
Continue with the next useful question
Understand the decision—not just this one answer.
These guides cover the closest decisions that normally affect the same campaign, page or budget.
Browse all practical guides →Common questions
Short, honest answers.
Does allowing GPTBot make ChatGPT recommend me?+
No. GPTBot’s documented purpose concerns possible training use, not a guaranteed commercial recommendation.
Does an llms.txt file override robots.txt?+
Do not rely on it to override access rules. Review the documented crawler controls and actual hosting behaviour.