A ChatGPT crawler access check you can run today
Before any outreach or content work, confirm that OpenAI's crawler can read your site. This takes about fifteen minutes and needs no special tools.
- Open your robots.txt. Type your domain followed by /robots.txt. Under the Robots Exclusion Protocol standard, a crawler follows the group that matches its name and otherwise falls back to the group marked with an asterisk. A blanket Disallow there covers OpenAI's bots too.
- Find OAI-SearchBot. OpenAI's crawler documentation says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links.
- Decide on GPTBot separately. GPTBot relates to training OpenAI's models. OpenAI says each setting is independent, so you can allow search and disallow training.
- Do not lean on ChatGPT-User rules. That agent acts when a user asks ChatGPT to visit a page. OpenAI says it does not decide whether content appears in search.
- Check your firewall. Security plugins and hosting firewalls can block bots even when robots.txt allows them. OpenAI publishes the IP addresses each crawler uses so your host can let them through.
- Wait a day, then retest. OpenAI notes it can take about 24 hours for its systems to adjust after a robots.txt update.
- Ask a local question and read the sources. OpenAI's web search documentation says the model decides whether to search based on the prompt, can use an approximate location such as a city, and answers with sourced citations. Note which sites it cites for your trade.
Passing this check only makes you eligible. The sites ChatGPT cites are usually where the work is, which is why listings management sits next to this service. See also how to get recommended by ChatGPT.
