=== AI Crawler Audit ===
Contributors: agentforgepro
Tags: seo, llms-txt, structured-data, robots-txt, ai-search
Requires at least: 6.0
Tested up to: 7.1
Requires PHP: 7.4
Stable tag: 1.0.0
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Checks whether ChatGPT, Perplexity, Claude and Copilot can actually reach and read your site. Every finding shows the measurement behind it.

== Description ==

Most tools that claim to check "AI visibility" read your robots.txt and stop there. That misses a
failure they cannot see at all.

Your robots.txt can say a crawler is welcome while your CDN, your host or a security plugin quietly
returns 403 to that same crawler. Nothing warns you. The site looks perfectly configured, your pages
rank in Google, and the crawlers behind ChatGPT and Perplexity have never successfully read a page.

This plugin checks both, from your own server:

* **What robots.txt says** — a per-crawler verdict with the exact rule line that produced it, so you
  can check the answer rather than trust it.
* **What your server actually does** — it requests your home page once as each crawler, and once as
  an ordinary browser as a control. If the browser gets 200 and a crawler gets 403 in the same minute
  from the same server, something between your site and that crawler is blocking it, and it is not
  robots.txt.

It also checks the things that decide whether a page is usable once a crawler does get in:

* `llms.txt` — present, and genuinely plain text rather than an HTML 404 pretending to be a file
* Sitemap — reachable, non-empty, and declared in robots.txt
* Per page, on your home page and your four most recent published posts or pages: title, meta
  description, canonical, `noindex`, heading structure, JSON-LD blocks and whether they parse, and how
  much readable text survives with JavaScript switched off

Findings are ordered worst first, and each one carries the raw measurement and a specific fix. There
is a plain-text report at the bottom you can copy and hand to whoever maintains the site.

**Everything runs on your own server.** The plugin requests your own pages and nothing else. It sends
no data to the author or to any third party, has no telemetry, no account, no API key and no external
service. It does nothing at all on the front end of your site and nothing on a schedule — it runs when
an administrator presses the button.

The whole plugin is free and every feature is included. There is no pro version, no upgrade prompt and
no locked feature.

== Installation ==

1. Upload the plugin to `/wp-content/plugins/`, or install it from the Plugins screen.
2. Activate it.
3. Go to **Tools → AI Crawler Audit** and press **Run the audit**.

It takes up to a minute, because it makes about a dozen requests to your own site.

== Frequently Asked Questions ==

= Does this send my data anywhere? =

No. Every request the plugin makes is to your own site. There is no telemetry, no analytics, no
external API and no account. You can read the source: all the network calls are in
`includes/class-aica-probe.php` and every one of them targets your own home URL.

= It says my server could not fetch my own home page. =

Some hosts block outbound HTTP requests from the site back to itself, which is called a loopback
restriction. That is a hosting configuration, not a crawler problem — but it does mean the live
crawler checks cannot run. Ask your host to allow loopback requests, or run the audit on a different
environment.

= A crawler is allowed in robots.txt but gets 403. Where do I look? =

In this order: your CDN or proxy's bot protection (Cloudflare's Bot Fight Mode and its AI-crawler
controls are the usual cause), then any security plugin with user-agent blocking, then server rules
in `.htaccess` or your nginx config. Fix it at whichever layer is responsible, then run the audit
again.

= Should I block GPTBot, ClaudeBot or CCBot? =

That is a real decision and the plugin does not make it for you. Those are training and dataset
crawlers, and blocking them does not stop ChatGPT or Perplexity citing you — that is done by
`OAI-SearchBot`, `ChatGPT-User` and `PerplexityBot`, which are reported separately. The plugin flags
training blocks as a note rather than a problem, so you can confirm it was a choice and not a default
your host switched on.

= Do I need an llms.txt? =

Probably not urgently. It is a proposed convention rather than a standard and no assistant is known to
require one. It costs nothing, and it is a good place to state plainly what your business does. An
`llms.txt` that describes your page builder instead of your business is worth nothing.

= Is there a paid version? =

No. Every feature is here.

== Changelog ==

= 1.0.0 =
* First release.
