Plugin Tag: crawler
-
FacetFence Product Filters
Protect WooCommerce filtered URLs from crawl floods with noindex/canonical controls, signed cookies, rate limits, logs, and rollback.
-
Valvai – AI Traffic Control Center
Block, log and monetise AI crawlers directly from your WordPress dashboard.
-
-
AI Kansoku Lab Tracker
Track AI crawler visits such as GPTBot, ClaudeBot, and PerplexityBot server-side and visualize them in WordPress.
-
AI Bot Monitor
Tracks and logs AI, Search and Social bots including ChatGPT, Perplexity, Googlebot, Bing and Meta crawlers.
-
GoVISIBLE AI Bot Tracker
Track visits from AI bots like ChatGPT, GPTBot, Claude, Gemini, and Perplexity.
-
Zicy AI Access Monitor
Monitor known AI crawlers and AI assistant fetches, with daily rollups, CSV exports, weekly summaries, and page briefs.
-
AI Visibility Control – Dynamic llms.txt & robots.txt Crawler Control
Manage AI crawler access. Block GPTBot, Claude & AI bots in robots.txt, generate dynamic llms.txt files, and track referral visits locally.
-
-
Virtual LLMS.TXT Generator
Safely creates a virtual llms.txt dynamically. Features interactive bot targeting, map auto-syncing, and decoupled custom instructions.
-
URLs Most Wanted for LiteSpeed LScache
A demand-driven sitemap for LiteSpeed Cache warmup. Record the URLs visitors actually use and warm only what people need.
-
Pixellize Site Issue Scanner
Powerful SEO auditor by Pixellize. Crawls your site to fix broken links, missing meta tags, and unoptimized images.
-
Whizz MD — Markdown Endpoints for AI Crawlers
Serve every WordPress post and page as a clean .md file with full SEO frontmatter — discoverable by AI crawlers and search engines on the fly.
