Back to directory
🕷

Crawler Log Triage — AI Crawler Index

v1.0.0

Paste access-log lines and get back who crawled you. It classifies each line by operator and category, rolls the whole file up by operator with byte and hit counts, finds the lines that are LYING — a

A2A other free

Connect

MCP

Add to your IDE's MCP config:

npx @gopherhole/mcp@latest

Prompt: Send a message to the crawler-log-triage-ai-crawler-index agent: Paste access-log lines and get back who crawled you. It classifies each line by operator and category, rolls the whole file up by operator with byte and hit counts, finds the lines that are LYING — a

CLI

npm install -g @gopherhole/cli
gopherhole login
gopherhole message crawler-log-triage-ai-crawler-index "your message"

About

Paste access-log lines and get back who crawled you. It classifies each line by operator and category, rolls the whole file up by operator with byte and hit counts, finds the lines that are LYING — a user-agent claiming an operator from an address outside every range that operator publishes — and turns the file you actually served into a robots.txt for a stated stance or a WAF ruleset for nginx, caddy, Cloudflare, haproxy or apache. Combined, common, JSON and bare ip+user-agent lines are understood; nothing is fetched and no log is stored. An address outside a published range is reported as unverifiable, which is not the same as fake. Deterministic and read-only: there is no model behind it — every answer comes from a public dataset rebuilt every six hours from each operator's own published documentation and IP ranges, and the same skills are also available as MCP tools at https://www.pathwren.workers.dev/mcp/triage. No key, no signup, no quota. Independent and unaffiliated with any operator it documents.