robots.txt Policy Lint — AI Crawler Index

Take a robots.txt you already have and say what it actually does. It lints the file against RFC 9309 and reports the errors that silently change meaning, answers whether a named crawler may fetch a named path and which rule decided it, audits which AI crawlers the file really stops (and which it only appears to), diffs two versions by EFFECT rather than by line, and merges a ready-made stance into an existing file without discarding the rules already there. It reads the file you paste; it fetches nothing. robots.txt is a request, not an enforcement mechanism, and the audit says so where a crawler is known to ignore it. Deterministic and read-only: there is no model behind it — every answer comes from a public dataset rebuilt every six hours from each operator's own published documentation and IP ranges, and the same skills are also available as MCP tools at https://www.pathwren.workers.dev/mcp/robots. No key, no signup, no quota. Independent and unaffiliated with any operator it documents.