Tag

#AI crawlers

1 post tagged.

AI dev workflow

llms.txt in 2026: I measured who actually reads it

Eleven of fifteen documentation sites serve a real llms.txt. Zero of the five major AI crawler operators document reading yours. Measured on 8 September 2026 with a plain HTTP client, including the commands, plus the one part of the file that does pay off.

9 min read61