Skip to content
Halloween: nothing is scarier than a noindex left in production. Free shipping from €59 Printed to order in Barjac, France

Definition for humans

Crawl

Google's robot going through a website page by page, like a very fast and not very chatty inspector.

In an SEO sentence: “Site is slow? Normal, Googlebot has been crawling since this morning.”

The products that laugh about it

Not orderable yetCrawl budget 2% – T-shirts
seoaddict.co › T-shirts

Crawl budget 2%

€33.90 Jargon level 4/5: Deep insider
Not orderable yetgit blame robots.txt – T-shirts
seoaddict.co › T-shirts

git blame robots.txt

€33.90 Jargon level 5/5: Hardcore
Not orderable yetx-robots-tag: noindex – T-shirts
seoaddict.co › T-shirts

x-robots-tag: noindex

€33.90 Jargon level 5/5: Hardcore
Crawled, not indexed – T-shirts
seoaddict.co › T-shirts

Crawled, not indexed

€33.90 Jargon level 4/5: Deep insider
Not orderable yetCrawling in progress – Kids and babies
seoaddict.co › Kids and babies

Crawling in progress

€26.90 Jargon level 4/5: Deep insider
Not orderable yetLong distance crawler – Caps, bucket hats, fanny packs and aprons
seoaddict.co › Caps, bucket hats, fanny packs and aprons

Long distance crawler

€27.90 Jargon level 4/5: Deep insider
Not orderable yetrobots.txt 2027 – T-shirts
seoaddict.co › T-shirts

robots.txt 2027

€33.90 Jargon level 4/5: Deep insider
Not orderable yetLogs: AI bots – T-shirts
seoaddict.co › T-shirts

Logs: AI bots

€33.90 Jargon level 4/5: Deep insider

← The whole glossary

Your cart