Pay-per-crawl Definition.

A mechanism Cloudflare launched in July 2025 (in beta) that lets publishers charge AI crawlers for each request to access their content, instead of blocking them or letting them scrape it for free. It reuses the HTTP status code 402 Payment Required to tell the crawler the price, paving the way for pay-per-use monetization between agents and websites. It is part of the same wave as the x402 machine-to-machine payment protocol.

See also

→ x402→ AI payment agent

Where this term appears

GuideAgentic payments around the worldGuideAgentic payments: the fundamentalsGuideStablecoins and machine-to-machine settlementGuideAgentic commerce: scenarios for 2030CourseAgentic payments: the fundamentalsCourseAgentic payments: integration and readiness

All 151 glossary terms · All guides