Architecture Studio

ArchStudioBot

ArchStudioBot is the crawler behind the Arch Studio product index. This page explains how it behaves, how to block or limit it, and how to reach us.

Public documentation · No account needed to read this↓ Download .md
01

What ArchStudioBot is

ArchStudioBot builds the product index behind Arch Studio, an LLM product search engine for architects. It reads the public product pages and sitemaps of brand websites and links designers back to the brand’s own product page.

It indexes product facts (name, maker, dimensions, materials, finishes, price) so designers can find the product. In Cloudflare’s terms, ArchStudioBot is a Search crawler: it crawls to build a search index and RAG database, which our AI uses to improve search results for designers.

02

User agent

Requests from the crawler identify themselves with this user agent string.

ArchStudioBot/1.0 (+https://architecturestudio.ai/bot)

03

How it behaves

  • Obeys robots.txt, including Crawl-delay.
  • Makes at most about one request every one to two seconds per site, one request at a time.
  • Backs off on 429 and 503 responses and honors Retry-After.
  • Stops crawling a site when blocked (a 403, a challenge page or a robots.txt disallow) until a person reviews it.
  • Never logs in. It never goes to cart, checkout or account pages on its own; it fetches one only if a user submits that URL.
  • Does not rotate IP addresses or solve CAPTCHAs.
  • Refetches a page only when its sitemap lastmod changes; otherwise it revisits on a slow schedule.
04

How to block or limit it

Add a User-agent: ArchStudioBot group to your robots.txt. To block the crawler entirely, disallow everything; to slow it down, set a Crawl-delay in seconds.

User-agent: ArchStudioBot
Disallow: /

# Or, to limit it instead:
# User-agent: ArchStudioBot
# Crawl-delay: 10

05

Contact

To report a problem with the crawler or ask us to stop crawling your site, email help@architecturestudio.ai. Include your domain and, if you can, the time and path of the requests.

06

Request signing

ArchStudioBot signs its requests with Web Bot Auth (HTTP Message Signatures, RFC 9421). Each request carries the Signature-Agent header below and a signature over the host it is sent to. Our public keys are in the key directory at the URL below. A request that claims to be ArchStudioBot without a valid signature is not from us.

Signature-Agent: "https://bot.architecturestudio.ai"