# https://www.rupeecase.com/robots.txt
# RupeeCase by QCAlpha | Tanmay Kurtkoti

# ONE group for every crawler, on purpose. A crawler obeys only the single
# most specific group that names it and ignores every other group (RFC 9309;
# Google documents the same rule). This file used to give Googlebot, Bingbot
# and each AI crawler its own group holding nothing but "Allow: /". For those
# crawlers that group REPLACED the wildcard one, so none of the Disallow rules
# ever applied to them: Googlebot was still crawling /cdn-cgi/, /_samples/ and
# the source maps. Listing every user agent at the head of one group gives them
# all the same rules. Do not split a bot back out into its own group unless it
# repeats every Disallow below. (GSC robots audit 2026-09-24)
#
# Removed 12 Oct 2026 (package C2): /_samples/ (deleted in package E1) and the Create React App build
# artifacts (/static/js and /static/css source maps, /asset-manifest.json, /service-worker.js). The v2 build is a
# static copy of public/ and produces none of them; each now 404s.
#
# Disallow lines come BEFORE "Allow: /". Google, Bing and Yandex pick the
# longest matching rule, so order does not matter to them, but some parsers
# take the first match, and for those an "Allow: /" on top would cancel every
# Disallow after it.
#
# Search engines, then AI and answer-engine crawlers. The AI crawlers are
# explicitly welcome so RupeeCase content surfaces in ChatGPT, Perplexity,
# Claude, Gemini and Apple Intelligence answers:
#   GPTBot            OpenAI training and ChatGPT search crawler
#   ChatGPT-User      live ChatGPT browse fetches
#   OAI-SearchBot     OpenAI search index crawler
#   PerplexityBot     Perplexity search index
#   Perplexity-User   live Perplexity citation fetches
#   CCBot             Common Crawl, used by most LLM training corpora
#   anthropic-ai      Anthropic training crawler
#   ClaudeBot         Anthropic index crawler (current name)
#   Claude-Web        live Claude browse fetches (legacy name)
#   Google-Extended   Gemini training control token
#   Applebot-Extended Apple Intelligence training control token
User-agent: *
User-agent: Googlebot
User-agent: Bingbot
User-agent: Slurp
User-agent: DuckDuckBot
User-agent: YandexBot
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: CCBot
User-agent: anthropic-ai
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: Google-Extended
User-agent: Applebot-Extended
Disallow: /manifest.json
Disallow: /podcast/sw.js
# Cloudflare's email-obfuscation endpoint. Cloudflare rewrites every mailto:
# on the site to /cdn-cgi/l/email-protection, which 404s for a crawler. It is
# infrastructure, not a page. (GSC audit 2026-09-21)
Disallow: /cdn-cgi/
Allow: /

# Deliberately NOT disallowed: /podcast/episode. It is noindexed by an
# X-Robots-Tag header in _headers, and a crawler can only see that header if
# it is allowed to fetch the URL. Blocking it here would keep it indexable.

# This file governs www.rupeecase.com only (the apex and the alpha-by-design
# alias 301 here, so their robots.txt resolves to this one). It cannot cover
# invest.rupeecase.com: that is a separate app on a separate origin, and a
# robots.txt applies only to the host that serves it. As of 2026-09-24
# invest.rupeecase.com/robots.txt returns 404, so that host needs its own.

# Sitemap (index only; Google fetches the child sitemaps from it, and
# listing them here too doubles the GSC sitemap-report accounting)
Sitemap: https://www.rupeecase.com/sitemap-index.xml