# Get It Done — AI-powered task management for teams.
#
# Machine-readable index for AI agents and answer engines:
#   https://nowgetitdone.com/llms.txt
# Full text, two corpora (same split as the two sitemaps below):
#   https://nowgetitdone.com/llms-full.txt          — the REST API, error codes, webhooks, changelog
#   https://nowgetitdone.com/docs/llms-full.txt     — every product and developer guide
# Individual pages also serve a Markdown twin at <page-url>.md
#   e.g. https://nowgetitdone.com/pricing.md, https://nowgetitdone.com/docs/getting-started.md

User-agent: *
Allow: /
Disallow: /api/
Disallow: /cdn-cgi/

# AI crawlers and assistant fetchers are welcome, stated explicitly.
# A live probe on 2026-09-14 (18 agents x 7 targets) returned 126/126
# byte-identical 200s with no Cloudflare AI block — so this group changes
# nothing today. It exists so a later tightening (a managed rule, a
# "block AI scrapers" toggle, a hand-edited wildcard group) has to argue
# with a named list in a reviewed diff instead of closing the door in
# silence. Being cited by an answer engine is the entire purpose of the
# .md twins and llms.txt above. Google-Extended and Applebot-Extended are
# AI-training/answer tokens only; they do not affect normal search
# crawling. Naming an agent here moves it OUT of the wildcard group, so
# the same Allow/Disallow pair is restated below on purpose.
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: anthropic-ai
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: Bingbot
User-agent: CCBot
User-agent: Meta-ExternalAgent
User-agent: Amazonbot
User-agent: DuckAssistBot
User-agent: cohere-ai
User-agent: YouBot
Allow: /
Disallow: /api/
Disallow: /cdn-cgi/

Sitemap: https://nowgetitdone.com/sitemap.xml
Sitemap: https://nowgetitdone.com/docs/sitemap.xml