# VideoTex — Robots Exclusion File # Canonical host: https://video.nevatal.tech User-agent: * Allow: / Allow: /watch Allow: /videos/ Allow: /shorts Allow: /results Allow: /feed/ Allow: /about/ Allow: /landing/ Allow: /upload/ Allow: /video/ Allow: /embed/ Allow: /static/ Allow: /media/uploads/thumbnails/ # Crawl budget: nothing below indexes as a page. The download and the # status poll are per-request work with no content, /api/ answers JSON, # and the playlist and queue pages are per-visitor or per-minute state. # /video/stream/ is deliberately NOT disallowed: the sitemap points # at it and Googlebot must be able to fetch it. Disallow: /admin/ Disallow: /api/ Disallow: /video/download/ Disallow: /video/status/ Disallow: /playlist/ Disallow: /status/queue/ Disallow: /media/uploads/original/ Sitemap: https://video.nevatal.tech/sitemap.xml