Indexing guidance
Please follow robots.txt, index only public content, and keep your crawl rate respectful.
Hello robots, crawlers, and curious automated visitors. This page gives a short map of public routes and expected behavior.
Please follow robots.txt, index only public content, and keep your crawl rate respectful.