CielStayBot
What our crawler does, how it behaves, and how to stop it.
CielStayBot/1.0 (+https://www.cielstay.com/bot)What it's for
CielStay is a discovery platform, not a booking engine. We never process guest payments and never take a percentage of a stay. When a guest finds a property with us, they book on the host's own site or the platform the host already uses — so the traffic our crawler makes possible goes to you, not away from you.
Our crawler reads publicly available property pages to keep the details we show accurate: description, photos, amenities, location, and where they can be booked. It also checks a page when a host is proving they control a property they want to claim.
How it behaves
- It identifies itself. Our crawler is CielStayBot, and this page tells you what it does and how to stop it. If you want to know whether traffic is ours, ask us and we will tell you.
- We read what a page shows the public. Property details, and the availability those pages display to any visitor. Some need a browser to render; where they do, we use one, and we read what the page shows — nothing more.
- We never defeat anything. No logins, no paywalls, no CAPTCHA solving, no proxies. Every request is made logged-out and from our own address. If content needs an account or a challenge to reach, we don't take it.
- We honour robots.txt on the site that publishes the property, and any
Crawl-delayit sets. Booking software often serves availability from shared infrastructure that blocks all crawlers for search-indexing reasons; where a property's own site permits us, we read the availability that page shows the public. - We don't collect catalogues. No bulk crawling of large booking platforms. Where we check a page on one, it is a single request to verify one specific thing.
- It stays small, and it stops when asked. We fetch at a low rate and don't re-fetch pages that haven't changed. We are a small company, not a data broker.
How to stop it
Add this to your robots.txt and we will stop, usually within a day:
User-agent: CielStayBot Disallow: /
Or email inquiries@cielstay.com and we will remove your properties and stop fetching them. You don't have to explain why, and we won't re-add them from the same source. Hosts can also use our .
If something looks wrong
If our crawler behaved differently from what is described here — too fast, ignoring your robots.txt, or not stopping when you asked — that is a bug and we want to know. Email inquiries@cielstay.com with a timestamp and we will find it.
More about where listings come from and what we show: .