For site operators
OfferCompassBot
The crawler behind OfferCompass's job search. If you found this URL in your access logs, everything it does is below, including how to make it stop.
- User-Agent
- OfferCompassBot/1.0
- What it reads
- The public, unauthenticated job feed your applicant tracking system already publishes, the same endpoint your own careers page calls to render its list of openings. It does not scrape your careers page HTML, does not log in, does not submit forms, and never touches an application.
- How often
- A board is re-read at most once every six hours, at three concurrent requests with 200ms between them, which is roughly five requests a second at the very most, and far less in steady state. A 429 or 503 pauses every request to your infrastructure for as long as your Retry-After header asks.
- robots.txt
- Read before any request and obeyed, including Crawl-delay. Both our own token and User-agent: * are honoured, and our token takes precedence when you name it.
- What we publish
- Your posting, attributed to you, linking to your own application URL. Never a reposted or rehosted apply flow, never an aggregator redirect, and no interstitial between the candidate and your form.
- When a role closes
- It leaves the index. A posting absent from your feed is retired, and we independently re-check the apply URL: a link that 404s closes the posting whatever the feed still says.
Blocking it yourself
Add this to your robots.txt and the next crawl will skip you. No approval and no waiting on us: robots.txt is checked before every request, so the block takes effect as soon as you publish it.
User-agent: OfferCompassBot
Disallow: /To slow it down rather than block it, add a Crawl-delay in seconds to the same group. If your postings are served by a hosted applicant tracking system on its own domain, the robots.txt that governs us is the one on the host serving the feed.
Asking us to remove you
Email support@offercompass.app from an address at your domain, naming the company or board. We delete the postings and stop crawling the board. We do not ask for a reason, and we do not make you argue for it. If you would rather not email, the robots.txt rule above does the same thing without us.
If it is behaving badly
Requests arriving faster than described above, ignoring a Retry-After, or hitting anything other than a job feed is a bug, and we would genuinely like to hear about it. Send a few log lines with timestamps to support@offercompass.app and we will find it.
Operated by Sven Güttner, Essen, Germany. Full details in the imprint.