How SiteMatchup checks a website
For site owners who saw our bot in their logs, and for anyone who wants to know what we measure.
- 1
GET /robots.txtRead first. If it disallows us, or we can’t read it, we stop here. - 2
GET http://…Does the plain address send visitors to https://? - 3
GET https://…The page itself, read once.
What our bot fetches
When someone runs a matchup, our bot asks each site for at most three public addresses:
/robots.txt, first. If it asks bots like ours to stay away from the page, we stop there and the report says so.- The page that was entered, over
http://andhttps://, to see whether visitors are sent to the secure version.
If the site redirects, for example from yoursite.com to www.yoursite.com, following the redirect is another request. We stop at 6 requests per site per run, redirects included. We don’t log in, submit forms, follow links deeper into the site or download images and scripts. We read the page’s HTML without running its scripts, so when a script adds the menu or a call link later, the report marks that row “Couldn’t verify” instead of calling it missing. Results are reused for 24 hours, and each site has a daily limit, so the tool can’t be used to load a site over and over.
How to recognise it
Our user agent is:
SiteMatchup by BYBOWU (+https://sitematchup.bybowu.com/tools/site-matchup/about)
To keep it out, add this to your robots.txt:
User-agent: SiteMatchup
Disallow: /
The speed test runs at Google
When available, speed rows come from Google PageSpeed Insights. Google’s own servers load the page on a test phone and send us the result. If this service is unavailable, the report labels those rows “Not checked” and leaves them out of the comparison. Real-visitor data comes from the Chrome UX Report, which Google publishes only for sites with enough visits.
Reports stay private
A report is visible only to people who have its link. It isn’t listed anywhere, search engines are told not to index it, and we never send it to the competitor or publish rankings of businesses. Free reports are deleted after 90 days.
Opting out
If you own a site and don’t want it checked at all, email [email protected] from an address at your domain. We add the domain to our exclusion list and SiteMatchup will refuse to check it from then on.
Privacy
What we store and why is in the SiteMatchup section of the BYBOWU privacy policy. SiteMatchup is a BYBOWU product, built and run by the BYBOWU team (Phoenix, AZ).