SiteMatchup

How SiteMatchup checks a website

For site owners who saw our bot in their logs, and for anyone who wants to know what we measure.

Our bot SiteMatchup by BYBOWU
  1. 1GET /robots.txtRead first. If it disallows us, or we can’t read it, we stop here.
  2. 2GET http://…Does the plain address send visitors to https://?
  3. 3GET https://…The page itself, read once.
Our bot asks each site for these three public addresses. Redirects between them count too, up to 6 requests per site per run. When mobile speed checks are available, Google loads the page separately from its servers.

What our bot fetches

When someone runs a matchup, our bot asks each site for at most three public addresses:

  • /robots.txt, first. If it asks bots like ours to stay away from the page, we stop there and the report says so.
  • The page that was entered, over http:// and https://, to see whether visitors are sent to the secure version.

If the site redirects, for example from yoursite.com to www.yoursite.com, following the redirect is another request. We stop at 6 requests per site per run, redirects included. We don’t log in, submit forms, follow links deeper into the site or download images and scripts. We read the page’s HTML without running its scripts, so when a script adds the menu or a call link later, the report marks that row “Couldn’t verify” instead of calling it missing. Results are reused for 24 hours, and each site has a daily limit, so the tool can’t be used to load a site over and over.

How to recognise it

Our user agent is:

SiteMatchup by BYBOWU (+https://sitematchup.bybowu.com/tools/site-matchup/about)

To keep it out, add this to your robots.txt:

User-agent: SiteMatchup
Disallow: /

The speed test runs at Google

When available, speed rows come from Google PageSpeed Insights. Google’s own servers load the page on a test phone and send us the result. If this service is unavailable, the report labels those rows “Not checked” and leaves them out of the comparison. Real-visitor data comes from the Chrome UX Report, which Google publishes only for sites with enough visits.

Reports stay private

A report is visible only to people who have its link. It isn’t listed anywhere, search engines are told not to index it, and we never send it to the competitor or publish rankings of businesses. Free reports are deleted after 90 days.

Opting out

If you own a site and don’t want it checked at all, email [email protected] from an address at your domain. We add the domain to our exclusion list and SiteMatchup will refuse to check it from then on.

Privacy

What we store and why is in the SiteMatchup section of the BYBOWU privacy policy. SiteMatchup is a BYBOWU product, built and run by the BYBOWU team (Phoenix, AZ).