Our Crawler
Our crawler collects public information from research peptide vendor websites: product feeds for prices, site status, and public pages such as shipping, refund and COA pages.
How it behaves
- It identifies itself in its user-agent string and links to this page.
- It obeys robots.txt.
- It waits at least 1.5 seconds between requests to the same site and makes only a handful of requests per site per day.
- It only reads public pages. It never logs in, submits forms, adds to cart, or tries to get past bot protection.
Opting out or reporting a problem
Block it in robots.txt, or email [email protected]. If we can't read your public product feed, we can't show your prices, but your transparency checks will still be based on whatever public pages we can verify.