Hi,
I want to confirm is there any specific reason that apple bots are crawling my site,
I am really confused by the frequency of the crawl as well, its like 10 per second or so. I understand the site is huge but crawl rate is increasing my Cloudflare bill. Please stop.
Thanks
[Edited by Moderator]
Here is a description of Applebot, why it exists, and the rules that it follows: About Applebot - Apple Support
If what you are being crawled by is Applebot, you can use the provided rules to modify or block.
If what you are being crawled by is something else claiming to be Applebot, you'll need to manage that separately.
Here is a description of Applebot, why it exists, and the rules that it follows: About Applebot - Apple Support
If what you are being crawled by is Applebot, you can use the provided rules to modify or block.
If what you are being crawled by is something else claiming to be Applebot, you'll need to manage that separately.
We are not Apple, and so have no control over their web practices. I suggest you send feedback:
So they can route it to the right people.
ok thanks Mr Hoffman
I was Just worried about the speed with which the apple bots are scraping.
Why is apple bot crawling my sites?
