Get a Quote

Apple’s Web Crawler: AppleBot

Tech savvy individuals have probably already heard the rumors about Apple’s new web crawler, AppleBot. It’s finally making its debate after everyone so patiently waited for. However, this was first developed back on October 15th of last year and wasn’t used until November of the same year. Everyone has remained silent about it since, but it’s finally going a bit more mainstream. Watch out Google because Siri and Spotlight are no longer going to be Apple’s only browsers.

It first made an appearance when a web developer saw hits from a 17.0.0.0/8 IP address, which belongs to Apple. Apple is one of very few companies that have control over an entire block of Internet addresses. When Jan Moesen discovered it back in October, he said it was written in Go programming language, which is said to be a programming tool that makes it simple to build software more efficiently.

The goal of AppleBot is so users don’t have to rely on third-party tools like Bing, Siri and Google to find what they need. For this software, Apple created a browser that gives search results directly. Although AppleBot may never be able to compete with the big boys like Google in a sense that it’s not as large, but for the city of Cupertino – home of Apple – it’s reducing resident’s needs for third-party search tools. Then again, since Apple release an official help document related to the development of AppleBot, who knows how far this web crawler will travel or how large it will get.

The crawler follows the following format string:

“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1).”

The name AppleBot will always appear in it. It follows the customary robots.txt rules. It even follows the typical robots meta tags.

When you don’t mention the term “AppleBot” in your robots.txt, Apple will automatically start following what you mention on Googlebot. The only way you can prevent this from happening is if you block Googlebot. It’s better to block each one individually.

Apple services AppleBot, so you can contact them any time you have a problem or notice unusual activity from AppletBot. You can reach Apple at Apple-NOC or by visiting apple.com.

Currently, it’s now known whether Apple plans on expanded AppleBot to compete with larger browsers like Google, but this program Apple to get a bit closer to it.

At CMIT Solutions, we pride outselves on providing information to help business make better, more informed decisions regarding technology. We want to be a cost efficient solution to managing IT for small businesses. Thus is why we provide you with up-to-date information regarding technology such as Apple’s new programs.

GOT QUESTIONS?

We can help. Whatever your technology problem is, chances are, we've seen it before.