Apple has been making plays on traditional Google turf for the last few years now, but today comes word of possibly the biggest so far. In a document on Apple’s Support repository, the company details what they’re calling ‘Applebot’, a web crawler that will provide data for Siri and Spotlight Suggestions.
Google has made a massive fortune indexing the web for nearly two decades. It’s no surprise, then, that as Apple have grown to become the most valuable company in the world, that they have begun to work on their own in-house search efforts. The support page doesn’t go into much detail, but notes that little, if anything, will need to be done by webmasters to allow Applebot to index their sites.
Apple will identify Applebot in user agent strings, and will read robots.txt files in the same way Googlebot does. In fact, the page also specifically says that if no instructions for Applebot are explicitly provided, the crawler will use Googlebot instructions.
Based on past interviews with Apple executives, as well as the above details, it doesn’t seem as though Apple is getting into the advertising business, just that they wish to bolster their own search services in Spotlight on the Mac and on iOS, and to allow Siri to provide better search results natively, instead of sending you to Safari.
[source]9to5Mac[/source]
