Slow down the MSNBot

SISTRIX expert author headshot
From:
Johannes Beus
13.06.2004
13.09.2022

    According to an article on webmasterworld.com, you are now able to limit the MSNBot:

    User-Agent: msnbot
Crawl-Delay: 20

    where20 is the amount of seconds that the MSNBot is supposed to wait between two pageviews.

    SISTRIX expert author headshot
    From:
    Johannes Beus
    13.06.2004
    Related posts
    13.8% of all page views go to searchengines
    At least that is what Hitwise, which uses userprofiles from several ISPs for their statistical data, is claiming. Google’s share comes in at 2.7%, while Yahoo! and MSN can show 1.7 and 1.1 percent, respectively....
    Johannes Beus
    Johannes Beus
    5. June 2004
    Yahoo!-Japan is also replacing Google
    After Yahoo! has already switched from their former index-supplier Google to their own technology in many different countries, they have now also made this move in Japan. Yahoo!-Japan is hoping to increase their advertisement-profits through this....
    Johannes Beus
    Johannes Beus
    3. June 2004
    PageRank and Backlink Update at Google
    It is time for another backlink- and PageRankupdate over at Google. The new values seem to be visible over at http://216.239.41.104/....
    Johannes Beus
    Johannes Beus
    31. May 2004