Tool for the Robots.txt

Johannes Beus
Johannes Beus
(Author)
08.02.2006

    Google is offering a tool though Google Sitemaps that checks the robots.txt. After reading out the file, the tool will display the allowed and restricted directories.

    Johannes Beus
    Johannes Beus
    (Author)
    08.02.2006
    Related posts
    Canonical Addresses
    Christoph Schneegans is giving some tips on how to avoid duplicate content: Canonical Addresses...
    Johannes Beus
    Johannes Beus
    7. February 2006
    Example 2.0
    After the harsh crackdown on doorwaypages at automobile.de, Google has removed bmw.de from the index. Even though it was known for a while that bmw.de was hosting a few doorwaypages with JavaScript-redirection, it seems that Google decided not to measure BMW with two different kinds of measures after this subject […]...
    Johannes Beus
    Johannes Beus
    4. February 2006
    Web.de wants to start local searchengine
    Web.de announced to start a local search for restaurants, hotels, etc. just in time for this years CeBit. The database will come from cooperations with yellow-pages (Neomo?) and their own data....
    Johannes Beus
    Johannes Beus
    1. February 2006