A sitemap is a list of all the pages (URLs) on a website. This file is used by search engines as an overview of all available content as well as a way to (better) understand the structure of the website.
If you create an XML-sitemap for your website, you can help search engines find pages on your site that might otherwise have been overlooked (for example because the website or the URL is too fresh and only has very few incoming links to the page, or because the page is nested very deeply in the site-structure).
By creating an HTML-sitemap you also support search engines during their crawl of your website. Additionally, HTML-sitemaps can benefit users, which can increase the usability of your website.
An XML-sitemap is also useful for search engines. And while it is not guaranteed that all pages will be crawled, it may help with identifying new pages and those nested deeply in your site-structure.
Please remember that only those pages that have been crawled by search engines can be added to their index, which then enables these pages to show up in the search results. While Google can crawl through the links ona website, a sitemap might make it easier for search engines.
- How do I create a sitemap for my website?
- How can I submit a sitemap to Google?
- Google-Index, Google-Bot & Crawler explained
Having an XML-sitemap is no guarantee for all your URLs being indexed
Creating an XML-sitemap is not hard and every webmaster should make a sitemap available for search engines. You can best think of a sitemap as a helpful tool which can assist the Googlebot with finding content more easily. Simply having a sitemap does not, however, guarantee that Google will also index all pages available in your sitemap.
Google’s algorithm for their crawling and indexing of websites decides which URLs will be indexed. If content is seen as being “thin”, or having a low quality, then the URL will not be indexed.
First and foremost, submitting a sitemap doesn’t guarantee the pages referenced in it will be indexed. Think of a sitemap as a way to help the Googlebot find your content. If the URLs were not included in the sitemap, the crawlers might have a harder time finding those URLs and thus they might be indexed slower.
Gary Illyes, Google-Engineer, Webmaster Central Help Forum
Another thing you may want to pay attention to is that the algorithms may decide not to index certain URLs at all. For instance, if the content is shallow, it may be completely indexed or it will not be indexed at all.
