JavaScript is one of the reasons why websites today can behave, in many ways, like regular computer programs instead of being mere static pages on the Internet. JavaScript can be used to add a number of neat interactive and practical elements to a web page, such as image carousels and automatically reloading page content.
For years, Google crawled websites based on textual content. Google says that it has been able to render web pages with dynamic Javascript content without needing any outside help since 2015, to the extent that they have said goodbye to their Ajax crawling scheme proposed in 2009.
Nevertheless, there are a few things to keep in mind:
- JavaScript and CSS files should not be blocked in robots.txt, so that the crawler can render and index the website properly
- If the web server is not able to handle the number of crawl requests for resources, this could have a negative impact on the renderability of web pages
- In some cases, JavaScript is too complex or obscure for Googlebot. The crawler will then not fully render the web page.
- JavaScript that removes and does not add content on a page usually causes problems with the indexing of content
- Web pages that render quickly are optimal for the crawler and indexing
What difference can JavaScript make?
To take a look at a remarkable, but not representative, example, here is the page layout of Run4Tiger, a WWF campaign which is completely based on JavaScript:
If we deactivate JavaScript in the browser and call up the same domain, the screen remains black.
The impact is not quite so remarkable on most other websites, however. Let’s take a look at a subpage on Zalando, for example.
First with JavaScript activated:
And then once again without:
Some functions of the website, such as image selection and the information text below the product, are not available without JavaScript. However, the page can still be called up, and I can look at the shoe and add it to my shopping basket.
JavaScript and search engine optimisation
When it comes to search engine optimisation, JavaScript can be both a blessing and a curse .
A blessing, because JavaScript offers a wide range of design and implementation possibilities for the strategic planning, conceptual design and implementation of the website.
It is a curse in terms of the search engine optimisation itself, however, as the additional costs of testing and monitoring for JavaScript will outweigh the benefits. Google is already quite good at executing JavaScript, but it is still far from perfect.
This means that the use of JavaScript should always be checked to see how Google can handle it. Fortunately, there is a feature in the Google Search Console called “Retrieve as by Google”, which can also be used to display the rendered web page from Google’s point of view.
In the above screenshot, which displays the seo.at start page, we can see that the interactive graphic on the right is not recognised by the Googlebot on the left. This is despite the fact that the visualisation itself is created via the Google visualisation API and loads extra JavaScript from Google for this purpose.
So if Google consistently refuses to use the content that it has created when ranking a page, it can be worth taking a look at the Search Console. Perhaps Google is not yet ready to completely render and adopt the JavaScript required for the content.
Play it safe with Progressive Enhancement
Google recommends building your website with the concept of Progressive Enhancement in mind. Planning the website, starting from a minimally usable version up to a full display, one ensures that the most important content is always available for users, even when they’re using slower or more basic browsers. And since the most important content should also always be available to Google, this is a win-win situation.
JavaScript Frameworks and Google
In recent years, more and more frameworks for JavaScript have emerged. These frameworks cover frequent use cases and can thus save a lot of time and nerves when developing your own website.
Two of the best-known frameworks are AngularJS, which is offered by Google under an open source licence, and React, which is offered by Facebook under an open source licence.
There are many other frameworks, all of which can save a lot of time when creating a website but cannot necessarily be fully evaluated by Google or other search engines when it comes to search engine optimisation – not even by Google’s own Angular framework.
When using JavaScript, please always consider the question of whether Google can read and render everything that you have painstakingly created.
