Category: Google Webmaster Blog

We get lots of questions every day – in our Webmaster office hours, at conferences, in the webmaster forum and on Twitter. One of the more frequent themes among these questions are links and especially those generated through JavaScript.

In our Webmaster Conference Lightning Talks video series, we recently addressed the most frequently asked questions on Links and JavaScript:

Note: This video has subtitles in many languages available, too.

During the live premiere, we also had a Q&A session with a few additional questions from the community and decided to publish those questions and our answers along with some other frequently asked questions around the topic of links and JavaScript.

What kinds of links can Googlebot discover?

Googlebot parses the HTML of a page, looking for links to discover the URLs of related pages to crawl. To discover these pages, you need to make your links actual HTML links, as described in the webmaster guidelines on links.

What kind of URLs are okay for Googlebot?

Googlebot extracts the URLs from the href attribute of your links and then enqueues them for crawling. This means that the URL needs to be resolvable or simply put: The URL should work when put into the address bar of a browser. See the webmaster guidelines on links for more information.

Is it okay to use JavaScript to create and inject links?

As long as these links fulfill the criteria as per our webmaster guidelines and outlined above, yes.

When Googlebot renders a page, it executes JavaScript and then discovers the links generated from JavaScript, too. It’s worth mentioning that link discovery can happen twice: Before and after JavaScript executed, so having your links in the initial server response allows Googlebot to discover your links a bit faster.

Does Googlebot understand fragment URLs?

Fragment URLs, also known as “hash URLs”, are technically fine, but might not work the way you expect with Googlebot.

Fragments are supposed to be used to address a piece of content within the page and when used for this purpose, fragments are absolutely fine.

Sometimes developers decide to use fragments with JavaScript to load different content than what is on the page without the fragment. That is not what fragments are meant for and won’t work with Googlebot. See the JavaScript SEO guide on how the History API can be used instead.

Does Googlebot still use the AJAX crawling scheme?

The AJAX crawling scheme has long been deprecated. Do not rely on it for your pages.

The recommendation for this is to use the History API and migrate your web apps to URLs that do not rely on fragments to load different content.

Stay tuned for more Webmaster Conference Lightning Talks

This post was inspired by the first installment of the Webmaster Conference Lightning Talks, but make sure to subscribe to our YouTube channel for more videos to come! We definitely recommend joining the premieres on YouTube to participate in the live chat and Q&A session for each episode!

If you are interested to see more Webmaster Conference Lightning Talks, check out the video Google Monetized Policies and subscribe to our channel to stay tuned for the next one!

Join the webmaster community in the upcoming video premieres and in the YouTube comments!

Posted by Martin Splitt, Googlebot whisperer at the Google Search Relations team

Google Webmaster Help forums are a great place for website owners to help each other, engage in friendly discussion, and get input from awesome Product Experts. We currently have forums operating in 12 languages.

We’re happy to announce the re-opening of the Polish and Turkish webmaster communities with support from a global team of Community Specialists dedicated to helping support Product Experts. If you speak Turkish or Polish, we’d love to have you drop by the new forums yourself, perhaps there’s a question or a challenge you can help with as well!

Current Webmaster Product Experts are welcome to join & keep their status in the new communities as well. If you have previously contributed and would like to start contributing again, you can start posting again, and feel free to ask others in the community if you have any questions.

We look forward to seeing you there!

Posted by Aaseesh Marina, Webmaster Product Support Manager

Fora pomocy dotyczące usług Google dla webmasterów są miejscem, w którym właściciele witryn mogą pomagać sobie nawzajem, dołączać do dyskusji i poznawać wskazówki Ekspertów Produktowych. Obecnie nasze fora działają w 12 językach.

Cieszymy się, że dzięki pomocy globalnego zespołu Specjalistów Społeczności, którzy z oddaniem wspierają Ekspertów Produktowych, możemy przywrócić społeczności webmasterów w językach polskim i tureckim. Jeśli posługujesz się którymś z tych języków, zajrzyj na nasze nowe fora. Może znajdziesz tam problem, który potrafisz rozwiązać?

Zapraszamy obecnych Ekspertów Produktowych usług Google dla webmasterów do dołączenia do nowych społeczności z zachowaniem dotychczasowego statusu. Jeśli chcesz, możesz wrócić do publikowania. W razie pytań zwróć się do innych członków społeczności.

Do zobaczenia!

Autor: Aaseesh Marina, Webmaster Product Support Manager

Each year we attend dozens of events and reach thousands of people with our keynotes, talks, and Q&As. We go to conferences and meetups, because we believe that our talks can potentially help online businesses flourish and we get to help people with their search related problems, but sometimes also listen to their success stories. It’s really uplifting when we hear that, by following our advice, they achieved something great!

We want people to hear about these success stories, so we’re starting a new blog post series that features case studies. They may, for example, help with convincing a boss’ boss that investing in SEO or implementing structured data can be good for the business.
In this first blog post we’re going to start with the overall basics of investing in Search Engine Optimization (SEO), and how investing in it helped a company.

We hope you’ll find this blog post useful. If you’re interested in contributing a case study, submit a talk proposal when signing up for a Webmaster Conference near you and we will consider featuring it. For more case studies and help content, head over to our developer site, help center, or YouTube channel. If you wanna get in touch with us, find us on Twitter.

Posted by Alice Kim and The Gary

Moon Tae Sung is a SEO Manager at Saramin, one of the largest job platforms in Korea. We had the opportunity to ask him a few questions about the effects of his team’s work on Google Search after a presentation he did at a Webmaster Conference in Seoul.

Saramin offers job posting recommendations, company and salary information, AI-based interviews, and AI-based headhunting services. According to Tae Sung, “people come to the Saramin site not only to look for jobs and submit applications, but to also gain a variety of information related to job searches and receive high-quality AI-based services for interview preparation.”

Saramin’s SEO process started with Google Search Console. In 2015 they verified the site in the tool and spent a year identifying and fixing crawling issues. “The task was simple, but still resulted in a 15% increase in the organic traffic“, Tae Sung said. The ROI prompted Saramin to invest more in SEO with the aim of even greater potential success. But first they needed to learn more about what else makes a site search engine friendly so they can better look for help resources. “We studied the Google Search developer’s guide and Help Center articles. These resources continue to provide up-to-date information for issues that we run into“, he told us.

SEO is a process that may take time to bear fruit, so they “started following the SEO guidelines more closely and implemented more changes. The goal was to make changes to the site so that Google Search would better understand it”, Tae Sung shared. They removed meta tags that were cluttered with unnecessary and unhelpful keywords, they used rel-canonical and removed duplicate content, and they explored the search gallery and applied applicable structured data, starting with Job Posting, Breadcrumb, and Estimated salary. 

In addition, they used various Google tools offered as they worked on improving their site. “Errors on our structured data are dealt with by checking URLs on the Structured Data Testing Tool. Other tools like Mobile Friendly Test, AMP Test, and PageSpeed Insight provide us valuable insights for making improvements and helping us offer a better experience for our users,” said Tae Sung.

Over time, Saramin saw the red-colored errors on Search Console’s Index Coverage report gradually turning valid green, and they knew they were headed in the right direction. The incremental changes reached a tipping point and the traffic continued to rise at a more remarkable speed. In the peak hiring season of September 2019, traffic doubled compared to the previous year.


“We are very happy about the traffic increase, but what’s more exciting is it also accompanied improvement in the quality of the traffic. We saw a 93% increase in the number of new sign ups and a 9% increase on the conversion. We believe this means Saramin’s optimization work was found delightful by our users,” said Tae Sung.

Saramin continues to invest in achieving their SEO goals. They’re trying to enhance their users’ experience by implementing more technologies and features from Google, and Tae Sung is enthusiastic about their work ahead: “This is only the beginning of our story.”

As a part of the Webmaster Conference series, last fall we held a Product Summit at Google’s headquarters in Mountain View, California. It was slightly different from our previous events, with a number of product managers and engineers from Google Search taking part. We recorded the talks held there, and are happy to be able to make these available to all of you now.

In the playlist you’ll find:

• Web deduplication – How does Google recognize duplicate content across the web? What happens once a duplicate is found? How is a canonical URL selected? How does localization play a role?
• Google Images best practices – Take a look at how Google Images has evolved over the years, and learn about some of the best practices that you can implement on your site when it comes to images.
• Rendering – Find out more about rendering, and what it takes to do rendering of the web at scale. Take a look behind the scenes, and learn about some things a site owner could watch out for with regards to rendering.
• Titles, snippets, and result previews – What’s the goal of titles, snippets, and previews in Search? How do Google’s systems pick and generate a preview for a page? What are some of the elements that help users decide which page to click in Search?
• Googlebot & web hosting – Starting with a look at the popularity of different web servers, and the growth of HTTPS, you’ll find out more about how Google’s crawling for Search works, and what you can do to control it.
• Claim your Knowledge Panel – Knowledge Panels are a great way for people and organizations to be visible in Search. Find out more about the ways you can claim and update them for yourself or for your business.
• Improving Search over the years – Are dogs the same as cats? Should pages about New York be shown when searching for York? How could algorithms ever figure this out? How many 😊’s does it take to get Google’s attention? Google’s Paul Haahr takes you on a tour of some changes in Search.

We hope you find these videos insightful, useful, and a bit entertaining! And if you are not subscribed to the Webmasters Youtube channel, here’s your chance!

Posted by John Mueller, Search Advocate, Google Switzerland

Health organizations are busier than ever providing information to help with the COVID-19 pandemic. To better assist them, Google has created a best practices article to guide health organizations to make COVID-19 information more accessible on Search. We’ve also created a new technical support group for eligible health organizations.

Best practices for search visibility

By default, Google tries to show the most relevant, authoritative information in response to any search. This process is more effective when content owners help Google understand their content in appropriate ways.

To better guide health-related organizations in this process (known as SEO, for “search engine optimization”), we have produced a new help center article with some important best practices, with emphasis on health information sites, including:

• How to help users access your content on the go
• The importance of good page content and titles
• Ways to check how your site appears for coronavirus-related queries
• How to analyze the top coronavirus related user queries
• How to add structured data for FAQ content

New support group for health organizations

In addition to our best practices help page, health organizations can take part in our new technical support group that’s focused on helping health organizations who publish COVID-19 information with Search related questions.

We’ll be approving requests for access on a case-by-case basis. At first we’ll be accepting only domains under national health ministries and US state level agencies. We’ll inform of future expansions here in this blog post, and on our Twitter account. You’ll need to register using either an email under those domains (e.g. name@health.gov) or have access to the website Search Console account.

Fill this form to request access to the COVID-19 Google Search group 

The group was created to respond to the current needs of health organizations, and we intend to deprecate the group as soon as COVID-19 is no longer considered a Public Health Emergency by WHO or some similar deescalation is widely in place.

Everyone is welcome to use our existing webmaster help forum, and if you have any questions or comments, please let us know on Twitter.

Posted by Daniel Waisberg, Search Advocate & Ofir Roval, Search Console Lead PM

As the effects of the coronavirus grow, we’ve seen businesses around the world looking for ways to pause their activities online. With the outlook of coming back and being present for your customers, here’s an overview of our recommendations of how to pause your business online and minimize impacts with Google Search. These recommendations are applicable to any business with an online presence, but particularly for those who have paused the selling of their products or services online. For more detailed information, also check our developer documentation.

Recommended: limit site functionality

If your situation is temporary and you plan to reopen your online business, we recommend keeping your site online and limiting the functionality. For example, you might mark items as out of stock, or restrict the cart and checkout process. This is the recommended approach since it minimizes any negative effects on your site’s presence in Search. People can still find your products, read reviews, or add wishlists so they can purchase at a later time.

It’s also a good practice to:

• Disable the cart functionality: Disabling the cart functionality is the simplest approach, and doesn’t change anything for your site’s visibility in Search.
• Tell your customers what’s going on: Display a banner or popup div with appropriate information for your users, so that they’re aware of the business’s status. Mention any known and unusual delays, shipping times, pick-up or delivery options, etc. upfront, so that users continue with the right expectations. Make sure to follow our guidelines on popups and banners.
• Update your structured data: If your site uses structured data (such as Products, Books, Events), make sure to adjust it appropriately (reflecting the current product availability, or changing events to cancelled). If your business has a physical storefront, update Local Business structured data to reflect current opening hours.
• Check your Merchant Center feed: If you use Merchant Center, follow the best practices for the availability attribute.
• Tell Google about your updates: To ask Google to recrawl a limited number of pages (for example, the homepage), use Search Console. For a larger number of pages (for example, all of your product pages), use sitemaps.

For more information, check our developers documentation.

Not recommended: disabling the whole website

As a last resort, you may decide to disable the whole website. This is an extreme measure that should only be taken for a very short period of time (a few days at most), as it will otherwise have significant effects on the website in Search, even when implemented properly. That’s why it’s highly recommended to only limit your site’s functionality instead. Keep in mind that your customers may also want to find information about your products, your services, and your company, even if you’re not selling anything right now.

If you decide that you need to do this (again, which we don’t recommend), here are some options:

• If you need to urgently disable the site for 1-2 days, then return an informational error page with a 503 HTTP result code instead of all content. Make sure to follow the best practices for disabling a site.
• If you need to disable the site for a longer time, then provide an indexable homepage as a placeholder for users to find in Search by using the 200 HTTP status code.
• If you quickly need to hide your site in Search while you consider the options, you can temporarily remove it from Search.

For more information, check our developers documentation.

Proceed with caution: To elaborate why we don’t recommend disabling the whole website, here are some of the side effects:

• Your customers won’t know what’s happening with your business if they can’t find your business online at all.
• Your customers can’t find or read first-hand information about your business and its products & services. For example, reviews, specs, past orders, repair guides, or manuals won’t be findable. Third-party information may not be as correct or comprehensive as what you can provide. This often also affects future purchase decisions.
• Knowledge Panels may lose information, like contact phone numbers and your site’s logo.
• Search Console verification will fail, and you will lose all access to information about your business in Search. Aggregate reports in Search Console will lose data as pages are dropped from the index.
• Ramping back up after a prolonged period of time will be significantly harder if your website needs to be reindexed first. Additionally, it’s uncertain how long this would take, and whether the site would appear similarly in Search afterwards.

Other things to consider

Beyond the operation of your web site, there are other actions you might want to take to pause your online business in Google Search:

• If you hold events, look over the new properties for making them virtual, postponed or canceled.
• Review the guidance from Google My Business on how to change your business hours or indicate temporary closures.
• Review the resources from Google for Small Business on how to communicate with customers and employees, for working remotely and modifying advertising campaigns.
• Understand how to recommend changes to your Google knowledge panel (or how to claim it, if you haven’t already).

Also be sure to keep up with the latest by following updates on Twitter from Google Webmasters at @GoogleWMC and Google My Business at @GoogleMyBiz.

FAQs

What if I only close the site for a few weeks?

Completely closing a site even for just a few weeks can have negative consequences on Google’s indexing of your site. We recommend limiting the site functionality instead. Keep in mind that users may also want to find information about your products, your services, and your company, even if you’re currently not selling anything.

What if I want to exclude all non-essential products?

That’s fine. Make sure that people can’t buy the non-essential products by limiting the site functionality.

Can I ask Google to crawl less during this time?

Yes, you can limit crawling with Search Console, though it’s not recommended for most cases. This may have some impact on the freshness of your results in Search. For example, it may take longer for Search to reflect that all of your products are currently not available. On the other hand, if Googlebot’s crawling causes critical server resource issues, this is a valid approach. We recommend setting a reminder for yourself to reset the crawl rate once you start planning to go back in business.

How do I get a page indexed or updated quickly?

To ask Google to recrawl a limited number of pages (for example, the homepage), use Search Console. For a larger number of pages (for example, all of your product pages), use sitemaps.

What if I block a specific region from accessing my site?

Google generally crawls from the US, so if you block the US, Google Search generally won’t be able to access your site at all. We don’t recommend that you block an entire region from temporarily accessing your site; instead, we recommend limiting your site’s functionality for that region.

Should I use the Removals Tool to remove out-of-stock products?

No. People won’t be able to find first-hand information about your products on Search, and there might still be third-party information for the product that may be incorrect or incomplete. It’s better to still allow that page, and mark it out of stock. That way people can still understand what’s going on, even if they can’t purchase the item. If you remove the product from Search, people don’t know why it’s not there.

——–

We realize that any business closure is a big and stressful step, and not everyone will know what to do. If you notice afterwards that you could have done something differently, everything’s not lost: we try to make our systems robust so that your site will be back in Search as quickly as possible. Like you, we’re hoping that this crisis finds an end as soon as possible. We hope that with this information, you’re able to have your online business up & running quickly when that time comes. Should you run into any problems or questions along the way, please don’t hesitate to use our public channels to get help.

Posted by John Mueller, working from home in Zurich, Switzerland

In the current environment and status of COVID-19 around the world, many events are being canceled, postponed, or moved to an online-only format. Google wants to show users the latest, most accurate information about your events in this fast-changing e…

Last year we organized Webmaster Conference events in over 15 countries. The spirit of WMConf is reflected in that number: we want to reach regions that otherwise don’t get much search conference love. Recently we have been monitoring the coronavirus (COVID-19) situation and its impact on planning this year’s events.

To wit, with the growing concern around coronavirus and in line with the travel guidelines published by WHO, the CDC, and other organizations, we are postponing all Webmaster Conference events globally. While we’re hoping to organize events later this year, we’re also exploring other ways to reach targeted audiences.

We’re very sorry to delay the opportunity to connect in person, but we feel strongly that the safety and health of all attendees is the priority at this time. If you want to get notified about future events in your region, you can sign up to receive updates on the Webmaster Conference site. If you have questions or comments, catch us on Twitter!

Posted by Cherry Prommawin and Gary

It’s been a few years now that Google started working on mobile-first indexing – Google’s crawling of the web using a smartphone Googlebot. From our analysis, most sites shown in search results are good to go for mobile-first indexing, and 70% of those shown in our search results have already shifted over. To simplify, we’ll be switching to mobile-first indexing for all websites starting September 2020. In the meantime, we’ll continue moving sites to mobile-first indexing when our systems recognize that they’re ready.

When we switch a domain to mobile-first indexing, it will see an increase in Googlebot’s crawling, while we update our index to your site’s mobile version. Depending on the domain, this change can take some time. Afterwards, we’ll still occasionally crawl with the traditional desktop Googlebot, but most crawling for Search will be done with our mobile smartphone user-agent. The exact user-agent name used will match the Chromium version used for rendering.  

In Search Console, there are multiple ways to check for mobile-first indexing. The status is shown on the settings page, as well as in the URL Inspection Tool, when checking a specific URL with regards to its most recent crawling.

Our guidance on making all websites work well for mobile-first indexing continues to be relevant, for new and existing sites. In particular, we recommend making sure that the content shown is the same (including text, images, videos, links), and that meta data (titles and descriptions, robots meta tags) and all structured data is the same. It’s good to double-check these when a website is launched or significantly redesigned. In the URL Testing Tools you can easily check both desktop and mobile versions directly. If you use other tools to analyze your website, such as crawlers or monitoring tools, use a mobile user-agent if you want to match what Google Search sees. 

While we continue to support various ways of making mobile websites, we recommend responsive web design for new websites. We suggest not using separate mobile URLs (often called “m-dot”) because of issues and confusion we’ve seen over the years, both from search engines and users. 

Mobile-first indexing has come a long way. It’s great to see how the web has evolved from desktop to mobile, and how webmasters have helped to allow crawling & indexing to match how users interact with the web! We appreciate all your work over the years, which has helped to make this transition fairly smooth. We’ll continue to monitor and evaluate these changes carefully. If you have any questions, please drop by our Webmaster forums or our public events.

Posted by John Mueller, Developer Advocate, Google Zurich

Having up-to-date information during large, public events is critical, as the landscape changes by the minute. This guide highlights some tools that news publishers can use to create a data rich and engaging experience for their users.

Add Article structured data to AMP pages

Adding Article structured data to your news, blog, and sports article AMP pages can make the content eligible for an enhanced appearance in Google Search results. Enhanced features may include placement in the Top stories carousel, host carousel, and Visual stories. Learn how to mark up your article.
You can now test and validate your AMP article markup in the Rich Results Test tool. Enter your page’s URL or a code snippet, and the Rich Result Test shows the AMP Articles that were found on the page (as well as other rich result types), and any errors or suggestions for your AMP Articles. You can also save the test history and share the test results.
We also recommend that you provide a publication date so that Google can expose this information in Search results, if this information is considered to be useful to the user.

Mark up your live-streaming video content

If you are live-streaming a video during an event, you can be eligible for a LIVE badge by marking your video with BroadcastEvent. We strongly recommend that you use the Indexing API to ensure that your live-streaming video content gets crawled and indexed in a timely way. The Indexing API allows any site owner to directly notify Google when certain types of pages are added or removed. This allows Google to schedule pages for a fresh crawl, which can lead to more relevant user traffic as your content is updated. For websites with many short-lived pages like livestream videos, the Indexing API keeps content fresh in search results. Learn how to get started with the Indexing API.

For AMP pages: Update the cache and use components

Use the following to ensure your AMP content is published and up-to-date the moment news breaks.

Update the cache

When people click an AMP page, the Google AMP Cache automatically requests updates to serve fresh content for the next person once the content has been cached. However, if you want to force an update to the cache in response to a change in the content on the origin domain, you can send an update request to the Google AMP Cache. This is useful if your pages are changing in response to a live news event.

Use news-related AMP components

• <amp-live-list>: Add live content to your article and have it updated based on a source document. This is a great choice if you just want content to reload easily, without having to set up or configure additional services on your backend. Learn how to implement <amp-live-list>.
• <amp-script>: Run your own JavaScript inside of AMP pages. This flexibility means that anything you are publishing on your desktop or non-AMP mobile pages, you can bring over to AMP. <amp-script> supports Websockets, interactive SVGs, and more. This allows you to create engaging news pages like election coverage maps, live graphs and polls etc. As a newer feature, the AMP team is actively soliciting feedback on it. If for some reason it doesn’t work for your use case, let us know.

If you have any questions, let us know through the forum or on Twitter.

Posted by Patrick Kettner and Naina Raisinghani, AMP team