Optimizing for Bandwidth on Apache and Nginx
Webmaster level: advancedEveryone wants to use less bandwidth: hosts want lower bills, mobile users want to stay under their limits, and no one wants to wait for unnecessary bytes. The web is full of opportunities to save bandwidth: pages served withou…
#NoHacked: a global campaign to spread hacking awareness
Webmaster level: All
This June, we introduced a weeklong social campaign called #NoHacked. The goals for #NoHacked are to bring awareness to hacking attacks and offer tips on how to keep your sites safe from hackers.
We held the campaign in 11 languages on multiple channels including Google+, Twitter and Weibo. About 1 million people viewed our tips and hundreds of users used the hashtag #NoHacked to spread awareness and to share their own tips. Check them out below!
Posts we shared during the campaign:
https://plus.google.com/+GoogleWebmasters/posts/1BzXjgJMGFU
https://plus.google.com/+GoogleWebmasters/posts/TMhfwQG3p8P
https://plus.google.com/+GoogleWebmasters/posts/AcUS4WhF6LL
https://plus.google.com/+GoogleWebmasters/posts/DUTpSGmkBUP
https://plus.google.com/+GoogleWebmasters/posts/UjZRbySM5gMSome of the many tips shared by users across the globe:
- Pablo Silvio Esquivel from Brazil recommends users not to use pirated software (source)
- Rens Blom from the Netherlands suggests using different passwords for your accounts, changing them regularly, and using an extra layer of security such as two-step authentication (source)
- Дмитрий Комягин from Russia says to regularly monitor traffic sources, search queries and landing pages, and to look out for spikes in traffic (source)
- 工務店コンサルタント from Japan advises everyone to choose a good hosting company that’s knowledgeable in hacking issues and to set email forwarding in Webmaster Tools (source)
- Kamil Guzdek from Poland advocates changing the default table prefix in wp-config to a custom one when installing a new WordPress to lower the risk of the database from being hacked (source)
Hacking is still a surprisingly common issue around the world so we highly encourage all webmasters to follow these useful tips. Feel free to continue using the hashtag #NoHacked to share your own tips or experiences around hacking prevention and awareness. Thanks for supporting the #NoHacked campaign!
And in the unfortunate event that your site gets hacked, we’ll help you toward a speedy and thorough recovery:
Posted by your friendly #NoHacked helpers
HTTPS as a ranking signal
Webmaster level: all
Security is a top priority for Google. We invest a lot in making sure that our services use industry-leading security, like strong HTTPS encryption by default. That means that people using Search, Gmail and Google Drive, for example, automatically have a secure connection to Google.
Beyond our own stuff, we’re also working to make the Internet safer more broadly. A big part of that is making sure that websites people access from Google are secure. For instance, we have created resources to help webmasters prevent and fix security breaches on their sites.
We want to go even further. At Google I/O a few months ago, we called for “HTTPS everywhere” on the web.
We’ve also seen more and more webmasters adopting HTTPS (also known as HTTP over TLS, or Transport Layer Security), on their website, which is encouraging.
For these reasons, over the past few months we’ve been running tests taking into account whether sites use secure, encrypted connections as a signal in our search ranking algorithms. We’ve seen positive results, so we’re starting to use HTTPS as a ranking signal. For now it’s only a very lightweight signal — affecting fewer than 1% of global queries, and carrying less weight than other signals such as high-quality content — while we give webmasters time to switch to HTTPS. But over time, we may decide to strengthen it, because we’d like to encourage all website owners to switch from HTTP to HTTPS to keep everyone safe on the web.
In the coming weeks, we’ll publish detailed best practices (we’ll add a link to it from here) to make TLS adoption easier, and to avoid common mistakes. Here are some basic tips to get started:
- Decide the kind of certificate you need: single, multi-domain, or wildcard certificate
- Use 2048-bit key certificates
- Use relative URLs for resources that reside on the same secure domain
- Use protocol relative URLs for all other domains
- Check out our Site move article for more guidelines on how to change your website’s address
- Don’t block your HTTPS site from crawling using robots.txt
- Allow indexing of your pages by search engines where possible. Avoid the noindex robots meta tag.
If your website is already serving on HTTPS, you can test its security level and configuration with the Qualys Lab tool. If you are concerned about TLS and your site’s performance, have a look at Is TLS fast yet?. And of course, if you have any questions or concerns, please feel free to post in our Webmaster Help Forums.
We hope to see more websites using HTTPS in the future. Let’s all make the web more secure!
Posted by Zineb Ait Bahajji and Gary Illyes, Webmaster Trends Analysts
Introducing the Google News Publisher Center
(Cross-posted on the Google News Blog)
Webmaster level: All
If you’re a news publisher, your website has probably evolved and changed over time — just like your stories. But in the past, when you made changes to the structure of your site, we might not have discovered your new content. That meant a lost opportunity for your readers, and for you. Unless you regularly checked Webmaster Tools, you might not even have realized that your new content wasn’t showing up in Google News. To prevent this from happening, we are letting you make changes to our record of your news site using the just launched Google News Publisher Center.
With the Publisher Center, your potential readers can be more informed about the articles they’re clicking on and you benefit from better discovery and classification of your news content. After verifying ownership of your site using Google Webmaster Tools, you can use the Publisher Center to directly make the following changes:
- Update your news site details, including changing your site name and labeling your publication with any relevant source labels (e.g., “Blog”, “Satire” or “Opinion”)
- Update your section URLs when you change your site structure (e.g., when you add a new section such as http://example.com/2014commonwealthgames or http://example.com/elections2014)
- Label your sections with a specific topic (e.g., “Technology” or “Politics”)
Whenever you make changes to your site, we’d recommend also checking our record of it in the Publisher Center and updating it if necessary.
Try it out, or learn more about how to get started.
At the moment the tool is only available to publishers in the U.S. but we plan to introduce it in other countries soon and add more features. In the meantime, we’d love to hear from you about what works well and what doesn’t. Ultimately, our goal is to make this a platform where news publishers and Google News can work together to provide readers with the best, most diverse news on the web.
Posted by Eric Weigle, Software Engineer
Promoting modern websites for modern devices in Google search results
Webmaster level: all A common annoyance for web users is when websites require browser technologies that are not supported by their device. When users access such pages, they may see nothing but a blank space or miss out a large portion of the page’s c…
Troubleshooting hreflang annotations in Webmaster Tools
If you are targeting users in more than one country, chances are you already heard about rel-alternate-hreflang. If you haven’t, in short, this annotation enables Google and other search engines to serve the correct language or regional version of pages to searchers, which can lead to increased user satisfaction.
Making sure the deployed annotations are usable by search engines can be rather difficult, especially on sites with many pages, and site owners all around the world haven’t been shy telling us about this. Today we’re releasing a feature that should make debugging rel-alternate-hreflang annotations much easier.
The Language Targeting section in the International Targeting feature enables you to identify two of the most common issues with hreflang annotations:
-
Missing return links: annotations must be confirmed from the pages they are pointing to. If page A links to page B, page B must link back to page A, otherwise the annotations may not be interpreted correctly.
For each error of this kind we report where and when we detected them, as well as where the return link is expected to be.
-
Incorrect hreflang values: The value of the hreflang attribute must either be a language code in ISO 639-1 format such as “es”, or a combination of language and country code such as “es-AR”, where the country code is in ISO 3166-1 Alpha 2 format.
In case our indexing systems detect language or country codes that are not in these formats, we provide example URLs to help you fix them.
Additionally, we’ve moved the geographic targeting setting to this part of Webmaster Tools, so that you can find all information relevant to international and multilingual targeting in the same place.
We hope you’ll find this new feature useful and that it helps you to identify issues with the rel-hreflang-implementation on your site. If you have comments or questions about the feature, please post in our Webmaster Help Forum.
Posted by Gary Illyes, Webmaster Trends
Android app indexing is now open for everyone!
Webmaster level: All
Do you have an Android app in addition to your website? You can now connect the two so that users searching from their smartphones and tablets can easily find and reach your app content.
App deep links in search results help your users find your content more easily and re-engage with your app after they’ve installed it. As a site owner, you can show your users the right content at the right time — by connecting pages of your website to the relevant parts of your app you control when your users are directed to your app and when they go to your website.
Hundreds of apps have already implemented app indexing. This week at Google I/O, we’re announcing a set of new features that will make it even easier to set up deep links in your app, connect your site to your app, and keep track of performance and potential errors.
Getting started is easy
We’ve greatly simplified the process to get your app deep links indexed. If your app supports HTTP deep linking schemes, here’s what you need to do:
- Add deep link support to your app
- Connect your site and your app
- There is no step 3 (:
As we index your URLs, we’ll discover and index the app / site connections and may begin to surface app deep links in search results.
We can discover and index your app deep links on our own, but we recommend you publish the deep links. This is also the case if your app only supports a custom deep link scheme. You publish them in one of two ways:
- Insert a rel=alternate elment in the section of each web page, or in your sitemap to specify app URIs. Find out how to implement these methods on our developer site.
- Use the App indexing API
There’s one more thing: we’ve added a new feature in Webmaster Tools to help you debug any issues that might arise during indexing app pages. It will show you what type of errors we’ve detected for the app page-web page pairs, together with example app URIs so you can debug:
We’ll also give you detailed instructions on how to debug each issue, including a QR code for the app deep links, so you can easily open them on your phone or tablet. We’ll send you Webmaster Tools error notifications as well, so you can keep up to date.
Give app indexing a spin, and as always, if you need more help ask questions on the Webmaster help forum.
Posted by Mariya Moeva, Webmaster Trends Analyst
Making site moves easier
Webmaster level: intermediate-advanced Few topics confuse and scare webmasters more than site moves. To help you avoid surprises, we’ve created an in-depth guide on how to handle site moves in a Googlebot-friendly way. So what, exactly, is a site move …
Directing smartphone users to the page they actually wanted
Webmaster level: all
Have you ever used Google Search on your smartphone and clicked on a promising-looking result, only to end up on the mobile site’s homepage, with no idea why the page you were hoping to see vanished? This is such a common annoyance that we’ve even seen comics about it. Usually this happens because the website is not properly set up to handle requests from smartphones and sends you to its smartphone homepage—we call this a “faulty redirect”.
We’d like to spare users the frustration of landing on irrelevant pages and help webmasters fix the faulty redirects. Starting today in our English search results in the US, whenever we detect that smartphone users are redirected to a homepage instead of the the page they asked for, we may note it below the result. If you still wish to proceed to the page, you can click “Try anyway”:
And we’re providing advice and resources to help you direct your audience to the pages they want. Here’s a quick rundown:
1. Do a few searches on your own phone (or with a browser set up to act like a smartphone) and see how your site behaves. Simple but effective. :)
2. Check out Webmaster Tools—we’ll send you a message if we detect that any of your site’s pages are redirecting smartphone users to the homepage. We’ll also show you any faulty redirects we detect in the Smartphone Crawl Errors section of Webmaster Tools:
3. Investigate any faulty redirects and fix them. Here’s what you can do:
- Use the example URLs we provide in Webmaster Tools as a starting point to debug exactly where the problem is with your server configuration.
- Set up your server so that it redirects smartphone users to the equivalent URL on your smartphone site.
- If a page on your site doesn’t have a smartphone equivalent, keep users on the desktop page, rather than redirecting them to the smartphone site’s homepage. Doing nothing is better than doing something wrong in this case.
- Try using responsive web design, which serves the same content for desktop and smartphone users.
If you’d like to know more about building smartphone-friendly sites, read our full recommendations. And, as always, if you need more help you can ask a question in our webmaster forum.
Posted by Mariya Moeva, Webmaster Trends Analyst
App Indexing in more languages
Webmaster level: all
In April, we launched App Indexing in English globally so deep links to your mobile apps could appear in Google Search results on Android everywhere. Today, we’re adding the first publishers with content in other languages: Fairfax Domain, MercadoLibre, Letras.Mus.br, Vagalume, Idealo, L’Equipe, Player.fm, Upcoming, Au Feminin, Marmiton, and chip.de. In the U.S., we now also support some more apps — Walmart, Tapatalk, and Fancy.
We’ve also translated our developer guidelines into eight additional languages: Chinese (Traditional), French, German, Italian, Japanese, Brazilian Portuguese, Russian, and Spanish.
If you’re interested in participating in App Indexing, and your content and implementation are ready, please let us know by filling out this form. As always, you can ask questions on the mobile section of our webmaster forum.
Finally, if you’re headed to Google I/O in June, be sure to check out the session on the “Future of Apps and Search”, where we’ll share some more updates on App Indexing.
Posted by Erik Hendriks, Software Engineer
Understanding web pages better
In 1998 when our servers were running in Susan Wojcicki’s garage, we didn’t really have to worry about JavaScript or CSS. They weren’t used much, or, JavaScript was used to make page elements… blink! A lot has changed since then. The web is full of rich, dynamic, amazing websites that make heavy use of JavaScript. Today, we’ll talk about our capability to render richer websites — meaning we see your content more like modern Web browsers, include the external resources, execute JavaScript and apply CSS.
Traditionally, we were only looking at the raw textual content that we’d get in the HTTP response body and didn’t really interpret what a typical browser running JavaScript would see. When pages that have valuable content rendered by JavaScript started showing up, we weren’t able to let searchers know about it, which is a sad outcome for both searchers and webmasters.
In order to solve this problem, we decided to try to understand pages by executing JavaScript. It’s hard to do that at the scale of the current web, but we decided that it’s worth it. We have been gradually improving how we do this for some time. In the past few months, our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on.
Sometimes things don’t go perfectly during rendering, which may negatively impact search results for your site. Here are a few potential issues, and – where possible, – how you can help prevent them from occurring:
-
If resources like JavaScript or CSS in separate files are blocked (say, with robots.txt) so that Googlebot can’t retrieve them, our indexing systems won’t be able to see your site like an average user. We recommend allowing Googlebot to retrieve JavaScript and CSS so that your content can be indexed better. This is especially important for mobile websites, where external resources like CSS and JavaScript help our algorithms understand that the pages are optimized for mobile.
-
If your web server is unable to handle the volume of crawl requests for resources, it may have a negative impact on our capability to render your pages. If you’d like to ensure that your pages can be rendered by Google, make sure your servers are able to handle crawl requests for resources.
-
It’s always a good idea to have your site degrade gracefully. This will help users enjoy your content even if their browser doesn’t have compatible JavaScript implementations. It will also help visitors with JavaScript disabled or off, as well as search engines that can’t execute JavaScript yet.
-
Sometimes the JavaScript may be too complex or arcane for us to execute, in which case we can’t render the page fully and accurately.
-
Some JavaScript removes content from the page rather than adding, which prevents us from indexing the content.
To make things easier to debug, we’re currently working on a tool for helping webmasters better understand how Google renders their site. We look forward to making it to available for you in the coming days in Webmaster Tools.
If you have any questions, please feel free to visit our help forum.
Posted by Erik Hendriks and Michael Xu, Software Engineers, and Kazushi Nagayama, Webmaster Trends Analyst
Making your site more mobile-friendly with PageSpeed Insights
Webmaster level: all
To help developers and webmasters make their pages mobile-friendly, we recently updated PageSpeed Insights with additional recommendations on mobile usability.
Poor usability can diminish the benefits of a fast page load. We know the average mobile page takes more than 7 seconds to load, and by using the PageSpeed Insights tool and following its speed recommendations, you can make your page load much faster. But suppose your fast mobile site loads in just 2 seconds instead of 7 seconds. If mobile users still have to spend another 5 seconds once the page loads to pinch-zoom and scroll the screen before they can start reading the text and interacting with the page, then that site isn’t really fast to use after all. PageSpeed Insights’ new User Experience rules can help you find and fix these usability issues.
These new recommendations currently cover the following areas:
- Configure the viewport: Without a meta-viewport tag, modern mobile browsers will assume your page is not mobile-friendly, and will fall back to a desktop viewport and possibly apply font-boosting, interfering with your intended page layout. Configuring the viewport to width=device-width should be your first step in mobilizing your site.
- Size content to the viewport: Users expect mobile sites to scroll vertically, not horizontally. Once you’ve configured your viewport, make sure your page content fits the width of that viewport, keeping in mind that not all mobile devices are the same width.
- Use legible font sizes: If users have to zoom in just to be able read your article text on their smartphone screen, then your site isn’t mobile-friendly. PageSpeed Insights checks that your site’s text is large enough for most users to read comfortably.
- Size tap targets appropriately: Nothing’s more frustrating than trying to tap a button or link on a phone or tablet touchscreen, and accidentally hitting the wrong one because your finger pad is much bigger than a desktop mouse cursor. Make sure that your mobile site’s touchscreen tap targets are large enough to press easily.
- Avoid plugins: Most smartphones don’t support Flash or other browser plugins, so make sure your mobile site doesn’t rely on plugins.
These rules are described in more detail in our help pages. When you’re ready, you can test your pages and the improvements you make using the PageSpeed Insights tool. We’ve also updated PageSpeed Insights to use a mobile friendly design, and we’ve translated our documents into additional languages.
As always, if you have any questions or feedback, please post in our discussion group.
Posted by Matthew Steele and Doantam Phan, PageSpeed Insights team
Creating the Right Homepage for your International Users
If you are doing business in more than one country or targeting different languages, we recommend having separate sites or sections with specific content on each URLs targeted for individual countries or languages. For instance one page for US and english-speaking visitors, and a different page for France and french-speaking users. While we have information on handling multi-regional and multilingual sites, the homepage can be a bit special. This post will help you create the right homepage on your website to serve the appropriate content to users depending on their language and location.
There are three ways to configure your homepage / landing page when your users access it:
- Show everyone the same content.
- Let users choose.
- Serve content depending on users’ localization and language.
Let’s have a look at each in detail.
Show users worldwide the same content
In this scenario, you decide to serve specific content for one given country and language on your homepage / generic URL (http://www.example.com). This content will be available to anyone who accesses that URL directly in their browser or those who search for that URL specifically. As mentioned above, all country & language versions should also be accessible on their own unique URLs.
Note: You can show a banner on your page to suggest a more appropriate version to users from other locations or with different language settings.
Let users choose which local version and which language they want
In this configuration, you decide to serve a country selector page on your homepage / generic URL and to let users choose which content they want to see depending on country and language. All users who type in that URL can access the same page.
If you implement this scenario on your international site, remember to use the x-default rel-alternate-hreflang annotation for the country selector page, which was specifically created for these kinds of pages. The x-default value helps us recognize pages that are not specific to one language or region.
Automatically redirect users or dynamically serve the appropriate HTML content depending on users’ location and language settings
A third scenario would be to automatically serve the appropriate HTML content to your users depending on their location and language settings. You will either do that by using server-side 302 redirects or by dynamically serving the right HTML content.
Remember to use x-default rel-alternate-hreflang annotation on the homepage / generic page even if the latter is a redirect page that is not accessible directly for users.
Note: Think about redirecting users for whom you do not have a specific version. For instance, French-speaking users on a website that has English, Spanish and Chinese versions. Show them the content that you consider the most appropriate.
Whatever configuration you decide to go with, you should make sure all the pages – including country and language selector pages:
- Have rel-alternate-hreflang annotations.
- Are accessible for Googlebot’s crawling and indexing: do not block the crawling or indexing of your localized pages.
- Always allow users to switch local version or language: you can do that using a drop down menu for instance.
Reminder: As mentioned in the beginning, remember that you must have separate URLs for each country and language version.
About rel-alternate-hreflang annotations
Remember to annotate all your pages – whatever method you choose. This will greatly help search engines to show the right results to your users.
Country selector pages and redirecting or dynamically serving homepages should all use the x-default hreflang, which was specifically designed for auto-redirecting homepages and country selectors.
Finally, here are a few useful reminders about rel-alternate-hreflang annotations in general:
- Your annotations must be confirmed from the other pages. If page A links to page B, page B must link back to page A, otherwise, your annotations may not be interpreted correctly.
- Your annotations should be self-referential. Page A should use rel-alternate-hreflang annotation linking to itself.
- You can specify the rel-alternate-hreflang annotations in the HTTP header, in the head section of the HTML, or in a sitemap file. We strongly recommend that you choose only one way to implement the annotations, in order to avoid inconsistent signals and errors.
- The value of the hreflang attribute must be in ISO 639-1 format for the language, and in ISO 3166-1 Alpha 2 format for the region. Specifying only the region is not supported. If you wish to configure your site only for a country, use the geotargeting feature in Webmaster Tools.
Following these recommendations will help us better understand your localized content and serve more relevant results to your users in our search results. As always, if you have any questions or feedback, please tell us in the internationalization Webmaster Help Forum.
Posted by Zineb Ait Bahajji and Gary Illyes, Webmaster Trends Analysts.
Webmaster Guidelines for sneaky redirects updated
Webmaster Level: AllRedirects are often used by webmasters to help forward visitors from one page to another. They are a normal part of how the web operates, and are very valuable when well used. However, some redirects are designed to manipulate or de…
Introducing our global Google+ page for webmasters
Webmaster Level: All
We’ve recently launched our global Google Webmasters Google+ page. Have you checked it out yet? Our page covers a plethora of topics:
- Official news and announcements like when our blog launched en français
- Tips such as how to handle 404s correctly
- Q&A office hours with our team
- Exclusive behind the scenes including planking puns and adventures with Googlebot
Follow us at google.com/+GoogleWebmasters and let us know in the comments what else you’d like to see on our page! If you speak Italian, Japanese, Russian or Spanish, be sure to also join one of our webmaster communities to stay up-to-date on language and region-specific news.
 |
| Hello from around the world! |
Posted by Mary Chen and Andrey Lipattsev, Webmaster Outreach Team
Surfacing your business’s contact and local info in Google
Webmaster level: All
Every day, searchers use Google to find information about businesses. Common queries include finding the phone number for customer service, the location of a business, and opening hours.
This information is typically found in a business’s location page or a “contact us” section of a company’s website. When Google correctly identifies these pages and is able to extract the relevant information from them, it is more likely to surface that information to searchers looking for the business.
Today we would like to share our recommendations for helping Google identify and surface this information.
Corporate phone numbers
National phone numbers for many companies are displayed prominently in Google Search results. For example, a searcher looking for Nest Labs’ customer service number will see:
Today, we are launching support for schema.org markup to help you specify your preferred phone numbers using structured data markup embedded on your website. Four types of phone numbers are currently supported:
- Customer service
- Technical support
- Billing support
- Bill payment
For each phone number, you can also indicate if it is toll-free, suitable for the hearing-impaired, and whether the number is global or serves a specific countries. Learn how to specify your national customer service numbers.
Recommendations for local business sites
Many people also turn to Google to find and discover local businesses, and the best information is often on a website’s contact us or branch locator page. These location pages typically include the address of the business, the phone number, opening hours, and other information.
Today we’re also introducing recommendations about the best way to build these location pages to make them easily accessible and understandable to Googlebot, and more importantly, Google’s users. Our recommendations cover both crawling, indexing and visual layout suggestions, as well as new structured data markup guidelines to help Google index pages more accurately.
In addition to building great location pages, businesses are encouraged to continue using Places for Business, which is a fast and easy way to update your information across Google’s service such as Google Maps, the Knowledge Graph and AdWords campaigns.
This blog post is only a brief summary of our recommendations for building location pages or branch locators. Please read the guidelines and, as always, please ask on our Webmaster Help forums if you have more questions.
Posted by Justin Boyan, Product Manager, Jonathan Sidi, Product Manager, Pierre Far, Webmaster Trends Analyst
App Indexing updates
Webmaster Level: Advanced
In October, we announced guidelines for App Indexing for deep linking directly from Google Search results to your Android app. Thanks to all of you that have expressed interest. We’ve just enabled 20+ additional applications that users will soon see app deep links for in Search Results, and starting today we’re making app deep links to English content available globally.
We’re continuing to onboard more publishers in all languages. If you haven’t added deep link support to your Android app or specified these links on your website or in your Sitemaps, please do so and then notify us by filling out this form.
Here are some best practices to consider when adding deep links to your sitemap or website:
- App deep links should only be included for canonical web URLs.
- Remember to specify an app deep link for your homepage.
- Not all website URLs in a Sitemap need to have a corresponding app deep link. Do not include app deep links that aren’t supported by your app.
- If you are a news site and use News Sitemaps, be sure to include your deep link annotations in the News Sitemaps, as well as your general Sitemaps.
- Don’t provide annotations for deep links that execute native ARM code. This enables app indexing to work for all platforms
When Google indexes content from your app, your app will need to make HTTP requests that it usually makes under normal operation. These requests will appear to your servers as originating from Googlebot. Therefore, your server’s robots.txt file must be configured properly to allow these requests.
Finally, please make sure the back button behavior of your app leads directly back to the search results page.
For more details on implementation, visit our updated developer guidelines. And, as always, you can ask questions on the mobile section of our webmaster forum.
Posted by Michael Xu, Software Engineer
More Precise Index Status Data for Your Site Variations
Webmaster Level: Intermediate
The Google Webmaster Tools Index Status feature reports how many pages on your site are indexed by Google. In the past, we didn’t show index status data for HTTPS websites independently, but rather we’d included everything in the HTTP site’s report. In the last months, we’ve heard from you that you’d like to use Webmaster Tools to track your indexed URLs for sections of your website, including the parts that use HTTPS.
We’ve seen that nearly 10% of all URLs already use a secure connection to transfer data via HTTPS, and we hope to see more webmasters move their websites from HTTP to HTTPS in the future. We’re happy to announce a refinement in the way your site’s index status data is displayed in Webmaster Tools: the Index Status feature now tracks your site’s indexed URLs for each protocol (HTTP and HTTPS) as well as for verified subdirectories.
This makes it easy for you to monitor different sections of your site. For example, the following URLs each show their own data in Webmaster Tools Index Status report, provided they are verified separately:
|
HTTP
|
HTTPS
|
The refined data will be visible for webmasters whose site’s URLs are on HTTPS or who have subdirectories verified, such as https://example.com/folder/. Data for subdirectories will be included in the higher-level verified sites on the same hostname and protocol.
If you have a website on HTTPS or if some of your content is indexed under different subdomains, you will see a change in the corresponding Index Status reports. The screenshots below illustrate the changes that you may see on your HTTP and HTTPS sites’ Index Status graphs for instance:
HTTP site’s Index Status showing drop
HTTPS site’s Index Status showing increase
An “Update” annotation has been added to the Index Status graph for March 9th, showing when we started collecting this data. This change does not affect the way we index your URLs, nor does it have an impact on the overall number of URLs indexed on your domain. It is a change that only affects the reporting of data in Webmaster Tools user interface.
In order to see your data correctly, you will need to verify all existing variants of your site (www., non-www., HTTPS, subdirectories, subdomains) in Google Webmaster Tools. We recommend that your preferred domains and canonical URLs are configured accordingly.
Note that if you wish to submit a Sitemap, you will need to do so for the preferred variant of your website, using the corresponding URLs. Robots.txt files are also read separately for each protocol and hostname.
We hope that you’ll find this update useful, and that it’ll help you monitor, identify and fix indexing problems with your website. You can find additional details in our Index Status Help Center article. As usual, if you have any questions, don’t hesitate to ask in our webmaster Help Forum.
Posted by Zineb Ait Bahajji, WTA, thanks to the Webmaster Tools team.
Introducing the new Webmaster Academy
Webmaster level: Beginner
Our Webmaster Academy is now available with new and targeted content!
Two years ago, Webmaster Academy launched to teach new and beginner webmasters how to make great websites. In addition to adding new content, we’ve now expanded and improved information on three important topics:
- Making a great site that’s valuable to your audience (Module 1)
- Learning how Google sees and understands your site (Module 2)
- Communicating with Google about your site (Module 3)
If you often find yourself overwhelmed by the depth or breadth of our resources, Webmaster Academy will help you understand the basics of creating a website and having it found in Google Search. If you’re an experienced webmaster, you might learn something new too.
Enjoy, learn, and share your feedback!
Posted by Mary Chen, Webmaster Outreach Team
App Engine IP Range Change Notice
Google uses a wide range of IP addresses for its different services, and the addresses may change without notification. Google App Engine is a Platform as a Service offering which hosts a wide variety of 3rd party applications. This post announces changes in the IP address range and headers used by the Google App Engine URLFetch (outbound HTTP) and outbound sockets APIs.
While we recommend that App Engine IP ranges not be used to filter inbound requests, we are aware that some services have created filters that rely on specific addresses. Google App Engine will be changing its IP range beginning this month. Please see these instructions to determine App Engine’s IP range.
Additionally, the HTTP User-Agent header string that historically allowed identification of individual App Engine applications should no longer be relied on to identify the application. With the introduction of outbound sockets for App Engine, applications may now make HTTP requests without using the URLFetch API, and those requests may set a User-Agent of their own choosing.
Posted by the Google App Engine Team
