Google Webmaster Central Blog - Official news on crawling and indexing sites for the Google index

Spring time design refresh!

Tuesday, May 12, 2009 at 3:30 PM

We've been listening to you at conferences, user studies, forums and blogs and we decided to start from the ground up with a brand new Webmaster Tools design! It was much needed, and the end result is beautiful in our eyes:



Highlights
  • One-stop Dashboard: We redesigned our dashboard to bring together data you view regularly: Links to your site, Top search queries, Sitemaps, and Crawl errors.
  • More top search queries: You now have up to 100 queries to track for impressions and clickthrough! In addition, we've substantially improved data quality in this area.
  • Sitemap tracking for multiple users: In the past, you were unable to monitor Sitemaps submitted by other users or via mechanisms like robots.txt. Now you can track the status of Sitemaps submitted by other users in addition to yourself.
  • Message subscription: To make sure you never miss an important notification, you can subscribe to Message Center notifications via e-mail. Stay up-to-date without having to log in as frequently.
  • Improved menu and navigation: We reorganized our features into a more logical grouping, making them easier to find and access. More details on changes.
  • Smarter help: Every page displays links to relevant Help Center articles and by the way, we've streamlined our Help Center and made it easier to use.
  • Sites must be verified to access detailed functionality: Since we're providing so much more data, going forward your site must be verified before you can access any features in Webmaster Tools, including features such as Sitemaps, Test Robots.txt and Generate Robots.txt which were previously available for unverified sites. If you submit Sitemaps for unverified sites, you can continue to do so using Sitemap pings or by including the Sitemap location in your robots.txt file.
  • Removal of the enhanced Image Search option: We're always iterating and improving on our services, both by adding new product attributes and removing old ones. With this release, the enhanced Image Search option is no longer a component of Webmaster Tools. The Google Image Labeler will continue to select images from sites regardless of this setting.
Go ahead, get started

The new user interface is available at http://www.google.com/webmasters/tools/new. The old user interface will continue to be available for a couple of weeks to give you guys time to adjust and provide feedback.

We did our best to get the product localized; however, you may notice a few missing translations in some areas of the user interface. We apologize for the inconvenience and when we switch everyone over in a couple of weeks, we'll fully support 40 languages. The one exception will be our Help Center, which will be available in 21 languages going forward.

We're really excited about this launch, and hope you are as well. Tell us what you think and stay tuned for more updates!

Introducing Rich Snippets

Webmaster Level: All

As a webmaster, you have a unique understanding of your web pages and the content they represent. Google helps users find your page by showing them a small sample of that content -- the "snippet." We use a variety of techniques to create these snippets and give users relevant information about what they'll find when they click through to visit your site. Today, we're announcing Rich Snippets, a new presentation of snippets that applies Google's algorithms to highlight structured data embedded in web pages.


Rich Snippets give users convenient summary information about their search results at a glance. We are currently supporting data about reviews and people. When searching for a product or service, users can easily see reviews and ratings, and when searching for a person, they'll get help distinguishing between people with the same name. It's a simple change to the display of search results, yet our experiments have shown that users find the new data valuable -- if they see useful and relevant information from the page, they are more likely to click through. Now we're beginning the process of opening up this successful experiment so that more websites can participate. As a webmaster, you can help by annotating your pages with structured data in a standard format.

To display Rich Snippets, Google looks for markup formats (microformats and RDFa) that you can easily add to your own web pages. In most cases, it's as quick as wrapping the existing data on your web pages with some additional tags. For example, here are a few relevant lines of the HTML from Yelp's review page for "Drooling Dog BarBQ" before adding markup data:


and now with microformats markup:


or alternatively, use RDFa markup. Either format works:


By incorporating standard annotations in your pages, you not only make your structured data available for Google's search results, but also for any service or tool that supports the same standard. As structured data becomes more widespread on the web, we expect to find many new applications for it, and we're excited about the possibilities.

To ensure that this additional data is as helpful as possible to users, we'll be rolling this feature out gradually, expanding coverage to more sites as we do more experiments and process feedback from webmasters. We will make our best efforts to monitor and analyze whether individual websites are abusing this system: if we see abuse, we will respond accordingly.

To prepare your site for Rich Snippets and other benefits of structured data on the web, please see our documentation on structured data annotations.

Now, time for some Q&A with the team:

If I mark up my pages, does that guarantee I'll get Rich Snippets?

No. We will be rolling this out gradually, and as always we will use our own algorithms and policies to determine relevant snippets for users' queries. We will use structured data when we are able to determine that it helps users find answers sooner. And because you're providing the data on your pages, you should anticipate that other websites and other tools (browsers, phones) might use this data as well. You can let us know that you're interested in participating by filling out this form.

What about other existing microformats? Will you support other types of information besides reviews and people?

Not every microformat corresponds to data that's useful to show in a search result, but we do plan to support more of the existing microformats and define RDFa equivalents.

What's next?

We'll be continuing experiments with new types (beyond reviews and people) and hope to announce support for more types in the future.

I have too much data on my page to mark it all up.

That wasn't a question, but we'll answer anyway. For the purpose of getting data into snippets, we don't need every bit of data: it simply wouldn't fit. For example, a page that says it has "497 reviews" of a product probably has data for 10 and links to the others. Even if you could mark up all 497 blocks of data, there is no way we could fit it into a single snippet. To make your part of this grand experiment easier, we have defined aggregate types where necessary: a review-aggregate can be used to summarize all the review information (review count, average/min/max rating, etc.).

Why do you support multiple encodings?

A lot of previous work on structured data has focused on debates around encoding. Even within Google, we have advocates for microformat encoding, advocates for various RDF encodings, and advocates for our own encodings. But after working on this Rich Snippets project for a while, we realized that structured data on the web can and should accommodate multiple encodings: we hope to emphasize this by accepting both microformat encoding and RDFa encoding. Each encoding has its pluses and minuses, and the debate is a fine intellectual exercise, but it detracts from the real issues.


We do believe that it is important to have a common vocabulary: the language of object types, object properties, and property types that enable structured data to be understood by different applications. We debated how to address this vocabulary problem, and concluded that we needed to make an investment. Google will, working together with others, host a vocabulary that various Google services and other websites can use. We are starting with a small list, which we hope to extend over time.


Wherever possible, we'll simply reuse vocabulary that is in wide use: we support the pre-existing vCard and hReview types, and there are a variety of other types defined by various communities. Sites that use Google Custom Search will be able to define their own types, which we will index and present to users in rich Custom Search results pages. Finally, we encourage and expect this space to evolve based on new ideas from the structured data community. We'll notice and reach out when our crawlers pick up new types that are getting broad use.

Google Trends for your website

Saturday, May 09, 2009 at 2:30 PM

Webmaster Level: All

In a recent post on the Official Google Blog, we mentioned our Google Trends gadget, and we thought it made sense to also post something here for all the webmasters that might be interested in having Trends on their website. Google Trends is a great way to see what's popular on the web -- people tend to search for what they care about -- and the Trends gadget makes it easy for you to put Trends on your website. Just cut and paste a small snippet of code, input your search terms, and you can show your readers how searches for Obama have changed during the last 30 days or who's the most popular American Idol contestant. So take a little piece of Google with you, and show your readers what's hot on the web.

More ways to engage your community

Friday, May 08, 2009 at 12:10 PM

Webmaster Level: All

Over the last few weeks, Google Friend Connect has added several new ways for you to strengthen the community that visits your site. These gadgets help to make your site more engaging and gives your visitors a new way to interact with your content and other visitors.

Here is a quick overview of these new gadgets:

Event gadget - Have an upcoming event you want to promote to your community? Embed this gadget on your site to let members get details about the event, see a map, indicate if they're coming, and see who else is attending. They can even add the event to their personal Google Calendars with just a click.



Polls gadget - Polls are a fun and easy way for your visitors to express themselves and a great tool for you to see what your users like. This gadget makes it easy to publish opinion polls and adds a social element by displaying the faces of the community members and friends who voted on each answer.



Get Answers gadget - Add the Get Answers gadget to your site to encourage visitors to ask questions of the community and answer questions posed by other community members. This gadget lets visitors browse questions, submit answers, and vote on answers they think are the best.


Comments gadget - Bring your site to life by adding the comments gadget to your pages. This gadget enables visitors to post comments and links to videos on your site. Visitors can even use the new translation feature to read comments in their preferred language.


Check out the Friend Connect gallery to see these gadgets along with all the other features Friend Connect has to offer. And keep an eye on the Social Web Blog for additional gadgets we'll soon be launching.

Survey says...

Friday, May 01, 2009 at 12:00 PM

Webmaster Level: All

Many thanks to the more than 1,600 people who filled out our survey in February. You gave us your feedback on the Webmaster Central Blog, Google Webmaster Tools, the Webmaster Help Forum, and our Webmaster Central videos on YouTube.

You told us what you like and want to see more of:
  • Webmaster Central gives users insight into Google: "[I like] being able to access, communicate, and see how my sites relate to Google."
  • Webmaster Central provides high quality information: "What I have enjoyed most of all, is reading Google's guidelines for webmasters, which is on-point with what I have been telling customers about SEO."
  • Webmaster Central collects several useful tools in one place: "It's an innovative central hub for all the tools supported and provided by the industry leader Google, for free."
We also learned about what you don't like and where we could be doing better. Our top finding is that beginner webmasters (about 20% of the survey respondents) are less satisfied than intermediate or advanced webmasters with Webmaster Central. Open-ended comments suggested that new webmasters want basic, less technical information from us. A common feedback that we received: "Many users like myself are not of the hi-tech, IT-savvy variety and prefer simplicity, whether we create a website for information or to generate revenue." Based on your responses, we've planned some new resources like a series of how-to videos especially for new webmasters (coming soon to YouTube).

We take your feedback seriously and will continue improving Webmaster Central and our other webmaster sites. Again, thanks for your participation in the survey. We want Webmaster Central to continue being a useful resource for you.

Google Webmaster Help Forums in more languages

Thursday, April 30, 2009 at 10:48 AM

Traditionally when we launch a new communication channel, we also give the shareholders a chance to introduce themselves. We did so when we opened webmaster help communities for European webmasters almost two years ago, and also more than a year ago, when we were able to expand and add groups in three more languages. Last December we were very happy to announce the re-launch of two of our Help Forums in a new and cool look and feel.

Today, we're happy to announce that we keep on increasing the global dialogue with webmasters, opening an Arabic and a Czech/Slovak Webmaster Help Forum. Furthermore, we would like to highlight the support we offer in Chinese, Japanese, and Korean. While we've offered support to Chinese webmasters for a little more than a year, the Japanese and Korean forums are only a few weeks old. Keeping with tradition, the guides monitoring our new forums would like to introduce themselves to the global webmaster family:

Arabic Webmaster Help Forum

مرحبا! My name is Adel and I'll be monitoring the Arabic Webmaster Help Forum. I'm originally from Beirut, Lebanon. After finishing computer science studies, I joined Google, some 18 months ago.

Besides working on search quality in Arabic and building a community on our forum, I enjoy traveling and listening to really loud heavy metal music; sometimes I get to do both at the same time! ;-)

I am looking forward to a lot of questions regarding Arabic Google Search and of course ranking and indexing issues on your sites to come. I hope I'll see you there soon!
- Adel

Czech/Slovak Webmaster Help Forum

Zdravím! I am Marcel, the Google Guide on the Czech/Slovak Webmaster Help Forum. I am originally from Slovakia. After graduating in New Media and Industrial Design, it took me some time and traveling around the globe before moving to Dublin and eventually joining Google some 3 years ago.

Ever since, I've been working in different teams. I was lucky to be part of the AdSense team where I participated in launching AdSense for Content for Czech and Slovak. Since my transition to Search Quality, I enjoy working on improving the quality of our natural search results in Czech, Slovak, and Polish.

Besides my work I have a few more passions, such as listening to live music in Irish pubs, challenging my colleagues in occasional Soulcalibur skirmishes on Playstation and testing burger places all over the world :-) If you want to discuss any of these topics or maybe something about your sites, please join the community. I am looking forward to meeting you there :-)
- Marcel

Chinese Webmaster Help Forum

你好! Hi from the Chinese Webmaster Help Forum team! The Chinese Webmaster Help Forum has received great support from webmasters since its launch in March 2008. In March 2009, the Chinese Webmaster Help Forum moved to a new system with many more user-friendly features for better information sharing. It has become a good platform for webmasters to share their knowledge of Google search and Webmaster Tools and to communicate with Google.

The Chinese Forum now has 6 Google Guides: Xiang (降龙十巴掌), Eric (趙錢孫李), Marina (小馬過河), Chris (城镇), Hyson (草帽路飞), and Fa (法人戴表). We are from many different provinces of mainland China. When not spending time in the forum, we enjoy playing ping-pong and foosball in the office. A few of us are huge video game fans. You may learn more about us when you participate in the forum :)

A big thank you to everyone who has taken part in forum discussions! We hope to see both familiar faces and newcomers join in the Chinese Webmaster Help Forum!
- Xiang (降龙十巴掌), Eric (趙錢孫李), Marina (小馬過河), Chris (城镇), Hyson (草帽路飞), and Fa (法人戴表)

Japanese Webmaster Help Forum

こんにちは! Hello from the Japanese Webmaster Help Forum team! Our names are Nao ( なお ), Kaede ( 楓 ), Haru ( ハル ), and Kyotaro. We are the four guides working in Google Search Quality for Japanese. We've just launched our forum on March 6th.

All of us were born in Japan and grew up here. Nao has also lived in Greece, the Netherlands, and New York. Haru is from the west side of Japan, which is known for its talkative culture and traditional Japanese comedy. Maybe you will read Haru's unique communication on our forum :)

As for our interests, we love eating and drinking! Between posting on the forum, we enjoy Google's excellent lunches and sweets a lot. After working, of course, we sometimes go out for a drink with our team members :) Kaede knows all the nice bars in Tokyo.

Nao and Kyotaro love Sumo wrestling. We've watched two tournaments this year with Googlers from other locations. Haru, of course, loves watching comedies!

We are really excited and happy to see many users joining our forum and sharing tips with each other. Looking forward to seeing you there!
- Nao ( なお ), Kaede ( 楓 ), Haru ( ハル ), and Kyotaro

Korean Webmaster Help Forum

안녕하세요! Hello everyone, my name is Joowon and I work in Google Search Quality for Korean. I was born in Germany and lived in Korea for a few years before moving to Hawaii, California and New York to attend high school and college. After all that traveling, I'm only fluent in Korean and English, with a bit of proficiency in Japanese. Some of the interests I've developed over the years are design, wine, cooking, yoga, and sustainability issues.

Currently I'm back in Seoul and enjoying the dynamic atmosphere here, with lots of interesting people and great food. The Korean Webmaster Help Forum was launched only a few weeks ago, and I'm very much looking forward to talking to all of you. See you in the forum!
- Joowon

Hello world! ;) I am Andrew and I am part of the Search Quality team in Seoul. I grew up in a port city in the southern part of Korea. Ironically, I don't eat seafood because it looks scary to me :( Many of my friends and colleagues love to make jokes about that, but I still don't eat any seafood yet. Playing drums, traveling and photography are my main interests. Currently I'm a drummer of "Spring Fingers", the first band of Google Korea, and we'll have our first concert at the end of April!

I love playing around with web technologies/APIs and find it very exciting to exchange information and ideas on the web. The Korean Webmaster Help Forum was recently launched and I hope to see you there!
- Andrew

If you're curious about our Webmaster Help Forums in other languages, please feel free to peak in. Here's a list of our currently monitored Webmaster Help Forums: Arabic, Chinese, Czech/Slovak, Dutch, Danish, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Swedish, and Turkish.

Research study of Sitemaps

Monday, April 27, 2009 at 1:20 PM

We've been tracking the growth of Sitemaps on the web. It's been just 2 years since Google, Yahoo and Microsoft co-announced the Sitemaps directive in robots.txt, and it is already supported in many millions of websites including educational and government websites! At the WWW'09 conference in Madrid, Uri Schonfeld presented his summer internship work studying Sitemaps from a coverage and freshness perspective. If you're interested in how some popular websites are using Sitemaps, and how Sitemaps complement "classic" webcrawling, take a look:


At Google, we care deeply about getting increased coverage and freshness of the content we index. We are excited about open standards that help webmasters open up their content automatically to search engines, so users can find relevant content for their searches.