Thursday, December 22, 2011 at 11:01 AM
Webmaster level: AdvancedFor all the developers who have expressed interest in getting programmatic access to the search queries data for their sites in Webmaster Tools, we've got some good news. You can now get access to your search queries data in CSV format using a open source Python script from the webmaster-tools-downloads project. Search queries data is not currently available via the Webmaster Tools API, which has been a common API user request that we're considering for the next API update. For those of you who need access to search queries data right now, let's look at an example of how the search queries downloader Python script can be used to download your search queries data and upload it to a Google Spreadsheet in Google Docs.
Example usage of the search queries downloader Python script
1) If Python is not already installed on your machine, download and install Python.
2) Download and install the Google Data APIs Python Client Library.
3) Create a folder and add the downloader.py script to the newly created folder.
4) Copy the example-create-spreadsheet.py script to the same folder as downloader.py and edit it to replace the example values for “website,” “email” and “password” with valid values for your Webmaster Tools verified site.
5) Open a Terminal window and run the example-create-spreadsheet.py script by entering "python example-create-spreadsheet.py" at the Terminal window command line:
python example-create-spreadsheet.py
If you just want to download your search queries data in a .csv file without uploading the data to a Google spreadsheet use example-simple-download.py instead of example-create-spreadsheet.py in the example above.
You could easily configure these scripts to be run daily or monthly to archive and view your search queries data across larger date ranges than the current one month of data that is available in Webmaster Tools, for example, by setting up a cron job or using Windows Task Scheduler.
An important point to note is that this script example includes user name and password credentials within the script itself. If you plan to run this in a production environment you should follow security best practices like using encrypted user credentials retrieved from a secure data storage source. The script itself uses HTTPS to communicate with the API to protect these credentials.
Take a look at the search queries downloader script and start using search queries data in your own scripts or tools. Let us know if you have questions or feedback in the Webmaster Help Forum.
14 comments:
Excellent - been waiting for this. Just curious to know why Google always seems to prefer Python over PHP
Thanks a such a helful holiday gift provided by Webmaster team. I think it will halp us more to analyse and improve our website for better user experience.
What a great Christmas present. Thank you so much for bringing great news Jonathan!
Schedule in Windows 7 http://windows.microsoft.com/en-US/windows7/schedule-a-task
Very Thanks!
+1 for transparency!
it is really great for us, and thanks for taking the time to create it.. I know there will more features coming up..
Can we have a feature of rank checker of our keyword in the webmaster it self..
I know we have queries which shows our position but we can not check particular keyword position in search engine of our website..
so if you guys can develop it will be very great for us.
Thanks again for developing this feature
Really great addition guys. Thanks! As an FYI to everyone else, I've found you can change the selected_downloads in the script from "TOP_QUERIES" to "TOP_PAGES" to pull you top pages report instead.
For those who like to use this to download the data for all their sites at once, I've changed the scripts somewhat: http://www.webkruscht.com/2011/downloading-data-from-google-webmaster-tools
This is really a great Christmas present. With a little help of JohnMu I've been able to pimp those scripts so their output becomes configurable with dates, regions, query types, pages or queries, etc.
Please add the mapping for TOP_QUERY_CHART and TOP_PAGES_CHART to the JSON list of downloadable files - this would really improve the tool.
Hi I'm stuck at 4) where example-create-spreadsheet.py is executed. There I get thid output:
Traceback (most recent call last):
File "C:\Python27\Lib\gdata-2.0.15\test2\exa
in
downloader.DoDownload(website, selected_do
File "C:\Python27\Lib\gdata-2.0.15\test2\dow
d
sites_json = json.loads(available)
File "C:\Python27\lib\json\__init__.py", lin
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line
obj, end = self.raw_decode(s, idx=_w(s, 0)
File "C:\Python27\lib\json\decoder.py", line
raise ValueError("No JSON object could be
ValueError: No JSON object could be decoded
The gdata python client works I've tested it with the example code from the installation HowTo. I've modified email password and website.
Does anyone has an idea?
PHP support?
waw .. incredible! Thanks for the info. Of course this is good also to me so that my blogging activities more smoothly.
When checking the site in the search engines I found this "Block all best-online-shopping-deals.com results". What does it mean, and how can I stop it?
D.
Post a Comment