YQL Tutorials

First YQL Application

Time Required: 10 min.

Introduction

This tutorial shows how to create a simple Web application that uses YQL to fetch an RSS feed. The call to YQL is made within an HTML script tag, and the returned JSON response is parsed with JavaScript.

Setting Up the Example
  1. From your Web server, create a new file called yql_news_app.html.
  2. Copy the HTML below into yql_news_app.html. The src attribute in the second script tag is given an empty value intentionally. We will assign a YQL statement to src later.

  3. Run the example query get rss feed from yahoo top stories in the YQL Console. Select the JSON radio button and click TEST.

    You should see the returned JSON response on the Formatted View tab.

  4. In the YQL statement, replace the word "title" with an asterisk "*". Click TEST.

    The asterisk will return all of the fields (rows) of the response.

  5. From the returned JSON response, find the results object. Notice that within the results object, there is the array item containing objects with the title, link, and description of each news article.
  6. Click Copy URL. From yql_news_app.html, paste the URL into the src attribute of the second script tag as seen below:

    Notice that the YQL statement has been URL-encoded and the format parameter specifies that the response be in JSON.

  7. At the end of the URL, replace the callback value 'cbfunc' with 'top_stories'.

    The new callback function top_stories will be called as soon as YQL returns the response and have access to the returned JSON.

  8. Point your browser to yql_news_app.html. Your YQL News Application should resemble the figure below.

    YQL News Application

    Figure 2.1. YQL News Application


  9. (Optional) Use almost the exact same code to create an application for finding sushi restaurants in San Francisco.

    Replace the YQL statement for the Yahoo! Top Stories in your YQL News Application with the example query find sushi restaurants in san francisco. Be sure that the format is JSON and that the name of the callback function is correct.

  10. Make a few changes below to the function top_stories as seen below:

  11. The YQL News Application is now the YQL Sushi Restaurant Finder Application.
    YQL Sushi Restaurant Finder Application

    Figure 2.2. YQL Sushi Restaurant Finder Application


Source Code
YQL Top News Application

Creating YQL Open Data Tables

Time Required: 15 min.

Introduction

This tutorial shows you how create a YQL Open Data Table for the Bay Area Region Transit (BART) Real Time ETA feed. You will then use the YQL console to run a YQL query using the Open Data Table that you created.

What Are Open Data Tables?

YQL already contains an extensive list of built-in tables for you to use that cover a wide range of Yahoo! Web services and access to off-network data. Developers can now add to this list of built-in tables by creating their own Open Data Tables. YQL allows you to create and use your own table definitions, enabling YQL to bind to any data source through the SQL-like syntax and fetch data. Once an Open Data Table is created, anyone can use these definitions in YQL.

By creating an Open Data Table for the feed, you can then use YQL to make queries on the data from the feed and mash up other data from existing YQL tables with the social data. For more information about Open Data Tables, see Using YQL Open Data Tables.

Creating and Using Your Open Data Table

  1. Copy bart.xml to your Web server. The file bart.xml defines the Open Data Table, which allows the YQL Web service to access data from the BART ETA feed. For a more detailed explanation, see A Closer Look at the Code.
  2. Go to the YQL console.
  3. From the YQL console, type the YQL statement below in the "Your YQL Statement" text area and click the button TEST. Be sure to replace "your_domain_name" with the domain name of your Web site.

    USE "http://your_domain_name/bart.xml" AS bart_table; SELECT * FROM bart_table;

  4. From the "Formatted View" tab, you should see results similar to those below:

  5. Try a few more YQL statements using the Open Data Tables. Below are a few sample YQL statements.

    • This statement will get you the station name and ETAs for any train going to the San Francisco International Airport.

      USE "http://your_domain_name/bart.xml" AS bart_table; SELECT name, eta FROM bart_table WHERE eta.destination LIKE "%SF%"

    • This statement gets the station name, the destination of the train, and the ETA to different destinations from the 24th and Mission Street station.

      USE "http://your_domain_name/bart.xml" AS bart_table; SELECT name, eta.destination, eta.estimate FROM bart_table WHERE name LIKE "%24%"

A Closer Look at the Code

This section will examine in greater detail at the Open Data Table and the YQL queries that are used in this tutorial.

Open Data Table

The Open Data Table must conform to the XML schema table.xsd, which is given as the XML namespace attribute for the element table. The element meta has child elements that provide general information about the Open Data Table such as the author, the location of the documentation, and a sample query, as shown below.

The element bindings allows YQL to map the data source so that information can be queried. The element select defines what repeating element of the XML from the data source will act as a row of a table with the attribute itemPath, which is <root><station>...</station> (given as root.element) in the code below. A good example of a repeating element in XML would be the element entry in an Atom 1.0 feed.

YQL Query

You invoke an Open Data Table with the verbs USE and AS. YQL fetches the Open Data Table defintion pointed to by the verb USE and then the verb AS creates an alias to that definition.

USE "http://your_domain_name/bart.xml" AS bart_table;

The alias bart_table can now be accessed with a YQL statement. The simple YQL query below returns all the rows from the BART feed.

SELECT * FROM bart_table;

Source Code

Executable JavaScript in Open Data Tables

Time Required: 15-20 min.

Introduction

This tutorial shows how to include JavaScript in an Open Data Table definition that is executed by the YQL Web service. The JavaScript in the Open Data Table for this tutorial includes external libraries and uses OAuth to make an authorized call to the Netflix API. With the Netflix Open Data Table, you can then run YQL statements from the YQL console or create an Open Application that searches and displays movies from the Netflix catalog.

Including JavaScript in an Open Data Table definition allows you to do the following:

  • Access external libraries
  • Make requests to Web services
  • Use E4X to structure the response in XML
  • Dynamically control the response with conditional logic
  • Perform Web authorization
Prerequisites
Executable JavaScript

JavaScript is placed in the element execute of an Open Data Table definition. When a SELECT statement calls an Open Data Table definition having the element execute, YQL executes the JavaScript on the backend and returns the response generated by the JavaScript. Open Data Table definitions having the element execute must return a response formed by the JavaScript.

In contrast, Open Data Tables not having the element execute make GET requests to the URI defined in the element url and return the response from that URI resource. We'll now look at a couple of examples of Open Data Tables with and without the element execute to illustrate the difference.

Example of Open Data Table Definition without JavaScript

The Open Data Table definition below does not include the element execute, so YQL makes a GET request to the URI defined in the element url: http://www.bart.gov/dev/eta/bart_eta.xml

Example of Open Data Table Definition with JavaScript

Because of the presence of the element execute, YQL executes the JavaScript and returns the response assigned to response.object, which will be discussed in Creating the Response.

A Closer Look at the Code

JavaScript in Open Data Tables definitions have access to the three global objects y, request, and response. From these global objects, methods can be called to include external libraries, perform 2-legged authorization , make requests to Web services, run YQL queries, convert data structures, such as XML to JSON, and more. We'll be looking at the methods used in this tutorial that allow you to include external libraries, make requests to Web services, and extract the results from a returned response.

To read more about the JavaScript global objects and the available methods, see the JavaScript Objects and Methods Reference.

Including External Libraries

The global object y includes the method include that allows you to reference external JavaScript libraries from an Open Data Table definition. Because calls to the Netflix API must be authorized with OAuth, you include external libraries to handle the authorization and sign the request as shown in the code below:

Making Requests

To make a request to a Web service, you use the method get from either the global object request or from an instance of request; the instance is created by calling the method rest from the global object y. Before looking at the code used in the Open Data Table definition for this tutorial, let's look at a couple of simple examples of using both the global object and an instance of request.

This example creates an instance of request by passing the URI resource to the method rest. From the instance, the method get can make the request to the Web service.

You can also call the method get directly from the global object request. The element url holds the URI to the Web service, which is stored in request.url.

You can also pass in parameters and define the content returned when making a request to a Web service, which is needed to make authorized requests, such as the request to the Netflix API in this tutorial.

To get the response from the Netflix API, the content type must be defined as 'application/xml', and the OAuth authorization header must be passed. The code snippet below also uses the objects response and response.object to extract results from the returned response and create the response that YQL returns. We'll examine the object response in Getting and Creating a Response.

Getting and Creating the Response

Extracting Data from the Returned Response

To get the data (results) from the returned response, you reference the object response . From this earlier code example, you can see the dot operator being used to reference the results of the returned response.

We also saw this example of using the response object to get the data from the Netflix Web service.

Creating the Response

When including JavaScript in an Open Data Table, the response from the YQL Web service is determined by the data that is assigned to the object response.object. You can use the returned value from a Web service or create your own with JSON or E4X.

In this code snippet, an XML literal is created with E4X and then assigned to response.object. To interpolate the variable into the XML literal, enclose the variable in braces.

The code from the tutorial assigns the returned XML response to response.object as seen below. If there is an error, a JSON object holding error message is returned.

    Paging Results

    The element paging lets you have more control over the results returned from the YQL query. We'll look in detail at the paging used in the Netflix Open Data Table that is shown below. For more information about paging, see the Open Data Tables Reference and Paging Results.

    The value 'offset' for the attribute model states that the YQL query can state an offset from the start of the results. The ability to get an offset from a result set depends on the source of the data, which in the tutorial is the Netflix API. Be sure to verify that your data source allows retrieving data from an offset when you create future Open Data Tables.

    <paging model="offset">

    The default offset is set by the attribute default in the element start, which is 0 or no offset in this example.

    <start id="start_index" default="0" />

    The maximum number of results that can be returned by a YQL query on this table is 100, which is defined by the attribute max.

    <pagesize id="max_results" max="100" />

    The attribute defaut in the element total states the default number of results returned for each page.

    <total default="20" />

    Setting Up This Example

    To set up the Netflix Open Data Table and run YQL queries:

    1. Copy netflix_table.xml to your Web server. The file netflix_table.xml defines the Open Data Table, which allows the YQL Web service to access data from the NetFlix API. For a more detailed explanation, see A Closer Look at the Code.
    2. If you do not have Netflix API keys, register for a Netflix developer account and apply for an API key.
    3. Go to the YQL Console. Make sure that the URL of the YQL Console begins with https.
    4. Select the first 20 results (see Paging Results) for movies that match the key word "Rocky". Be sure to use your Consumer Key and Consumer Secret from your Netflix account for the values for ck and cs.

      USE "http://your_domain/netflix.xml" AS netflix_table; SELECT * FROM netflix_table WHERE term="rocky" AND ck="your_consumer_key" AND cs="your_consumer_secret"

    5. In this YQL statement, you ask for the first 10 results for movies, and the results will include the film title, the URL of the box art image, and the average rating.

      USE "http://yourdomain/netflix.xml" AS netflix_table; SELECT catalog_title.title, catalog_title.box_art.medium, catalog_title.average_rating FROM netflix_table(10) WHERE term="rocky" AND ck="your_consumer_key" AND cs="your_consumer_secret"

    6. You can also specify an offset for the returned results. The YQL statements below returns the five titles and ratings for Star Trek movies with average ratings over 3.0. The results are sorted by the rating in descending order, and the offset is five.

      USE "http://yourdomain/netflix.xml" AS netflix_table; SELECT catalog_title.title.regular, catalog_title.average_rating FROM netflix_table(5,5) WHERE term="star trek" AND ck="your_consumer_key" AND cs="your_consumer_secret" AND catalog_title.average_rating > "3.0" | sort(field="catalog_title.average_rating") | reverse()

    7. Optional: Create a Open Application that uses the Netflix Open Data Table

      Copy yql_netflix_app.html to your Web server.

    8. Edit yql_netflix_appl.html and insert your Netflix keys and the location of your Netflix Open Data Table in the code as shown below:

    9. From My Projects, create a new Open Application.
    10. In the Application Editor, specify the URL to netflix_app.html in the "Application URL" field.
    11. Click "Preview" to see the canvas view. Your application will have a text field for entering a movie title. Enter "Rocky" and click the button "Find Movie" to see results similar to those in the figure below.

    12. Try modifying the YQL statement in the code to more precisely return the data that you want from the Netflix API. Take a look at the XML response from a catalog/titles request to the Netflix API to see the available data and how it is structured.
    Source Code
    Netflix Open Data Table

    Netflix Application

    Table of Contents