This document describes how cost data upload works. It provides details of the feature and important considerations when uploading cost data to Google Analytics.
More
|
Introduction
The Google Analytics cost data upload feature allows you to take exported cost data for non-Google paid campaigns and upload it to Google Analytics. The cost data is then joined with user-interaction data, enabling detailed ROI analysis for all of your paid campaigns.
Custom Data Sources
A custom data source can represent a single or multiple external data source(s) in Google Analytics, such such as paid search, affiliate marketing, etc. Custom data sources are important because all of the metadata for uploaded data is associated with a custom data source, such as which profiles have access to uploaded data.
In addition to the custom data source, there is another related resource, a daily upload, that represents the actual data uploaded to Google Analytics for a particular date. Daily uploads are important because they include because it includes the uploaded data and related metadata such as: the custom data source it is associated with, date, and a history of recent changes.
Under the Google Analytics data model, a custom data source entity is a child entity of a property. One property can have multiple custom data sources. To understand how the custom data source and daily upload resources fit within the Google Analytics data model, the following diagram provides an example:
- A property has 3 profiles as children and Custom Data Source 1 is linked with two profiles — Profile 1 and Profile 3.
- Cost data has been uploaded to Custom Data Source 1 for dates 2012-04-03 and 2012-04-04.
- Data uploaded to Custom Data Source 1 will be available for report queries on either Profile 1 or Profile 3.
A custom data source entity has the following attributes:
| Attribute | Description |
|---|---|
| ID: | A unique identifier for the custom data source. This is
automatically generated when a custom data source is created. In the
Google Analytics web interface the ID name is UID. Using
the Management API, the ID name is customDataSourceId.
|
| Name: | The name of the custom data source. |
| Description: | The description of the custom data source. |
| Data Type(s): | The type of data that can be uploaded to the custom data source.
For cost data uploads the data type is "cost". |
| Linked Profile(s): | A list of profiles linked to a custom data source. Uploaded cost data will only be available to profiles that are linked to a custom data source. The set of profiles linked to a custom data source can be modified at any point by adding or removing profiles. At least one profile should be linked to a custom data source. |
To learn more about interacting with custom data source or daily upload resources using the Management API, read the following:
Creating a Custom Data Source
Before cost data can be uploaded to Google Analytics, a custom data source needs to be created. Currently, custom data sources can only be created through the Google Analytics web interface, under the admin section at the property level.
It is important to note that each custom data source has a unique ID and that this ID is a required to upload data. It can be retrieved using the Management API or through the Google Analytics web interface as shown in the following screenshot.
Once a custom data source is created and you have an ID, cost data can be uploaded using the Management API. For detailed instructions on how to create a custom data source, see the corresponding section in Measuring Performance Of Paid Campaigns.
Best Practices
Create a custom data source for each external cost data source. For example, you could create a custom data source for one paid search source, another one for an affiliate marketing source, and a third one for a display ad network source. This makes it easier to control which profiles have access to external data sources and if you need to delete data from one source, you can do so without affecting cost data uploads from other, unrelated sources. For example, if paid search and affiliate marketing cost data are uploaded to a single custom data source and the paid search cost data needs to be deleted, it would also cause the affiliate marketing cost data to be deleted. Maintaining separate custom data sources prevents this issue.
Cost Data Format and Constraints
In order for cost data to be uploaded to Google Analytics, it must be in a properly formatted CSV file, contain values for required dimensions and metrics, and meet certain file contraints. The sections below outline all of the dimensions and metrics available for upload, including those that are required, as well as the relevant CSV file and format constraints.
Dimensions & Metrics
There is a pre-defined set of dimensions and metrics, schema, that can be used for cost data upload. Only values that match this specific set of dimensions and metrics can be uploaded.
Dimensions
| Dimension | Required? | Description |
|---|---|---|
ga:source |
required | The campaign source. e.g. ad network, affiliate.
This should correspond to source identified with utm_source used in URLs with campaign tracking
parameters. |
ga:medium |
required | Identifies the medium related to the Cost Data, such as
cost-per-click (cpc), email etc.
This should correspond to medium identified with utm_medium used in URLs with campaign tracking
parameters. |
ga:campaign |
optional | The name of the campaign.
This should correspond to campaign name identified with utm_campaign used in tagged campaigns. |
ga:adwordsCampaignId* |
optional | The numeric ID of the campaign. Range of 0 to 232-1.
Google Analytics will parse this into a long. |
ga:adGroup |
optional | The name of the ad group. |
ga:keyword |
optional | The related keyword for the ad.
This should correspond to keywords identified with utm_term used in URLs with campaign tracking
parameters. |
ga:adMatchedQuery |
optional | The search query that triggered impressions of the ad. |
ga:adContent |
optional | Content used in the ad.
This should correspond to content identified with utm_content used in URLs with campaign tracking
parameters. |
ga:referralPath |
optional | The path of the referring URL. |
ga:adwordsCriteriaId* |
optional | The numeric ID of the ad criteria. Range of 0 to 232-1.
Google Analytics will parse this into a long. |
ga:adSlot |
optional | The location of the advertisement on the hosting page. |
ga:adSlotPosition* |
optional | The numeric ad slot position in which the ad appeared. Range of 0 to 232-1. Google Analytics will parse this into a long. |
ga:adDisplayUrl |
optional | The URL the ad displayed. |
ga:adDestinationUrl |
optional | The URL to which the ad referred traffic. |
Metrics
| Metric | Required | Description |
|---|---|---|
ga:adCost** |
required1 | The numeric cost for the campaign. Currency can not be set when
uploading cost data. The reporting currency will be based on currency
settings for the linked profile. Range of 0 to 264-1.
Google Analytics will parse this into a double. A decimal point is used as a delimiter between the whole and fractional portion of the value. The precision is up to 6 decimal places. |
ga:adClicks* |
required1 | The total number of times users clicked on the ad. Range of 0 to 232-1. Google Analytics will parse this into a long. |
ga:impressions* |
required1 | Total number of campaign impressions. Range of 0 to 232-1 . Google Analytics will parse this into a long. |
** Double should be formatted as (\d+) | (\d*\.\d+). No other formats are supported for double.
1 Each row uploaded to a custom data source requires that a value be included for at least one of the three available metrics. Any metric that is omitted will be assumed to have a value of 0.
CSV Format and Constraints
In addition to being properly formatted, the CSV file needs to satisfy a set of constraints before it can be considered valid for upload.
If any of the constraints below are violated or if there is a parsing error, the upload request will fail with error message(s). The API will report up to 10 errors in a single response to help users reduce the number of invalid uploads.
File Constraints
- Encoding: UTF-8
- File size limit: 5 MB
Header Constraints
- Cannot be empty or contain empty cells.
- Can only contain dimensions and metrics defined in the cost data schema
ga:sourceandga:mediumdimensions are required column headers.- At least one metric —
ga:adCost,ga:adClicks, orga:impressions— is required to be included in the header. Omitted metrics will be assumed to have a value of 0. - Every append for a particular upload date should have the same header as the first append. If headers do not match, you will receive an error response.
Row Constraints
- Non-empty values for the required dimensions:
ga:sourceandga:medium - A non-empty value for at least one of the following metrics:
ga:adCost,ga:adClicks, orga:impressions. Omitted metrics will be assumed to have a value of0.
Cell Formatting
- Cells can have a maximum length of 2048 characters.
- Empty values can be specified by including empty cell values.
E.g.,
source1,medium1,,100— The third cell value is empty. - Based on the header specified for a column, each corresponding cell value should adhere to the type and format listed in the cost data schema.
- Leading and trailing spaces for a cell will be stripped.
- To specify a comma within a cell, the value of the cell should be
inside double quotes. E.g.,
source1,"medium,ether",group1,100— The second cell value ismedium,ether. - To specify double quotes within a cell, in addition to start and end
double quotes, escape the double quote with another double quote.
E.g.,
source1,"medium""ether",group1,100— The second cell value ismedium"ether. - New line characters are not allowed within cell values.
Sample CSV Files
ga:source,ga:medium,ga:keyword,ga:adGroup,ga:adCost,ga:adClicks,ga:impressions
ad network,cpc,phone,Ad group #1,13.95,38,17882
ad network,cpc,pictures,Ad group #1,10.75,43,22377
ad network,cpc,phone,Ad group #1,5.14,32,11866
Because ga:adCost has been omitted, a value of 0 will be automatically used for this metric.
ga:source,ga:medium,ga:keyword,ga:adClicks,ga:impressions
ad network,cpc,phone,38,17882
ad network,cpc,pictures,43,22377
ad network,cpc,phone,5.14,32,11866
Cost Data Operations
After you've exported cost data from an external source, prepared it as a CSV file, and created a custom data source, the cost data is ready to be uploaded to Google Analytics. In addition to the upload operation, there are additional operations available to append additional cost data, or delete unwanted cost data.
The operations listed below require the authorized user to be an admin of the property. However, admin access is not required to view reports for profiles linked to custom data sources.
Upload/Append
Cost data is uploaded as a CSV file to a custom data source for a single date. Additional data can be appended for a given date, up to a maximum of 20 appends per date. For example, you might have multiple CSV files containing cost data that you've exported from a non-Google paid search provider for two separate campaigns for 2012-10-31. In this case, you would append the first CSV file to the custom data source for the date 2012-10-31 and then append the second CSV file to the same custom data source and date. It is important to plan ahead and be efficient so that you can upload all related daily cost data within the limit of 20 appends per date.
All appends for an upload date need to be completed within 10 days of the first append. Any append attempts after the 10 day period will return an error response. If that occurs, you will need to perform a reset for that date, and upload all cost data for that date again. For example, if the first append for an upload date takes place on August 14, additional appends for the upload date will be successful until August 24th, assuming the append limit of 20 appends per date is not reached first.
This operation shows up as APPEND in the recent changes
history for a an upload date.
Reset
Reset is an optional parameter that can be set during an upload. If
true, the upload effectively becomes a delete operation
followed by an upload operation. In other words, all of the cost data
for the upload date will be first deleted (i.e. reset) and then the CSV
file specified in the upload request will be uploaded to the custom data
source for the same date. This operation will also reset the 20 appends
per date limit described in the section above.
This operation shows up as RESET in the recent changes
history for a an upload date.
Delete
If you want to remove cost data for a date and a subsequent upload is not required, then a delete operation can be performed. Deleting will only remove cost data that you have previously uploaded. It does not affect or alter any of your existing Google Analytics data such as pageviews, goal completions, revenue, etc. This also means you don't have to worry about "carrupting" existing Google Analytics data when performing cost data uploads, since you can simply delete the cost data at any time.
This operation shows up as DELETE in the recent changes
history for a an upload date. Note that this operation only deletes the
cost data for an upload date and that the recent changes history is
maintained.
To learn more about cost data operations see:
- Measuring Performance Of Paid Campaigns — Explains how to upload cost data to Google Analytics.
- Upload method reference.
- Delete method reference.
Reporting
Once cost data has been uploaded to Google Analytics it can take up to 12 hours for it to be processed, after which it will be become available in reports for profiles that are linked to the custom data source. Reports can be queried using the The Core Reporting API or through the web interface using custom reports or the standard Cost Analysis Report, available under the Traffic Source section.
The cost data upload feature does not introduce any new dimensions or metrics. Therefore, there is no impact or change on how to carry out reporting through the web interface or Core Reporting API. However, there are additional considerations such as how to ensure cost data is properly joined with user-interaction data. This is covered in Best Practices.
For an example on reporting with cost data see Report Using The Core Reporting API Or Web Interface.
Best Practices
There are important considerations when planning your cost data upload strategy that can have an impact on reports and analysis:
URL Tagging and Naming Consistency
The primary benefit of uploading cost data to Google Analytics is that it can be joined with user-interaction data for corresponding non-Google paid campaigns. In order for the join to be successful, campaign tracking parameters used to tag URLs for paid campaigns must be consistent with cost data values uploaded to a custom data source. If the values do not match, cost data cannot successfully joined to user-interaction data in Google Analytics.
As an example, consider the following scenario:
A website owner runs an ad campaign on a search engine for the keyword
books. The URL of the ad is tagged with campaign tracking
parameters so that it is possible for the website owner to determine
which visits came from users who clicked on the ad for this campaign.
To properly join the cost data from the paid search campaign with the user-interaction data in Google Analytics, the website owner must ensure that cost data uploaded for the paid search campaign matches the values used for the campaign tracking parameters in the ad URL.
Here is the URL used for the ad:
http://www.examplepetstore.com/page.html?utm_source=ad%2Bnetwork&utm_medium=cpc&utm_campaign=summer_sale&utm_term=books&utm_content=ad_variation1
The following table shows that the values used in the ad URL should match the cost data values uploaded to the custom data source in Google Analytics:
| Campaign Tracking Parameter added to URL | Corresponding dimension name for cost data upload | Value that should be used |
|---|---|---|
utm_source |
ga:source |
ad network |
utm_medium |
ga:medium |
cpc |
utm_campaign |
ga:campaign |
summer_sale |
utm_term |
ga:keyword |
books |
utm_content |
ga:adContent |
ad_variation1 |
To learn more about campaign tracking parameters see:
- Campaign Tracking Parameters
- Dimensions and Metrics for cost data upload.
- Measuring Performance Of Paid Campaigns — An example of uploading cost data to Google Analytics
Mapping External Sources to Cost Data Schema
External cost data sources may not use the same naming conventions as Google Analytics for dimensions and metrics. It is up to you to determine how the data from these sources should be mapped to the dimensions and metrics available in the cost data schema. Once you've created a custom data source, uploading consistent daily data will lead to consistent reporting.
Historical Upload vs. Ongoing Uploads
It is possible to upload cost data for past dates (as far back as 2005-01-01). If you have relevant historical cost data, a simple strategy may be to first upload historical data as a one-time operation. Then continue with daily uploads for new data. Plan accordingly based on the Quota Policy since the initial historical upload may require a large number of sequential uploads.
Appending Incremental Cost Data vs Aggregate Cost Data
The recommended approach to uploading cost data to Google Analytics is to wait until the end of the day and perform one upload of aggregate data for each cost data source.
The data upload feature is designed for uploads of aggregated daily data and not incremental, intra-day, cost data uploads. If for some reason you cannot upload daily aggregate data and will have to perform multiple appends for an upload date and external data source, then you should make sure to upload incremental cost data, not aggregate counts. This is because when querying for reports, Google Analytics will aggregate values over the report period. If you want to do intra-day uploads but cannot easily get incremental values, one option is to use aggregate cost data but perform a reset with each upload.
For example, consider a scenario of intra-day cost data uploads where the following aggregate data is retrieved from an external data source:
| Time of intra-day data pull from external source | Aggregate values | Incremental values |
|---|---|---|
| Start-of-Day | cost = $0, clicks = 0, impressions = 0 | cost = $0, clicks = 0, impressions = 0 |
| Mid-Day | cost = $2, clicks = 2, impressions = 10 | cost = $2, clicks = 2, impressions = 10 |
| End-of-Day | cost = $5, clicks = 6, impressions = 60 | cost = $3, clicks = 4, impressions = 50 |
In this scenario, if aggregate values are uploaded throughout the day to a Custom Data Source in Google Analytics by making multiple append requests, the report for this period would return values of $7, 8, and 70 for cost, clicks, and impressions. This would be incorrect. Instead, the incremental values should be appended or another option would be to perform a reset and upload with aggregate values. Either method would result in the correct reporting of cost, clicks, and impressions, which is $5, 6, and 60.
Resources
- Measuring Performance Of Paid Campaigns — Learn about the steps required to upload cost data to Google Analytics.
- See the Management API quota policy for details on upload request quota.
- The API will report up to 10 errors in a single response to help users reduce the number of invalid uploads. For a list of error messages, see HTTP status code and reason.
- Learn about the difference between Direct vs Resumable Upload