Data Recipes

Generate a Graph or Map of Rainfall Totals for a Given Date Range / Location Using Giovanni

1. Search and select an interested variable

  • Go to the Giovanni website using the web browser of your choice: http://giovanni.gsfc.nasa.gov.
  • Type IMERG in the Keyword field and press the Search button. The resulting webpage should look like Figure 1 below.

The GIOVANNI webpage seen while searching for IMERG
Figure 1. The GIOVANNI webpage seen while searching for IMERG

There are currently 17 IMERG variables that will show up when you search. The various datasets all have a horizontal resolution of 0.1° x 0.1° and temporal resolutions of either 30 minutes or 1 month. Click here for a table with user-friendly descriptions of all the variables.

  • We will use the variable labeled “Multi-satellite precipitation estimate with gauge calibration – Final Run (recommended for general use)” as the focus of this tutorial. The blue oval in Figure 1 indicates this dataset for reference.
    • You can also plot monthly averaged data by choosing the 1st variable in the list above.
    • You can sort any of the parameters in the columns displayed above by clicking the column name.
  • Select the checkbox next to the variable of interest. This will automatically cause Giovanni to set boundaries for the Date Range so that you cannot select times without data.

2. Select plot type

Choose the type of plot you'd like to make. This tutorial will focus on making a standard “Time Averaged Map” that is selected by default. There is a red oval in Figure 1 around the menus that are used to select a type of plot.

3. Select temporal and spatial range

  • Choose the date range for the time averaged map. You can either type the date in or click on the small calendar icon next to the dates. For this tutorial we will choose August 1, 2014 at 00:00 UTC – August 2, 2014 at 23:59 UTC to plot the average of two days of data. Your date range should look like Figure 2 below.
     

Figure 2. The date range used for this tutorial.

Figure 2. The date range used for this tutorial.

  • Select the region over which you'd like to plot the data. IMERG data are currently available between 60°S – 60°N so Giovanni will not plot data beyond those bounds regardless of what is selected.
  • You can choose to only plot the data over a specific geographic region by either manually entering the latitude and longitude boundaries or by clicking the Show Map or Show Shapes button. For
  • This tutorial we will choose to plot the data over the United States shapefile.
  • Click on the Show Shapes button. Then click on Countries. Scroll down and click on the United States selection and then click the Done button in the lower-left corner of the shapes window. This is shown in Figure 3.

Figure 3. The shape selection window.

Figure 3. The shape selection window.

4. Make Plot

  • Now it's time to plot the data. Simply click the green Plot Data button in the lower right-hand corner of the page. 

You'll notice that precipitation over the United States is only plotted to 60°N. You can pan and zoom the map using the arrows and the + and – buttons in the upper-left corner of the map (or the mouse and mouse wheel). Figure 4 shows the resulting map focused over the continental United States.

Figure 4. The resulting map showing IMERG data focused on the continental United States.

Figure 4. The resulting map showing IMERG data focused on the continental United States.

5.  Download image or data

  • You can now either view the map within Giovanni or download the map in GeoTIFF or PNG format by pressing the Image button in the upper right hand corner of the image (circled in red in Figure 4).
  • You can also download a netCDF file containing the data plotted in the figure. To do this, click on the Downloads link on the right-hand side of the page under the History heading. You will see a link to download the netCDF file on the following page.

Discussion: 

For more information on GPM data at the GES DISC, see: http://disc.sci.gsfc.nasa.gov/gpm

To view a list of all of the GPM data holdings at GES DISC, see: http://disc.sci.gsfc.nasa.gov/uui/#/search/IMERG

Other Map Customizations

Giovanni allows you to make certain customizations to the plots you generate. This optional section will teach you how to change the color scale range, color palette, and data smoothing options.

Color Scale Range

Giovanni automatically chooses a color scale appropriate for the image based on the minimum and maximum values of the data being plotted. You can see that for this image Giovanni automatically chose the range of 0 mm/hr – 2.988 mm/hr.

To change this click on the Options button next to the Image button that is circled in Figure 4. In the window that pops up called Map Options you can change the minimum and maximum values.

Change Color Palette

The default blue to red color palette might not be appropriate for all users. To change this, click on the Options button next to the Image button that is circled in Figure 4. In the window that pops up called Map Options you can change the palette.

Change Data Smoothing

The data smoothing option, in the same Map Options window described above, allows you to apply GrADS' gxout shaded and bicubic smoothers to the data. Although this can be handy for low-resolution data, the effects on the 0.1° IMERG data are negligible.

Figure 5 below shows the resulting map created by setting the data range to 0 – 2 mm/hr, the color palette to White-Blue-Red-Yellow and the smoothing to On.

Figure 5. Map of IMERG data that uses all three optional tweaks.

Figure 5. Map of IMERG data that uses all three optional tweaks.

Read GPM IMERG Data Using Python

Overview: 
This recipe shows how to read data from the Global Precipitation Measurement (GPM) mission's IMERG dataset using Python.

Best When: 
The user wants to read in GPM IMERG data using Python

Example: 
Example data: GPM Level 3 IMERG Monthly 0.1 x 0.1 degree Precipitation (GPM_3IMERGM) on July 2015.

Estimated Time to complete the following procedures: 20 minutes

Requirements: Python and the free packages: numpy, matplotlib, basemap, and h5py. Matplotlib and basemap are only needed for plotting.

 

Procedure: 

1. Download the data

In this recipe, we use data search engine, Mirador, but you may obtain the data by using other data services at GES DISC.

  • In a web browser, go to: http://mirador.gsfc.nasa.gov.
  • In the Keyword field, enter IMERG and press Search GES-DISC.
  • The first result shows monthly precipitation estimates, GPM Level 3 IMERG Monthly 0.1 x 0.1 degree Precipitation (GPM_3IMERGM). Click on View Files (see red arrow in Figure 1).
  • Click on the first file name in the list to download it. It is called, 3B-MO.MS.MRG.3IMERG.20150701-S000000-E235959.07.V03D.HDF5.

Note: this recipe works for any IMERG data, not just monthly estimates.

Example of getting IMERG data from Mirador

Figure 1: Example Mirador search results for IMERG.

 

2.    Run the following Python script:

# This is a sample script that reads and plots the IMERG data.

from mpl_toolkits.basemap import Basemap, cm

import matplotlib.pyplot as plt

import numpy as np

import h5py as h5py

 

dataset = h5py.File('/path/to/data/3B-MO.MS.MRG.3IMERG.20150801-S000000-E235959.08.V03D.HDF5', 'r') # Change this to the proper path

 

precip = dataset['Grid/precipitation'][:]

precip = np.transpose(precip)

 

theLats= dataset['Grid/lat'][:]

theLons = dataset['Grid/lon'][:]

 

# Plot the figure, define the geographic bounds

fig = plt.figure(dpi=300)

latcorners = ([-60,60])

loncorners = ([-180,180])

 

m = Basemap(projection='cyl',llcrnrlat=latcorners[0],urcrnrlat=latcorners[1],llcrnrlon=loncorners[0],urcrnrlon=loncorners[1])

 

# Draw coastlines, state and country boundaries, edge of map.

m.drawcoastlines()

m.drawstates()

m.drawcountries()

 

# Draw filled contours.

clevs = np.arange(0,1.26,0.125)

 

# Define the latitude and longitude data

x, y = np.float32(np.meshgrid(theLons, theLats))

 

# Mask the values less than 0 because there is no data to plot.

masked_array = np.ma.masked_where(precip < 0,precip)

 

# Plot every masked value as white

cmap = cm.GMT_drywet

cmap.set_bad('w',1.)

 

# Plot the data

cs = m.contourf(x,y,precip,clevs,cmap=cmap,latlon=True)

 

parallels = np.arange(-60.,61,20.)

m.drawparallels(parallels,labels=[True,False,True,False])

meridians = np.arange(-180.,180.,60.)

m.drawmeridians(meridians,labels=[False,False,False,True])

 

# Set the title and fonts

plt.title('August 2015 Monthly Average Rain Rate')

font = {'weight' : 'bold', 'size' : 6}

plt.rc('font', **font)

 

# Add colorbar

cbar = m.colorbar(cs,location='right',pad="5%")

cbar.set_label('mm/h')

 

plt.savefig('testIMERGmap.png',dpi=200)

 

Display GPM / TRMM HDF Data Files Using THOR

Overview

Orbit Viewer THOR is a tool for displaying the satellite data files in the archive of the Tropical Rainfall Measuring Mission (TRMM). THOR stands for “Tool for High-resolution Observation Review”. It is a point-and-click program written in IDL that  runs on Linux, Mac OS X, and Windows. This viewer enables you to display on a map of the Earth TRMM observations at the full instrument resolution.  Even if you plan on reading and analyzing GPM HDF5 files using your own IDL programs, it can help to have the THOR data viewer installed to do quick checks on the HDF5 files you are analyzing.

Full THOR Install Documentation: http://pps.gsfc.nasa.gov/thorrelease.html

Instructions

  1. Download the THOR data viewer .zip file from PPS: ftp://gpmweb2.pps.eosdis.nasa.gov/pub/THOR/version_2.2/

    1. Username: anonymous

    2. Password: [your email address]

  2. Download the version that is most appropriate for your system. Once the zip file is downloaded, unzip it, and run one of the following scripts to install:

    1. OSX: setupMAC.command

    2. Linux / Unix: setupUNIX.sh

    3. Windows: setupWIN.bat

  3. Once the setup script has finished, run the new executable file that was created to launch THOR. Full documentation for using THOR is located in the unzipped THOR install directory under “Doc” -> “Tutorial.pdf”.

  4. In order to download data from the TRMM FTP you must first register your email address with the Precipitation Processing System, using this page: http://registration.pps.eosdis.nasa.gov/registration/newContact.html

  5. Download your desired HDF data files from the TRMM FTP: ftp://arthurhou.pps.eosdis.nasa.gov/pub/trmmdata/  

    1. The structure of directories indicates the type of data you will be accessing. For example:

      1. “ByDate” -> “V07” -> “2014” -> “06” -> “03” Will lead you to TRMM data files using organized by date, using the Version 7 algorithm, from June 3rd.

      2. “By Instrument” -> “V06” -> “PRC” -> “2014” -> “05” -> “14” will lead you to TRMM Precipitation Radar data files using the Version 6 algorithm, from May 14th, 2014.

    2. The HDF file name also indicates the data contained. The format is: [algorithm ID].[dataset end date in YYYYMMDD format].[orbit number (for swath products only)].[data processing version].[file format].gz

  6. Unzip the HDF data file, and use the THOR program to open the file using “File” -> “Open File”

  7. You can navigate the structure of the HDF file using the left panel. The right panel displays your selected info, depending on what kind of data you are viewing. Each algorithm may contain different fields and data types, full documentation is available here: http://pps.gsfc.nasa.gov/ppsdocuments.html#version7

  8. Here are a few examples:

    1. To view surface precipitation rate by date, download a file from this location in the FTP: “ByDate” -> “V07” -> [select your desired date and a “Level 3” algorithm file]. Then in THOR load the file and select “Grid” -> “surfacePrecipRate” to view average precipitation on a global map. Screenshot of precip data within THOR

    2. To view TRMM Precipitation Radar data in 3D, download a file from this location in theFTP: “ByInstrument” -> “V07” -> “PCR” -> [select your desired date and a “Level 2” algorithm file]. Then in THOR load the file and select “Swath” -> “Rain” and choose an area on the map that has precipitation and is covered by the swath, then click “View” -> “Swath” -> “3D View”Screenshot of precip data in THOR


 

 

Import Gridded GPM / TRMM Data in NetCDF Format into ArcGIS

Overview: 

Satellite observation and climate model data become more and more widely used in GIS.  ArcGIS is one of the dominant software packages in the GIS community. NetCDF format is not a traditionally used GIS format although it is getting popular in the community.  This recipe shows how to import a grided model or satellite (Level 3 or Level 4) data file in NetCDF format into ArcGIS. 

Best When: 

The data is in CF-complaint NetCDF format 

Task: 
Viewing Data
 
Example: 
Import a TRMM monthly  precipitation data file into ArcGIS.
 
Estimated time to complete the folllowing procedure: 5 min
 

Procedure: 

1) Getting data in NetCDF format

The original data format of TRMM products is HDF.  There are a number ways to obtain the data in CF-compliant NetCDF format from GES DISC.  In this example, the monthly data (Sep 2012) was downloaded from OPeNDAP service at:
 
http://disc2.nascom.nasa.gov/opendap/ncml/TRMM_L3/TRMM_3B43/2012/245/3B43.20120901.7.HDF.ncml.html​
 
Clicking on “Get as NetCDF” button to download the file:
nc_3B43.20120901.7.HDF.ncml.nc

2) Import data into ArcGIS

  • Start an ArcGIS Application, for example, ArcMap

  • Open the ArcToolbox window with the Show/Hide ArcToolbox Window button  found on the standard toolbar or by clicking Geoprocessing -> ArcToolbox (Figure 1)

ARCGis Example screenshot

Figure 1: Sample of ArcMap start page

  • From ArcToolbox --> Multidimensional Toolbox --> Make NetCDF Raster Layer (Figure 2)

ARCGis Example screenshot

Figure 2: Sample of ArcToolbox of ArcMap

  • Select the input file, the variable to be imported, and the X and Y dimensions of the data.  In this example, the variable to be imported is “precipitation” and the X dimension is “nlon” and the Y dimension is “nlat” (Figure 3).  The X and Y dimensions are the coordinate variables included in the NetCDF file and they are usually very easy to be identified from the pull down menu, e.g., often being named something like “longitude” and “latitude”

  • You may modify the Output Raster Layer name, default is "precipitation_Layer 1"

  • Click “Ok”  at the bottom to add the variable and an image will be displayed as in Figure 4

ARCGis Example screenshot

Figure 3: Sample of Make NetCDF Raster Layer in ArcMap

 

ARCGis Example screenshot

Figure 4: Sample TRMM Level 3 monthly precipitation displayed in ArcMap

 

Discussion: 

CF-compliant NetCDF data will be imported with georeferencing information being correctly carried over.  For none CF-compliant NetCDF data, georeferencing information may be not exist or not be imported correctly.  It is important to correctly identify the dimension in the data file.
 
The ArcGIS “Make netCDF Raster Layer” tool requires that the data have equal spacing along each dimension.  Thus, satellite swath data (Level 1 or Level 2) cannot be imported using above method, in which case the “Make netCDF Feature Layer” can be used.  See the recipe "How to Import Swath Data in NetCDF format into ArcGIS" for more information.
 
Some satellite data contains vertical and/or time dimension.  These dimensions can be defined in ArcGIS. See the recipe "How to Define Vertical and Temporal Dimensions in ArcGIS" for more information