Data Resources: FAQ on Reading PSD netCDF files

I get a compiler error when I try to compile my code that includes the 'udunits.inc' file.
Try the latest version of the udunits package from Unidata:
http://www.unidata.ucar.edu/packages/udunits
Some earlier versions had problems, but they seem to have been corrected.
I look at the data with ncdump and they look like a bunch of large negative integers. The parameter should range from -6 to 40. What's going on here?
The vast majority of the data that we distribute in netCDF files have been packed into short integers to save space. You must unpack that data to get the correct floating point representation of the data. Each netCDF variable that has been packed has an add_offset and scale_factor attribute associated with it. The scale_factor and add_offset can vary with each netCDF file even for the same variable from the same data set, though this doesn't happen often.

The formula to unpack the data is:

unpacked value = add_offset + ( (packed value) * scale_factor )
I look at the data file with ncdump and all the time values look like big floating point numbers that don't make any sense.
First of all, there is a filter for ncdump that will translate the times in our files to a human readable form. You can find a copy of read.ncdump here.

 

All of the times are double values which represent the number of days or hours from an origin date. The units attribute for the time variable is the key to decoding the time values. For example, a file could have time units of "days since 1-1-1 00:00:0.0". With those units, an encoded value of 725738. would be the value for the date 1988-01-01. Now, if you just did some quick arithmetic in your head and you came up with a different value, it's not too surprising. There are many calendar systems that have been existence at various times in history and there are various shifts that won't be accounted for by all algorithms. The translation between the times in the file and human readable times and vice versa can be greatly simplified by using the UDUNITS library from UniData. Additionally, Numerical Recipes implements the correct algorithm as well.
Where can I find on-line versions of the NCEP/NCAR 40-years reanalysis project?
"The NCEP/NCAR 40-years Reanalysis Project" (Bulletin of the American Meteorological Society, Vol. 77, No. 3, March 1996). On-line version. It's also available here.
How can I purchase CDROMs from the Annual Reanalysis CDROM series?
CDROMs from the Annual Reanalysis CDROM series can be purchased from NCAR. To find out about which years are available, and how to order the CDROMs from the Web page, Click Here.
How can I obtain historical daily weather summaries for many stations in the United States and throughout the world?
The data in our archive are 2.5 and 1.0 degree global gridded data. Contact the National Climatic Data Center, they make available historical daily weather summaries for many stations in the United States and throughout the world. Click Here . In addition, some general information about climate and weather is available. This page includes a link to a list of state climatologists. State climatologists often have information about specific stations in their state. You might also try the Regional Climate Center for your region.
Does Unidata's netCDF package run on PCs? Does the GrADS analysis and plotting package read netCDF and work on PCs?
Unidata's netCDF package runs fine on PCs. Unidata's main NetCDF web page is here. The FTP version of the same information is here.
Also, the GrADS analysis and plotting package reads netCDF and works on PCs. However, it requires an X11 server on the PC. For information, see: Getting Started with X/Win32 GrADS and PGetting Started with Win32e GrADS. Further information may be found here.
What is 'reverse name look-up' and how does it pertain to my IP address?
When your machine connects to our machine via ftp we choose to initiate a process called a reverse name look-up. That is to say we ask domain name service to translate your IP address back into a fully qualified domain name. This is a process that many sites are begining to use as an added security feature. In this way we can verify that you are who you say you are. Now of course, we know that you're not trying to do anything sneaky, but this is something that we require of everyone that connects to us.

 

At your site, your computer managers maintain two lists of domain names. One is the forward name and the other is the reverse. Often, only the forward name is kept up-to-date. That's the one that is used when you say you want to connect to your machine and give its name. The forward name translates back to the IP address. But, there is also a table that keeps track of the reverse name, the one that answers the question, what name belongs to this IP address.

 

Hopefully your system manager will understand from this message what's being asked and can correct the problem with a couple minutes of work. Another solution might be to try your ftp from a different machine.
How can I find out more information regarding skin temperature?
Over land and sea ice, the skin temperature is a prognostic variable. Over open water, the skin temperature is fixed at its initial value (from the Reynolds SST data). The Reynolds' SST analyses were done weekly and the reconstructed SST done monthly. The analyses were linearly interpolated to daily values which were used for all four analyses (i.e. 0, 6, 12 and 18Z have the same SST values).

 

The files that contain skin temperature 4 times daily are of the form:

/Datasets/ncep.reanalysis/surface_gauss/skt.sfc.gauss.*.nc

 

The files that contain skin temperature daily averages are of the form:

/Datasets/ncep.reanalysis.dailyavgs/surface_gauss/skt.sfc.gauss.*.nc

 

The file that contains the monthly skin temperature values is:

/Datasets/ncep.reanalysis.derived/surface/skt.mon.mean.nc.

Gridded Air Temperatures from NCEP/NCAR Reanalysis at PSD

pressure/air.YY.nc: Temperature (K) at 17 levels. Class A. (Page 463 Column 1)

surface/air.sig995.YY.nc: Temperature at the lowest sigma level (K). Class B. (Page 463 Column 1)

surface_gauss/air.2m.gauss.YY.nc: Temperature at 2 meters (K). Class B. (Page 464 Column 1)

surface_gauss/skt.sfc.gauss.YY.nc: Temperature at surface (skin temperature) K. Class B. (Page 464 Column 1)

tropopause/air.tropp.YY.nc: Temperature at the tropopause (K). Class A. (Page 463 Column 1)

Class A indicates that the analysis variable is strongly influenced by observed data and, hence, it is the most reliable class.

Class B indicates that, although there are observational data that directly affect the value of the variable, the model also has a very strong influence on the analysis value.

What is the procedure to FTP a file subset?
  1. First, go to the general search page to find the data you want to retrieve.
  2. Highlight (click on) the dataset(s) and/or variable(s) that are of interest.

    3.   For example, choose:

      Data Set: NCEP Variable: Sea Level Pressure

     

  3. Push the Submit button. This will return to you a list of SLP files from the NCEP data set.
  4. Choose the map icon to create a plot or a subset for the data of interest.
  5. Enter the geographic region you are interested in.
  6. Select the start and end time. If you want the whole time span you should select the first time on the left menu and the last time on the right menu.
  7. If you want to create the subset without making a plot, first select the "Create a subset without a making a plot" radio button and push the "Create Plot or Subset of Data" button.
  8. At this point you should have a web page containing a link to the file subset. If you follow this link, most browsers are set up to prompt you to save the file on your local disk. Alternatively, position your cursor over the link and hold down your right mouse button. This action should result in a browser menu with a "Save Link As..." option. Select this option to save the file on your local disk.

    9. NOTE: If you make a plot it will be the AVERAGE of the time steps (and other ranges like the levels in the z direction), but the file contains the individual time steps.

How is the tropopause in the reanalysis computed? How is the tropopause identified?
The tropopause in the Reanalysis is computed from temperature analysis on model sigma level fields. This is to avoid unnecessary interpolation from sigma to pressure. The definition of tropopause in NCEP's post-processor is as follows:

THE TROPOPAUSE IS IDENTIFIED BY THE LOWEST LEVEL ABOVE 450 MB WHERE THE TEMPERATURE LAPSE RATE -DT/DZ BECOMES LESS THAN 2 K/KM. THE TROPOPAUSE IS NOT ALLOWED HIGHER THAN 85 MB. INTERPOLATIONS OF VARIABLES TO TROPOPAUSE ARE DONE LINEARLY IN LOG OF PRESSURE.

No one has carefully looked at the product so that you may have to do some verification against Radiosonde observation before using it. It may be a little noisy, therefore, some spatial filtering may be necessary.

What does the following statement mean: 'The data in the netCDF files are packed'?
Most of the data in our netCDF files are packed. That is to say they have been transformed by a scale factor and an add offset to reduce the storage needed to two bytes per value. When you extract the short integers, you must unpack the data to recover the correct floating point data values. Data files that contain packed data will have a non-zero add offset and/or a scale factor not equal to 1.

 

The transformation is:

float_value = (short_value * scale_factor) + add_offset

 

Why do I get a blank page when I request a subset?
The subsetting process can take a fair amount of time, and thus cause an HTTP timeout, leaving you with a blank page in your browser. We have not been successful with determining an exact threshold for this, as it would seem to be load dependent. However, a year's worth of data is a "rule of thumb" maximum. The NCO utilities:
http://nco.sourceforge.net/
can be used to "stitch back together" thus-fragmented subsets at your end, if this is felt to be necessary, up to the 2 Gbyte netCDF limit.
What goes into the plot (from the subset and plot web page) when I request many levels and many timesteps?
If time or level vary in your subset selection, the subsequent plot (if you choose to make one) will be averaged along those axes, but the subset file will have the individual timesteps and levels.
What are sigma levels/what's the pressure at sigma 0.NNN?
In some of the data sets archived at PSD, notably the NCEP-NCAR Reanalysis 1 data set, a Z axis called "sigma levels" is used. This is used in some models as an alternative to pressure levels. For example, on sigma level 0.995, the pressure in a given grid cell at a specific time step is 0.995 times the surface pressure in that grid cell at that time. Thus, one needs the surface pressure in order to obtain the pressure at the sigma level, and it varies as a function of space and time.
What are these extreme values over the ocean that I see with Reanalysis skin temperatures?
The NCEP-NCAR Reanalysis 1 skin temperature values include values over sea-ice, as the skin temperature is a composite variable, using SST values over open ocean, and a model prognostic otherwise. Thus, even with a land-sea mask, you will see non-SST values in the polar regions. The trick is to plot with smaller range, such as -6 to 6 degrees Centigrade. If one can find a mask for the sea-ice, applying it would be ideal.
In which direction are positive u winds? positive v winds?
We use the meteorologists' convention that positive u winds (zonal) are FROM the west and called westerlies while positive v winds (meridional) are from the south and call southerlies.
Why doesn't monthly mean wind speed match the square root of the sum of the squares of the monthly U and V winds?
This applies to data sets with winds of higher temporal resolution, such as daily. Wind speed over a month should be calculated ideally by averaging the wind speed at a higher temporal frequency (e.g., daily). By deriving wind speed from the monthly average U and V winds, a significant part of the wind speed is missed. You might notice this when making wind vector plots from our various web pages. For some data sets, a wind speed variable is made available that is separately calculated from higher temporal frequency U and V winds. You should use that if available. Note that the vector wind direction will be correct when using the monthly mean U & V winds, although the speed will not.