Data Resources: FAQ on Reading PSD netCDF files
- I get a compiler error when I try to compile my code that includes the 'udunits.inc' file.
- I look at the data with ncdump and they look like a bunch of large negative integers. The parameter should range from -6 to 40. What's going on here?
- I look at the data file with ncdump and all the time values look like big floating point numbers that don't make any sense.
- Where can I find on-line versions of the NCEP/NCAR 40-years reanalysis project?
- How can I purchase CDROMs from the Annual Reanalysis CDROM series?
- How can I obtain historical weather summaries for many stations in the United States and throughout the world?
- Does Unidata's netCDF package run on PCs? Does the GrADS analysis and plotting package read netCDF and work on PCs?
- What is 'reverse name look-up' and how does it pertain to my IP address?
- How can I find out more information regarding skin temperature?
- What is the procedure to FTP a file subset?
- How is the tropopause in the reanalysis computed? How is the tropopause identified?
- What does the following statement mean: 'The data in the netCDF files are packed' ?
- Why do I get a blank page when I request a subset?
- What goes into the plot (from the subset and plot web page) when I request many levels and many timesteps?
- What are sigma levels/what's the pressure at sigma 0.NNN?
- What are these extreme values over the ocean that I see with Reanalysis skin temperatures?
- In which direction are positive u winds? positive v winds?
- Why doesn't monthly mean wind speed match the square root of the sum of the squares of the monthly U and V winds?
- I get a compiler error when I try to compile my code that includes the 'udunits.inc' file.
-
Try the latest version of the udunits package from Unidata:
http://www.unidata.ucar.edu/packages/udunits
Some earlier versions had problems, but they seem to have been corrected. - I look at the data with ncdump and they look like a bunch of large negative integers. The parameter should range from -6 to 40. What's going on here?
-
The vast majority of the data that we distribute in netCDF files have
been packed into short integers to save space. You must unpack that data
to get the correct floating point representation of the data. Each netCDF
variable that has been packed has an add_offset and scale_factor attribute
associated with it. The scale_factor and add_offset can vary with each
netCDF file even for the same variable from the same data set, though
this doesn't happen often.
The formula to unpack the data is:
unpacked value = add_offset + ( (packed value) * scale_factor )
- I look at the data file with ncdump and all the time values look like big floating point numbers that don't make any sense.
-
First of all, there is a filter for ncdump that will translate the
times in our files to a human readable form. You can find a copy of
read.ncdump
here.
- Where can I find on-line versions of the NCEP/NCAR 40-years reanalysis project?
- "The NCEP/NCAR 40-years Reanalysis Project" (Bulletin of the American Meteorological Society, Vol. 77, No. 3, March 1996). On-line version. It's also available here.
- How can I purchase CDROMs from the Annual Reanalysis CDROM series?
- CDROMs from the Annual Reanalysis CDROM series can be purchased from NCAR. To find out about which years are available, and how to order the CDROMs from the Web page, Click Here.
- How can I obtain historical daily weather summaries for many stations in the United States and throughout the world?
- The data in our archive are 2.5 and 1.0 degree global gridded data. Contact the National Climatic Data Center, they make available historical daily weather summaries for many stations in the United States and throughout the world. Click Here . In addition, some general information about climate and weather is available. This page includes a link to a list of state climatologists. State climatologists often have information about specific stations in their state. You might also try the Regional Climate Center for your region.
- Does Unidata's netCDF package run on PCs? Does the GrADS analysis and plotting package read netCDF and work on PCs?
-
Unidata's netCDF package runs fine on PCs. Unidata's main NetCDF web page is
here.
The FTP version of the same information is
here.
Also, the GrADS analysis and plotting package reads netCDF and works on PCs. However, it requires an X11 server on the PC. For information, see: Getting Started with X/Win32 GrADS and PGetting Started with Win32e GrADS. Further information may be found here. - What is 'reverse name look-up' and how does it pertain to my IP address?
-
When your machine connects to our machine via ftp we choose to initiate a
process called a reverse name look-up. That is to say we ask domain name
service to translate your IP address back into a fully qualified domain
name. This is a process that many sites are begining to use as an added
security feature. In this way we can verify that you are who you say you
are. Now of course, we know that you're not trying to do anything sneaky,
but this is something that we require of everyone that connects to us.
- How can I find out more information regarding skin temperature?
-
Over land and sea ice, the skin temperature is a prognostic variable.
Over open water, the skin temperature is fixed at its initial value (from
the Reynolds SST data). The Reynolds' SST analyses were done weekly and
the reconstructed SST done monthly. The analyses were linearly
interpolated to daily values which were used for all four analyses (i.e.
0, 6, 12 and 18Z have the same SST values).
/Datasets/ncep.reanalysis/surface_gauss/skt.sfc.gauss.*.nc
/Datasets/ncep.reanalysis.dailyavgs/surface_gauss/skt.sfc.gauss.*.nc
/Datasets/ncep.reanalysis.derived/surface/skt.mon.mean.nc.
Gridded Air Temperatures from NCEP/NCAR Reanalysis at PSDpressure/air.YY.nc: Temperature (K) at 17 levels. Class A. (Page 463 Column 1)
surface/air.sig995.YY.nc: Temperature at the lowest sigma level (K). Class B. (Page 463 Column 1)
surface_gauss/air.2m.gauss.YY.nc: Temperature at 2 meters (K). Class B. (Page 464 Column 1)
surface_gauss/skt.sfc.gauss.YY.nc: Temperature at surface (skin temperature) K. Class B. (Page 464 Column 1)
tropopause/air.tropp.YY.nc: Temperature at the tropopause (K). Class A. (Page 463 Column 1)
Class A indicates that the analysis variable is strongly influenced by observed data and, hence, it is the most reliable class.
Class B indicates that, although there are observational data that directly affect the value of the variable, the model also has a very strong influence on the analysis value.
- What is the procedure to FTP a file subset?
-
- First, go to the general search page to find the data you want to retrieve.
- Highlight (click on) the dataset(s) and/or variable(s) that are of interest.
- Push the Submit button. This will return to you a list of SLP files from the NCEP data set.
- Choose the map icon to create a plot or a subset for the data of interest.
- Enter the geographic region you are interested in.
- Select the start and end time. If you want the whole time span you should select the first time on the left menu and the last time on the right menu.
- If you want to create the subset without making a plot, first select the "Create a subset without a making a plot" radio button and push the "Create Plot or Subset of Data" button.
- At this point you should have a web page containing a link to the file subset. If you follow this link, most browsers are set up to prompt you to save the file on your local disk. Alternatively, position your cursor over the link and hold down your right mouse button. This action should result in a browser menu with a "Save Link As..." option. Select this option to save the file on your local disk.
For example, choose:
Data Set: NCEP Variable: Sea Level Pressure
NOTE: If you make a plot it will be the AVERAGE of the time steps (and other ranges like the levels in the z direction), but the file contains the individual time steps.
- How is the tropopause in the reanalysis computed? How is the tropopause identified?
-
The tropopause in the Reanalysis is computed from temperature analysis on
model sigma level fields. This is to avoid unnecessary interpolation from
sigma to pressure. The definition of tropopause in NCEP's post-processor is
as follows:
THE TROPOPAUSE IS IDENTIFIED BY THE LOWEST LEVEL ABOVE 450 MB WHERE THE TEMPERATURE LAPSE RATE -DT/DZ BECOMES LESS THAN 2 K/KM. THE TROPOPAUSE IS NOT ALLOWED HIGHER THAN 85 MB. INTERPOLATIONS OF VARIABLES TO TROPOPAUSE ARE DONE LINEARLY IN LOG OF PRESSURE.
No one has carefully looked at the product so that you may have to do some verification against Radiosonde observation before using it. It may be a little noisy, therefore, some spatial filtering may be necessary.
- What does the following statement mean: 'The data in the netCDF files are packed'?
-
Most of the data in our netCDF files are packed. That is to say they have been
transformed by a scale factor and an add offset to reduce the storage needed to
two bytes per value. When you extract the short integers, you must unpack the
data to recover the correct floating point data values. Data files that
contain packed data will have a non-zero add offset and/or a scale factor not
equal to 1.
float_value = (short_value * scale_factor) + add_offset
- Why do I get a blank page when I request a subset?
-
The subsetting process can take a fair amount of time, and
thus cause an HTTP timeout, leaving you with a blank page in your browser.
We have not been successful with determining an exact threshold for this,
as it would seem to be load dependent. However, a year's worth of data
is a "rule of thumb" maximum. The NCO utilities:
http://nco.sourceforge.net/
can be used to "stitch back together" thus-fragmented subsets at your end, if this is felt to be necessary, up to the 2 Gbyte netCDF limit. - What goes into the plot (from the subset and plot web page) when I request many levels and many timesteps?
- If time or level vary in your subset selection, the subsequent plot (if you choose to make one) will be averaged along those axes, but the subset file will have the individual timesteps and levels.
- What are sigma levels/what's the pressure at sigma 0.NNN?
- In some of the data sets archived at PSD, notably the NCEP-NCAR Reanalysis 1 data set, a Z axis called "sigma levels" is used. This is used in some models as an alternative to pressure levels. For example, on sigma level 0.995, the pressure in a given grid cell at a specific time step is 0.995 times the surface pressure in that grid cell at that time. Thus, one needs the surface pressure in order to obtain the pressure at the sigma level, and it varies as a function of space and time.
- What are these extreme values over the ocean that I see with Reanalysis skin temperatures?
- The NCEP-NCAR Reanalysis 1 skin temperature values include values over sea-ice, as the skin temperature is a composite variable, using SST values over open ocean, and a model prognostic otherwise. Thus, even with a land-sea mask, you will see non-SST values in the polar regions. The trick is to plot with smaller range, such as -6 to 6 degrees Centigrade. If one can find a mask for the sea-ice, applying it would be ideal.
- In which direction are positive u winds? positive v winds?
- We use the meteorologists' convention that positive u winds (zonal) are FROM the west and called westerlies while positive v winds (meridional) are from the south and call southerlies.
- Why doesn't monthly mean wind speed match the square root of the sum of the squares of the monthly U and V winds?
- This applies to data sets with winds of higher temporal resolution, such as daily. Wind speed over a month should be calculated ideally by averaging the wind speed at a higher temporal frequency (e.g., daily). By deriving wind speed from the monthly average U and V winds, a significant part of the wind speed is missed. You might notice this when making wind vector plots from our various web pages. For some data sets, a wind speed variable is made available that is separately calculated from higher temporal frequency U and V winds. You should use that if available. Note that the vector wind direction will be correct when using the monthly mean U & V winds, although the speed will not.