Satellite Data Ingest
at NCEP
Dennis Keyser -
NOAA/NWS/NCEP/EMC
(Last Revised 2/5/2009)
Please take a moment to read the Disclaimer
for this non-operational web page.
The ingest of most satellite data into the NCEP /dcom database on
the
IBM
Central Computer System (IBM-CCS) machines is performed separately
from
the NCEP/NCO
decoding
system which handles data primarily on GTS.
Most of the satellite data described here are provided to NCEP from
NESDIS (either directly or as a conduit from other centers). Files are transferred, via ftp, from various servers directly
to
the NCEP IBM-CCS machine as they become available. A schedule has been
established for the initiation of jobs on the IBM-CCS which perform
this
file transfer (see Table
1). Each time a particular ingest job runs, a query is first
performed which returns a listing of files for the particular data type
currently residing on the server . This list is compared against a
history
of files that have previously been transferred and processed. If any
new
files are present, then those files are transferred. Otherwise, no
action
is taken.
The following types are currently part of the satellite ingest:
- NESDIS/GOES sounding and brightness temperature data (for ingest schedule see type
"goes_radsnd"
in Table
1)
- 1x1 field-of-view satellite soundings, retrievals, level 1c sounder
brightness
temperatures pulled
from NESDIS satepsdist1e.nesdis.noaa.gov server via anonymous ftp
- files /retcld/ret1/satsnd.goesW.D??????????.T????Z.1B1 [GOES-11 (W)]
- files /retcld/ret1/satsnd.goesE.D??????????.T????Z.1B1 [GOES-12 (E)]
- 1x1 field-of-view cloud data (cloud top pressure, temperature)
pulled
from NESDIS satepsdist1e.nesdis.noaa.gov server via anonymous ftp
- files /retcld/cld/goesW.RMD.D?????.T????.Z [GOES-11 (W)]
- files /retcld/cld/goesE.RMD.D?????.T????.Z [GOES-12 (E)]
- 11x17 field-of-view level 1c imager (clear-sky) brightness temperatures
pulled
from
NESDIS satepsdist1e.nesdis.noaa.gov server via anonymous ftp
- files /csbt/bufr/gicsbt.???????.????.goes11NB [GOES-11 (W)]
- files /csbt/bufr/gicsbt.???????.????.goes12NB [GOES-12 (E)]
- FNMOC/DMSP SSM/I data
(for ingest schedule see type "ssmi" in Table
1)
- EDRs (derived products) pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.EDRR.S7.D?????.S????.E????.????????.NS (DMSP F-13)
- files pull/NPR.EDRR.S9.D?????.S????.E????.????????.NS (SMPS F-15)
- SDRs (1c brightness temperatures) pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.SDRR.S7.D?????.S????.E????.????????.NS (DMSP F-13)
- files pull/NPR.SDRR.S9.D?????.S????.E????.????????.NS (SMPS F-15)
- NESDIS/POES ATOVS sounding and radiance data (for ingest
schedule
see “tovs” in Table
1)
- ATOVS soundings and level 1c sounder brightness
temperatures pulled from NESDIS dds.nesdis.noaa.gov
machine via ftp
account
- files pull/NPR.ATNC.NN.D?????.S????.E????.???????? (NOAA-18)
- files pull/NPR.ATNC.M2.D?????.S????.E????.???????? (METOP-2)
- AMSU-A 1b sounder brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.AMAX.NK.D?????.S????.E????.????????.?? (NOAA-15)
- files pull/NSS.AMAX.NL.D?????.S????.E????.????????.?? (NOAA-16)
- files pull/NSS.AMAX.NN.D?????.S????.E????.????????.?? (NOAA-18)
- files pull/NSS.AMAX.M2.D?????.S????.E????.????????.?? (METOP-2)
- AMSU-B 1b sounder brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.AMBX.NK.D?????.S????.E????.????????.?? (NOAA-15)
- files pull/NSS.AMBX.NL.D?????.S????.E????.????????.?? (NOAA-16)
- files pull/NSS.AMBX.NM.D?????.S????.E????.????????.?? (NOAA-17)
- MHS 1b sounder brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.MHSX.NN.D?????.S????.E????.????????.?? (NOAA-18)
- files pull/NSS.MHSX.M2.D?????.S????.E????.????????.?? (METOP-2)
- HIRS-3 1b sounder brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.HIRX.NK.D?????.S????.E????.????????.?? (NOAA-15)
- files pull/NSS.HIRX.NL.D?????.S????.E????.????????.?? (NOAA-16)
- files pull/NSS.HIRX.NM.D?????.S????.E????.????????.?? (NOAA-17)
- HIRS-4 1b sounder brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.HIRX.NN.D?????.S????.E????.????????.?? (NOAA-18)
- files pull/NSS.HIRX.M2.D?????.S????.E????.????????.?? (METOP-2)
- NESDIS/GOES and NASA/POES-MODIS satellite derived winds
(for ingest schedule see "satwnd" in Table
1)
- GOES IR imager winds pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /bufrwinds/satwnd.bufrCD.goesW.D?????.T??:??:??Z [GOES-11 (W)]
- files /bufrwinds/satwnd.bufrCD.goesE.D?????.T??:??:??Z [GOES-12 (E)]
- GOES water vapor imager winds pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /bufrwinds/satwnd.bufrWV.goesW.D?????.T??:??:??Z [GOES-11 (W)]
- files /bufrwinds/satwnd.bufrWV.goesE.D?????.T??:??:??Z [GOES-12 (E)]
- GOES visible imager winds pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /bufrwinds/satwnd.bufrVZ02.goesW.D?????.T??:??:??Z [GOES-11 (W)]
- files /bufrwinds/satwnd.bufrVZ02.goesE.D?????.T??:??:??Z [GOES-12 (E)]
- GOES water vapor sounder winds pulled from NESDIS
satepsdist1e.nesdis.noaa.gov server
via anonymous ftp
- files /bufrwinds/satwnd.bufrWV11.goesW.D?????.T??:??:??Z [GOES-11 (W), channel 11]
- files /bufrwinds/satwnd.bufrWV10.goesW.D?????.T??:??:??Z [GOES-11 (W), channel 10]
- files /bufrwinds/satwnd.bufrWV11.goesE.D?????.T??:??:??Z [GOES-12 (E), channel 11]
- files /bufrwinds/satwnd.bufrWV10.goesE.D?????.T??:??:??Z [GOES-12 (E), channel 10]
- MODIS IR imager winds pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /bufr/modis_winds/satwnd.bufrcdft.aqua.D?????.T??:??:??Z (AQUA)
- files /bufr/modis_winds/satwnd.bufrcdft.terra.D?????.T??:??:??Z (TERRA)
- MODIS water vapor imager winds pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /bufr/modis_winds/satwnd.bufrwvap.aqua.D?????.T??:??:??Z (AQUA)
- files /bufr/modis_winds/satwnd.bufrwvap.terra.D?????.T??:??:??Z (TERRA)
- ESA/ERS data (for ingest schedule see "euscatter" in Table
1)
- RA (radar altimeter) data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.URAX.E2.D?????.S????.E???? (ERS-2)
- SAR (Synthetic Aperture Radar) data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.UWAX.E2.D?????.S????.E???? (ERS-2)
- WS (Wind Scatterometer) data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.UWIX.E2.D?????.S????.E???? (ERS-2)
- NASA/QuikSCAT and EUMETSAT/ASCAT data (for ingest schedule see "qscatter"
in Table
1)
- QuikSCAT wind scatterometer data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.QBUF.QS.D?????.S????.E????.???????? (QUIKSCAT)
- ASCAT wind scatterometer data at 50 km resolution pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/ascat_????????_??????_metopa_?????_srv_o_250_ovw.l2_bufr (METOP-2)
- FNMOC/WindSat data (for ingest schedule see "windsat"
in Table
1)
- wind scatterometer data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.E068.WS.D?????.S????.E???? (CORIOLIS)
- NASA/AQUA and EUMETSAT/METOP/IASI brightness temperature data (for
ingest
schedule
see "airs" in Table
1)
- AIRS and AMSU-A every field-of-view 1b
sounder brightness temperatures pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.AIBT.AQ.D?????.S????.E????.????.C0300.U8 (AQUA)
- AMSR-E 1b brightness temperatures pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR.ARBT.AQ.D?????.S????.E????.????????.U? (AQUA)
- METOP/IASI 616 channel every field-of-view 1c brightness temperatures pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/NPR_IARD.M02_D?????_S??????_E??????_G???_C0616_U6 (METOP-2)
- NESDIS/POES AVHRR/GAC brightness temperature data (for
ingest
schedule
see "avhrr" in Table
1)
- NOAA-17 every field-of-view GAC brightness temperatures pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.GHRR.NM.D?????.S????.E????.????????.??
- NOAA-18 every field-of-view GAC brightness temperatures pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/NSS.GHRR.NN.D?????.S????.E????.????????.??
- NASA/TRMM (Tropical Rainfall Measuring Mission) data (for
ingest schedule see “tmi” in Table
1)
- TMI (TRMM Microwave Imager) rainfall data pulled from NASA trmmrt.gsfc.nasa.gov server via ftp account
- files /develop/people/trmmreal/TMI/2A12.????-??-??T??-??-??Z.rt.gz (TRMM)
- POES SST data
(for ingest schedule see "poes_sst" in Table
1)
- Navy-generated sea-surface temperature retrievals,
AVHRR brightness temperature and albedo pulled from
NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/navysst.tmpobs.*.????.???.???? (NOAA-17, -18, METOP-2)
- NESDIS-generated sea-surface temperature retrievals, AVHRR brightness temperature
and albedo pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/sst.tmpobs.*.????.???.???? (NOAA-16, -17, -18, METOP-2)
- GOES-sea-surface temperature data (for ingest schedule see "goes_sst" in Table
1)
- Navy-generated 24-hour average sea-surface temperature retrievals pulled from NESDIS satepsdist1e.nesdis.noaa.gov
server
via anonymous ftp
- files /gsst_bayes/daily/sst24o_????_??? [GOES-11 (W) and -12 (E)]
- NESDIS/POES and METOP-2 orbital ozone data (for ingest
schedule
see “ozone_orbit” in Table
1)
- Solar Backscatter UltraViolet (SBUV-2) ozone data in Version 8 BUFR pulled from NESDIS dds.nesdis.noaa.gov
machine via ftp
account
- files pull/PRD.OZONE.PMFV8.N17.D?????.S????.E????.?????? (NOAA-17)
- files pull/PRD.OZONE.PMFV8.N16.D?????.S????.E????.?????? (NOAA-16)
- files pull/PRD.OZONE.PMFV8.N18.D?????.S????.E????.?????? (NOAA-18)
- Global Ozone Monitoring Experiment (GOME-2) ozone data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- files pull/GOME_M02_8TO3_??????????????Z_??????????????Z.BUFR (METOP-2)
- NESDIS/POES daily ozone data (for ingest schedule see
"ozone14" in Table
1)
- NOAA-18 Solar
Backscatter UltraViolet (SBUV-2) ozone data in Version 8
BUFR pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- file pull/PRD.OZONE.PMF.N18.BUFR
- NOAA-16 Solar
Backscatter UltraViolet (SBUV-2) ozone data in Version 8
BUFR pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- file pull/PRD.OZONE.PMF.N16.BUFR
- NOAA-17 Solar
Backscatter UltraViolet (SBUV-2) ozone data in Version 8
BUFR pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- file pull/PRD.OZONE.PMF.N17.BUFR
- Aerosol and Green Vegetation Fraction data (for ingest schedule see
"aerosol"
in Table
1)
- 100 km field of weekly-averaged global
NESDIS aerosol analyses pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/aer.field.100km_global.n18.weekly (NOAA-18)
- 100 km field of monthly-averaged
global NESDIS aerosol analyses pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- files pull/aer.gridmean.100km_global.n18.monthly (NOAA-18) pulled from NESDIS
dds.nesdis.noaa.gov machine via ftp
account
- NESDIS weekly real-time 0.144-degree global Green Vegetation Fraction (GVF) field
- file /pull/NPR.VGWG.NN.D?????.GRIBF (NOAA-18)
- MODIS Aerosol Optical Depth (MODIS_AOD) data pulled from NASA nanuk.eosdis.nasa.gov machine via ftp account
- Daily snow and ice analyses
(for
ingest schedule see "snow" in Table
1)
- NESDIS IMS 16'th
mesh global snow-cover/sea-ice field (ASCII) pulled from
NESDIS satepsdist1e.nesdis.noaa.gov
server via anonymous ftp (later converted to GRIB1, GRIB1 file copied
forward to the new day until a new file is present for the new day)
- files /snow/asc/ims???????.asc
- NESDIS IMS
96'th mesh global snow-cover/sea-ice field (ASCII) pulled from
NESDIS satepsdist1e.nesdis.noaa.gov
server via anonymous ftp (later converted to GRIB1, GRIB1 file
copied forward to the new day until a new file is present for the new
day)
- files /snow/6144asc/ims???????.asc
- USAF 8'th mesh
Northern Hemisphere snow-depth/sea-ice field (binary) pulled from
NESDIS dds.nesdis.noaa.gov machine via ftp
account (later converted to GRIB1, binary and GRIB1
files copied forward to the new day until a new file is present
for the new day)
- files pull/PRD.SPPROD.SNODEPH.NHMAMAP.D?????
- USAF 8'th mesh Southern Hemisphere snow-depth/sea-ice field (binary) pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account (binary file copied forward to the new day until a new file is present for the new day)
- files pull/PRD.SPPROD.SNODEPH.SHMAMAP.D?????
- NESDIS 4 km Southern Hemisphere "autosnow" (yes/no) snow
cover product field (GRIB2) pulled from NESDIS
satepsdist1e.nesdis.noaa.gov
server via anonymous ftp (GRIB2 file copied forward to the new day, normally always 1-day behind)
- files /autosnow/4kmSH/SH4km_autosnow_????????.grb
- USAF 16'th mesh Northern Hemisphere snow-depth/sea-ice field (GRIB1) pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account (GRIB1 file copied forward to the new day until a new file is present for the new day)
- files /pull/NPR.SNWN.SP.D?????.S1200.MESH16
- USAF 16'th mesh Southern Hemisphere snow-depth/sea-ice field (GRIB1) pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account (GRIB1 file copied forward to the new day until a new file is present for the new day)
- files /pull/NPR.SNWS.SP.D?????.S1200.MESH16
- NESDIS daily sea-surface temperature analyses (for ingest schedule see
"sstfld"
in Table
1)
- 50 km field of daily-averaged global data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- file pull/sst.field.50km_global.n18 (NOAA-18)
- 14 km field of daily-averaged North American data pulled from NESDIS dds.nesdis.noaa.gov machine via ftp
account
- file pull/sst.field.14km_namerica.n18 (NOAA-18)
- Daily global 0.5 degree and T382 and snow depth analysis (GRIB)
(for ingest schedule see
"snowgrib" in Table
1)
- Cloud analyses (for ingest schedule see "cloud" in Table
1)
- AFWA (Air Force Weather Agency) Northern Hemisphere cloud
cover analyses (GRIB) pulled from NESDIS dds.nesdis.noaa.gov
machine via ftp
account
- files /pull/NPR.CLON.SP.D?????.S????
- AFWA (Air Force Weather Agency) Southern Hemisphere cloud
cover analyses (GRIB) pulled from NESDIS dds.nesdis.noaa.gov
machine via ftp
account
- files /pull/NPR.CLOS.SP.D?????.S????
- CLAVR (Clouds from AVHRR) analyses (GRIB) for
00, 06, 12 and 18Z pulled from NESDIS dds.nesdis.noaa.gov machine
via ftp
account
- files /pull/clavrx_??_??_?_????_???.cell.hdf
- GOES IR imager data (for ingest schedule see "goes_ir" in Table
1)
- GOES-11 11 micron channel (chn. 4) IR from the 3-axis
stabilized GOES imager pulled from NESDIS dds.nesdis.noaa.gov machine
via ftp
account
- files pull/GIM.G11.D?????.S????.T????.????????.????????
- GOES-12 11 micron channel (chn. 4) IR from the 3-axis
stabilized GOES imager pulled from NESDIS dds.nesdis.noaa.gov machine
via ftp
account
- files pull/GIM.G12.D?????.S????.T????.????????.????????
The following non-satellite types are processed in a manner similar to the satellite ingest:
- Hourly precipitation data from Office of Hydrology's HADS system
(for ingest schedule see "shf" in Table
1)
- 24-hour precipitation data from Office of Hydrology's HADS system
(not
satellite data, but runs as part of satellite ingest; for ingest schedule see
"shef" in Table
1)
- NEXRAD WSR-88D radial wind data (for ingest schedule see "radwnd" in Table
1)
- Level 3 superobs from NWS Radar Multicast ("NIDS" format)
- Level 2 on-site superobs from Open RPG (referred to as Level 2.5)
NCO also maintains a NESDIS satellite ingest web page - click here.