The Next Generation Weather Radar (NEXRAD) is a network of 160 high-resolution Doppler radar sites that detects precipitation and atmospheric movement and disseminates data in approximately 5 minute intervals from each site. NEXRAD enables severe storm prediction and is used by researchers and commercial enterprises to study and address the impact of weather across multiple sectors.
The real-time feed and full historical archive of original resolution (Level II) NEXRAD data, from June 1991 to present, is now freely available on Amazon S3 for anyone to use. This is the first time the full NEXRAD Level II archive has been accessible to the public on demand. Now anyone can use the data on-demand in the cloud without worrying about storage costs and download time.
We are making NEXRAD data available as part of our research agreement with the US National Oceanic and Atmospheric Administration (NOAA) to enable new product development and analysis.
This page includes information on data structure and sample use cases to help you get started. You can find much more detailed information about NEXRAD Level II data from NOAA and other online sources.
You can browse the contents of the NEXRAD Level II archive via the AWS JavaScript S3 Explorer.
The NEXRAD Level II archive data is hosted in the “noaa-nexrad-level2” Amazon S3 bucket in S3’s US East region. The address for the public bucket is:
http://noaa-nexrad-level2.s3.amazonaws.com
https://noaa-nexrad-level2.s3.amazonaws.com
Each volume scan file is its own object in Amazon S3. The basic data format is the following:
/<Year>/<Month>/<Day>/<NEXRAD Station>/<filename>
Where:
<Year> is the year the data was collected
<Month> is the month of the year the data was collected
<Day> is the day of the month the data was collected
<NEXRAD Station> is the NEXRAD ground station (map of ground stations)
<filename> is the name of the file containing the data. These are compressed files (compressed with gzip). The file name has more precise timestamp information.
All files in the archive use the same compressed format (.gz). The data file names are, for example, KAKQ20010101_080138.gz. The file naming convention is:
GGGGYYYYMMDD_TTTTTT
Where:
GGGG = Ground station ID (map of ground stations)
YYYY = year
MM = month
DD = day
TTTTTT = time when data started to be collected (GMT)
Note that the 2015 files have an additional field on the file name. It adds “_V06” to the end of the file name. An example is KABX20150303_001050_V06.gz.
We’ve uploaded the full historical archive from NOAA from June 1991 to present and are in the process of untarring the archival data so you can access each volume scan as its own object in Amazon S3 in the aforementioned “noaa-nexrad-level2” Amazon S3 bucket. If you don’t see the data you need in that bucket, you can find it in the “noaa-nexrad-l2” Amazon S3 bucket in S3’s US East region. If you don’t see the data you need in either bucket, let us know and we’ll work with NOAA to fill in the gap if possible.
The “noaa-nexrad-l2” bucket has the full archive stored in .tar files, which is how NOAA archives the NEXRAD Level II data. The basic data format is the same:
/<Year>/<Month>/<Day>/<NEXRAD Station>/ <filename>
The “noaa-nexrad-l2” bucket is temporary. We are not adding near-real-time data to it and will delete it once all the volume scan files have been added to the “noaa-nexrad-level2” bucket.
If you would like to show us what you can do with NEXRAD on AWS or would like to receive updates on the project, please fill out the form below.
Educators, researchers and students can also apply for free credits to take advantage of the utility computing platform offered by AWS, along with Public Datasets such as NEXRAD on AWS. If you have a research project that could take advantage of NEXRAD on AWS, you can apply for an AWS Grant.
The NEXRAD Level II real-time data is hosted in the “unidata-nexrad-level2-chunks” Amazon S3 bucket in S3’s US East region. The address for the public bucket is:
http://unidata-nexrad-level2-chunks.s3.amazonaws.com/
https://unidata-nexrad-level2-chunks.s3.amazonaws.com/
Each chunk of each volume scan file is its own object in Amazon S3. The basic data format is the following:
/<Site>/<Volume_number>/<YYYYMMDD-HHMMSS-CHUNKNUM-CHUNKTYPE>
Where:
<Site> is the NEXRAD ground station (map of ground stations)
<Volume_number> is the volume id number (cycles from 0 to 999)
<YYYYMMDD> is the date of the volume scan
<HHMMSS> is the time of the volume scan
<CHUNKNUM> is the chunk number
<CHUNKTYPE> is the chunk type
All files in the real-time feed use bzip2 compression. Additional documentation on the chunks is available on the last page, B-1, of this Interface Control Document.
We have set up public Amazon SNS topics that create a notification for every new object added to the Amazon S3 chunks and archive buckets for NEXRAD on AWS. To start, you can subscribe to these notifications using Amazon SQS and AWS Lambda. This means you can automatically add new real-time and near-real-time NEXRAD data into a queue or trigger event-based processing if the data meets certain criteria such as geographic location.
The Amazon Resource Name (ARN) for the real-time data is arn:aws:sns:us-east-1:684042711724:NewNEXRADLevel2Object
The ARN for the archive data is arn:aws:sns:us-east-1:811054952067:NewNEXRADLevel2Archive
For more information on subscribing to SNS topics, visit http://docs.aws.amazon.com/sns/latest/dg/SubscribeTopic.html.
For more information on Amazon S3 event message structure, visit http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html.
This tutorial from the Climate Corporation shows you how to read and display the NEXRAD Level II archive data from your Python programs.
This tutorial from CartoDB shows you how to create static and animated maps in CartoDB. You can see an example of NEXRAD on AWS in action in their visualization of Hurricane Arthur. They’ve also mapped the NEXRAD station locations.
This open source java tool from students at Purdue University streamlines the process of running a MapReduce job with NEXRAD data on AWS.
Unidata has made the NEXRAD Level II archive data available via their THREDDS Data Server (TDS) to users with .edu domains. Visit the TDS page for more information on using this software. This Jupyter notebook shows you how to use Python to access the NCEI radar archive on Amazon S3 through Unidata's THREDDS server.
You can browse the contents of the NEXRAD Level II archive via the AWS JavaScript S3 Explorer. Visit the project’s Github page if you’d like to use this application to make the contents of your S3 buckets easy to browse via a web browser.
Other tools for accessing this data include NOAA’s Weather and Climate Toolkit and Unidata’s Integrated Data Viewer. There are more tools available at: http://www.ncdc.noaa.gov/data-access/radar-data/radar-display-tools.
You can also:
- Start an Amazon EC2 instance and copy the data from the S3 bucket to the storage on the instance (either local storage or Amazon EBS storage). You can use the AWS Command Line Interface to do this.
- Use an AWS SDK (http://aws.amazon.com/tools/) to write a script that interacts with the S3 bucket.
- Mount the entire bucket as a file system on Amazon EC2 instances.
“Volume scan” refers to the data collected by the Doppler radar site as it scans the atmosphere. The NEXRAD site breaks these volume scans into “chunks” – small packages of data that are quickly transmitted as a real-time feed.
We are storing the real-time chunks and archive (volume scan files) Level II data in two public Amazon S3 buckets. Data flows into the real-time bucket via Unidata’s Local Data Manager (LDM) system with minimal latency from the NEXRAD sites. The chunks are then assembled into volume scan files and added to the archive bucket within seconds or minutes of production. This creates a continuously updated, near-real-time archive of volume scan files.
The real-time bucket retains the latest 24 hours of chunks data and the archive bucket retains all the volume scan files.
The script used to assemble the volume scan files is available at: https://github.com/Unidata/ldm-alchemy.
| Source | National Oceanic and Atmospheric Administration |
| Category | Earth Science, Sensor Data, Natural Resource, Meteorological |
| Format | Binary format specific to NEXRAD |
| License | There are no restrictions on the use of this data. NOAA’s Digital Object Identifier (DOI) for this data is: doi:10.7289/V5W9574V. More information on NEXRAD data citation and terms of use is available from NOAA. |
Storage Service |
Amazon S3 |
| Location | s3://noaa-nexrad-level2 s3://unidata-nexrad-level2-chunks In US Standard (N. Virginia) |
Update Frequency |
New Level II data is added as soon as it is available. |
