############################################################################## AutoDRM User's Guide National Earthquake Information Center U. S. Geological Survey ------------------------------------------------------------------------ This Automatic Data Request Manager (AutoDRM) has been established to provide data sets collected or derived at the National Earthquake Information Center (NEIC). Primary data sets include waveform data collected from the U. S. National Seismograph Network (USNSN) and bulletin information derived by the National Earthquake Information Service (NEIS). The AutoDRM also provides "beta" data for the United Nations Committee on Disarmament, Group of Scientific Experts (GSE) Technical Test (GSETT-3) in partial fulfillment of the NEIC's role as an integral part of the U. S. preliminary National Data Center (NDC). Note that other real-time and near real-time waveform data sets collected at the NEIC to support the NEIS Earthquake Early Alerting Service mission are also available through the AutoDRM. This includes real- time short period data from the U. S. regional seismograph networks and the Global Telemetered Seismograph Network (GTSN) and near real-time (1-2 hr) waveform data from the Incorporated Research Institutions for Seismology (IRIS) Data Management Center (DMC) Sypder program. The AutoDRM has been completely rewritten (first phase completed on 13 June 1995) to meet the GSE2.0 specifications for command language and response format. All commands available in the previous version of the AutoDRM will continue to work and will produce output in the same format. However, there are significant differences in the over all output formatting and there are significant extensions to some of the commands. To understand these changes it is important to establish some nomenclature. The previous version of the AutoDRM was derived (circa 1 March 1994) from the original AutoDRM developed by Dr. Urs Kradolfer for GSETT-2. Thus, the command language of the old AutoDRM will be referred to as the GSE1.0 version. Output formats developed either for GSETT-2 and adopted in the original AutoDRM or developed for the original AutoDRM will also be referred to as GSE1.0 formats. Formats specific to the old NEIC AutoDRM will be referred to as USN1.0 formats. The new AutoDRM supports commands in both the GSE1.0 and GSE2.0 formats, but the output structure adheres to the GSE2.0 specification. In this scheme, the response has a strictly formatted prologue starting with a BEGIN statement and some message identification lines. Following this, different types of information are provided following DATA_TYPE identifiers. On the DATA_TYPE line there are one or two arguments, the first specifying the type of data that follows and the second (if applicable) specifying the format of the data. The response is terminated with a STOP statement. In the GSE1.0 mode, all data formats default to GSE1.0 or USN1.0 (note that the SLIST and CLIST command output has been changed and is now designated USN1.1) as appropriate to maintain backwards compatibility. In supporting the GSE2.0 command language, some extensions were necessarily made to the GSE1.0 command language as well. In particular, all commands are now executed as they are encountered and complete UNIX style ? and * wildcarding are supported for all station and channel identification fields (be careful of using just a * for station and/or channel though---there are over 140 stations and almost 800 channels available now). Processing the commands in order means that it is now possible to set a date-time range, extract data, set a new date-time range, extract more data, etc. However, it also means that if you set the FTP or LARGE options after a WAVEF command that exceeds the E-mail response length limit, your request will fail. While most of the data formats used are self explanatory, the waveform and response formats may be less obvious. Currently, most waveforms are returned in the GSE1.0 format with the CMP6-2 compression. Most of the important information in the WID1 header is fairly obvious except for the first three fields on the second line which are sensitivity (columns 1-9), response type flag (column 10), and reference period (columns 11-17). The sensitivity gives absolute nm/count, nm/s/count, or nm/s/s/count at the reference period for response type flag values of 0, 1, and 2 respectively. A program (CODECO) for interpreting the GSE data formats is available through the SOURC command. Responses are also returned in the GSE1.0 format given as poles and then zeros in radians and a scale factor in counts/nm. Together, these factors specify the Laplace transformed transfer function which converts ground displacement into the digital seismic channel output. Note that FIR filter contributions are neglected. A program which extracts response and waveforms from AutoDRM response files and converts them into various other formats (including SAC) is also available through the SOURC command (CVTGSE). It is also now possible to retrieve waveforms and responses in the GSE2.0 format. Again the data is returned in the CMP6-2 compression, but the header is in the WID2 format. In this case, the last seven fields are: number of samples, sample rate, sensitivity in nm/count, reference period, instrument type, component azimuth, and vertical orientation (plunge + 90 degrees). The GSE2.0 responses are headed with a CAL2 line which includes the sensitivity, reference period, and sample rate. The poles and zeros are headed with a PAZ2 line which is new. It includes the scale factor and numbers of poles and zeros, followed by the poles and then the zeros themselves. All units are the same as for the GSE1.0 responses. A version of CVTGSE which supports both the GSE1.0 and GSE2.0 responses and waveforms will be available shortly. This users guide is divided into four parts: part 1 is about electronic mail, part 2 defines the complete GSE1.0 command set, part 3 gives a brief description of the GSE2.0 commands currently supported, and part 4 gives illustrative examples of some request mail messages. 1) Sending and Receiving Electronic Mail from AutoDRM ===================================================== In order to obtain data from the NEIC, simply send an E-mail request via the InterNet to the AutoDRM. The AutoDRM automatically reads request mail sent to the address: autodrm@usgs.gov. The request E-mail must contain commands specifying what data are desired. Commands are case independent (they are shown in upper case here for clarity), but must begin in column one. The first request statment must be BEGIN. No AutoDRM commands except HELP and PLEASE HELP will be processed before a BEGIN. The last request statement must be a STOP. All requests should also include an EMAIL statement specifying the E-mail address to which the response should be sent. Note that even if you request that the response be sent to you by ftp, an E-mail response is still generated telling you the status of the request and where the response can be found. If no EMAIL command is found, the new AutoDRM will attempt to send the response to your return address. This can fail as it is possible to send E-mail with an invalid return address. When entering your return address in the EMAIL command, please BE CAREFUL! Typographical errors in the return addresses are the leading cause of failed AutoDRM responses. If you are not sure of the exact form of your return address, try omitting the EMAIL command to see if the AutoDRM can figure it out for you. Generally, the response of the AutoDRM is sent to you via electronic mail. Since some E-mail systems can fail for messages longer than about 100 kilobytes, the AutoDRM will enforce this limit by truncating the response unless the limit is increased using the LARGE command (a limit of about 1 megabyte) or the response is requested via ftp (File Transfer Protocol). It is also possible to split requests by giving several different, completely separate requests (each delimited by their own BEGIN and STOP commands) in the same E-mail request. Each request will generate a response which will be transmitted as a separate E-mail message. With the FTP and INTER commands it is possible to arrange for the response to be prepared and left in the AutoDRM's local anonymous ftp area (for the user to "pull") or sent ("pushed") to the anonymous ftp area on the user's local server. If both the FTP and INTER commands are used, a "push" is specified. If only the FTP command is given, a "pull" is inferred. A short E-mail will be sent for ftp requests with information on how the transfer went or on where the file is and how to get it. Therefore, every request should include one of the following combinations of commands: 1) EMAIL (response returned by E-mail), 2) EMAIL and FTP (response left on the AutoDRM server to be "pulled" by the user), or 3) EMAIL, FTP, and INTER (response to be "pushed" to the user's server). Note that the format of the response is the same for ftp as for E-mail. Ftp requests have a response limit of 10 megabytes. 2) GSE1.0 AutoDRM Commands ========================== Command line: Meaning: ------------ ------- BEGIN Must precede all other AutoDRM commands. If the BEGIN command is not found, no processing will be done. EMAIL address Establishs the return address to which the mail containing the AutoDRM response is to be sent. FTP filename Specifies that the response be written to the file name given for transfer to the user via ftp. If no file name is given, the AutoDRM will create one. INTER IP_number In conjunction with the FTP command, INTER specifies the numerical InterNet address to which the response should be "pushed" via anonymous ftp. TITLE your subject Sets the subject line for the response mail (optional). SUBJE your subject Same as TITLE. LARGE Changes the default limit for a response by E-mail from 100 kilobytes to 1 megabyte. QUIET Suppresses error messages (not recommended). GUIDE Send this (User's Guide) information. INFOR Same as GUIDE. HELP Same as GUIDE. AVAIL Send a list of time intervals for which data is currently available (USN1.0 format). AMI A Send a list of the 100 most recent locations of the "Alert Message Information" received (GSE1.0). At the NEIC, this is the same near real-time bulletin of reviewed locations available from the "finger quake@usgs.gov" list. SLIST arg Send a list of stations for which waveforms and/or calibration data is available (USN1.1). The argument specifies the sub-list desired. Possibilities are: "NSN" for USNSN broadband stations only, "DRM" (default) for all real-time broadband data, "BB" for all available broadbands (including spydered IRIS stations), or "ALL" for all of the above as well as the short period vertical and long period stations from the regional seismograph networks recorded at the NEIC (real-time). CLIST Send a list of channels for which waveforms and/or calibration data is available. The association between USNSN net, node, and channel IDs and the station-channel names is shown as well as the coordinates for the station (USN1.1). DATE1 yyyymmddhhmm Define the start of a date-time interval for subsequent commands. The date-time is given as year, month, day, hour, and minute. For example: 24 Feb 1992 15:46 would be 199202241546. DATE2 yyyymmddhhmm Define the end of a date-time interval for subsequent commands. DETEC Send a list of all events located within the specified DATE1-DATE2 time interval. At the NEIC, this is implemented as all automatic and revised locations determined by the automated USNSN processing system (GSE1.0). PPICK stn Send arrival estimates from station "stn" for all detected phases within the DATE1-DATE2 time interval (GSE1.0). OUTAG stn Send a list of periods for which station "stn" was not recording data during the DATE1-DATE2 time interval (USN1.0). WAVEF stn chn Send waveforms from station "stn", channel "chn" within the DATE1-DATE2 time interval. Since most of the broadband and short period data available is triggered, one request may result in several waveforms returned. Standard for the Exchange of Earthquake Data (SEED) format conventions are followed for channel names. If no channel is given, BHZ is assumed (GSE1.0). CALIB stn chn Send response information valid for station "stn", channel "chn" at time DATE1. If DATE1 has not been set, the current time is used. If no channel is given BHZ is assumed (GSE1.0). SOURC program Send FORTRAN source code of AUTODRM related programs. Currently two programs are available, CODECO, which allows the conversion between various GSE-formats and CVTGSE which parses the results of CALIB and WAVEF commands and converts the results to either the RETRV, SAC, or GSE formats. STOP Terminates the command sequence. 3) GSE2.0 AutoDRM Commands ========================== GSE2.0 command sequences should always start with the BEGIN, MSG_TYPE, and MSG_ID commands (though the use of MSG_TYPE and MSG_ID are not enforced in this implementation). In the GSE1.0 format, the DATE1 and DATE2 commands establish an "environment" which affects subsequent commands. This concept is greatly expanded in the GSE2.0 format. All commands which cause data to be returned take only the format in which the data is desired as arguments. The networks, stations, channels, and/or date-time interval desired are all set in the environment via the AUX_LIST, STA_LIST, CHAN_LIST, and TIME commands prior to the actual request command. An auxiliary ID (set with AUX_LIST) is made up of the station's two character Federation of Broadband Seismograph Networks (FDSN) standard network code concatenated with the station's SEED location code. Because all stations available through this AutoDRM have blank location codes, the auxiliary ID serves as a network code only. Station codes are all internationally registered and channel codes are all SEED standard. If the AUX_LIST is set to "*", the STA_LIST and CHAN_LIST work in the obvious way (e.g., STA_LIST set to LBNH and CHAN_LIST set to BH* specifies the three broadband components of station LBNH). However, the AUX_LIST provides a very powerful selection tool for channel selection and the CHAN_LIST can modify the selection of stations in very useful ways. For example, AUX_LIST set to US, STA_LIST set to *, and CHAN_LIST set to BHZ specifies all USNSN stations that have a broadband vertical component or all BHZ channels belonging to USNSN stations. The other significant difference in the GSE2.0 format is that E-mail responses may be continued. That is, if the response exceeds 100 kilobytes, it is terminated with a CONTINUED line rather than a STOP line and transmitted. The message is then continued is a new response file started with a CONTINUATION line rather than a BEGIN line. Up to 9 continuations are allowed for a total message length of 1 megabyte. The last continuation is terminated with a STOP line. Note that all parts of one response are identified with the same unique response identification code (given on the MSG_ID or CONTINUATION lines) and that continuations are given sequence numbers (also on the CONTINUATION line). Command line: Meaning: ------------ ------- BEGIN GSE2.0 Must precede all other AutoDRM commands. If the BEGIN command is not found, no processing will be done. MSG_TYPE REQUEST Identifies this as an AutoDRM request. MSG_ID msg_id source Identifies the request with a unique identification and request source. If given, the msg_id and source are included in the response in a REF_ID line. E-MAIL address Establishs the return address to which the mail containing the AutoDRM response is to be sent. FTP address Requests that the response be copied into the AutoDRM server ftp anonymous area for the user to "pull". The argument is the E-mail address to which the accompanying E-mail response is to be sent. Note that GSE2.0 ftp responses are never "pushed" and that the AutoDRM always makes up the ftp file name. Also, note that in the GSE2.0 format, either FTP or E-MAIL is required, but not both as it is for the GSE1.0 format. HELP Send this (User's Guide) information. TIME start TO end Define the environment date-time interval. Start and end are date-times in the format yyyy/mm/dd hh:mm:ss.sss (year, month, day, hour, minute, second). Although it is legal to omit either the start or end date-time or both, the infinite time interval implied is not currently supported by any available command. When a date-time is given, a complete date is required. However, trailing time fields omitted (including the hour) will be taken to be zero. For example: 24 Feb 1992 15:46 would be 1992/02/24 15:46. Effectively, there is no default TIME interval. AUX_LIST aid, aid,... Set the environment to include the auxilary IDs listed. Wild cards are supported in each entry. This list replaces the current AUX_LIST environment. Long lists can be specified by continuing the input line using a backslash ("\") character at the end of each line to be continued. The AUX_LIST default is *. STA_LIST sta, sta,... Set the environment to include the stations listed. Wild cards are supported in each entry. This list replaces the current STA_LIST environment. Long lists can be specified by continuing the input line using a backslash ("\") character at the end of each line to be continued. Note that the STA_LIST default is none. CHAN_LIST chn, chn,... Set the environment to include the channels listed. Wild cards are supported in each entry. This list replaces the current CHAN_LIST environment. Long lists can be specified by continuing the input line using a backslash ("\") character at the end of each line to be continued. The CHAN_LIST default is *. STATION fmt Send a list of stations in the format specified which match the current STA_LIST environment. Default GSE2.0 format. CHANNEL fmt Send a list of channels in the format specified which match the current STA_LIST and CHAN_LIST environments. Default GSE2.0 format. EVENT fmt Send a list of all automatic and revised locations determined by the automated USNSN processing system within the current TIME environment interval in the format specified (GSE1.0). ORIGIN fmt Currently a synonym for EVENT. DETECTION fmt Send arrival estimates in the specified format for all detected phases from stations matching the STA_LIST, channels matching the CHAN_LIST, and within the TIME interval set in the environment (GSE1.0). ARRIVAL fmt Currently a synonym for DETECTION. OUTAGE fmt Send a list of periods for which stations matching the STA_LIST were not recording data during the TIME interval set in the environment in the format specified (GSE2.0). WAVEFORM fmt sfmt Send waveforms in the requested format and sub-format from stations matching STA_LIST, channels matching CHAN_LIST, and within the TIME interval set in the environment. The sub-format is the specific data format in cases where the format type supports more than one data format (e.g., GSE2.0 supports CMP6-2, denoted as CM6, among other formats). Default GSE2.0. RESPONSE fmt Send response information in the specified format valid for stations matching STA_LIST, channels matching CHAN_LIST, and at the begin TIME set in the environment. If a start TIME has not been set, the current time is used. Default GSE2.0. STOP Terminates the command sequence. NOTE: - The LINE_LEN command is recognized, but currently has no effect on the output produced. - The powerful ASSOCIATE command is not currently implemented. - GSE2.0 data formats for the STATION, CHANNEL, WAVEFORM, RESPONSE, and OUTAGE commands have been implemented. The old formats can still be requested. All other GSE2.0 commands return data in the same format as the corresponding GSE1.0 command. 4) Examples of AUTODRM mail files one could send to autodrm@usgs.gov ============================================================================== Example 1: --------- Many users send this kind of mail regularly (usually on a daily basis) to us (though one copy of the guide should be enough for anyone). Command line: Meaning: BEGIN Start of a request GUIDE Send the information printed here AMI A Send a list of the associated "Alert Messages" EMAIL juser@mine.anyu.edu E-mail address where output should be sent STOP End of a request Example 2: --------- The FTP and INTER commands in this example are not required; without them you will receive the full response via E-mail. Command line: Meaning: BEGIN Start of a new request DATE1 199307180055 Start of time interval: 1993Jul18 00:55 GMT DATE2 199307180056 End of time interval : 1993Jul18 00:56 GMT DETEC Send the detections made within time interval PPICK CBM \ Send all available P-picks from the stations PPICK CEH / CBM and CEH for the specified time interval FTP pub/incoming/out.fil Pathname where response should be written INTER 5.5.5.123 Internet-address of the user's computer EMAIL juser@mine.anyu.edu E-mail address where output should be sent STOP End of the request Example 3: --------- This sample mail shows how to retrieve waveform and calibration data. Command line: Meaning: BEGIN Start of a new request DATE1 199307190800 Start of time interval: 1993Jul19 08:00 GMT DATE2 199307192030 End of time interval : 1993Jul19 20:30 GMT WAVEF CBM BHZ \ Send all waveforms from the stations CBM and WAVEF CEH BHZ / CEH recorded within the time interval CALIB CBM BHZ Send calibration data for station CBM EMAIL juser@mine.anyu.edu E-mail address where output should be sent STOP End of the request Example 4: --------- Examples 2 and 3 repeated using the GSE2.0 command format. Note that the format returned to you will be identified on the DATA_TYPE line in the response, but may not be the format you requested (if it is not supported). Command line: Meaning: BEGIN GSE2.0 Start of a new request MSG_TYPE REQUEST Identify this as a request message MSG_ID ID_0001 GSE_IDC Set the message ID and source FTP juser@mine.anyu.edu \ E-mail address where acknowledgement should / be sent TIME 1993/07/18 00:55 TO 1993/07/18 00:56 \ Set time interval: / 1993Jul18 00:55 GMT to 1993Jul18 00:56 GMT EVENT GSE1.0 Send events located within time interval AUX_LIST GT Set the AUX_LIST to GTSN stations STA_LIST C*, P* \ Set the STA_LIST to stations starting with / the letters C and P DETECTION GSE1.0 \ Send all available P-picks from the stations / CPUP and PLCA for the specified interval TIME 1993/07/19 08:00:00.000 TO 1993/07/19 20:30:00.000 \ Set time interval: / 1993Jul19 08:00 GMT to 1993Jul19 20:30 GMT CHAN_LIST BH* Set the CHAN_LIST to all broadband channels WAVEFORM GSE2.0 CM6 \ Send all broadband waveforms from stations / CPUP and PLCA recorded within the interval STA_LIST CBM Reset the STA_LIST to CBM CHAN_LIST LHZ Reset the CHAN_LIST to LHZ RESPONSE GSE2.0 Send responses data for the channel CBM/LHZ STOP End of the request The command set will be extended in the near future. It is recommended to use the command GUIDE from time to time in order to be up to date with the newest implementations of the NEIC/USA_NDC AutoDRM. Suggestions/complaints please via E-mail to: buland@usgs.gov Golden, 10 July 1995 / rb ##############################################################################