• Realtime News
• Introduction
• General differences between realtime and production data
• Preparing your realtime data for analysis

New! Realtime Pseudo-Production Data is Better!

1. Quick FITS files produced within a few hours of the end of an observation.
2. Quick pseudo-production level files made approximately 1 day after the end of an observation.
3. Full production available within a couple of weeks of an observation
   The changes being made to the FITS files will only affect type 2) above, the quick pseudo-production level files.

   Previously both types 1) and 2) differed from the production data in that several keywords were missing from the FITS files and the directory structure differed.

   The new quick pseudo-production files will:

   • - contain previously omitted keywords: RA, DEC and OBSID
   • - the directory structure will now be identical to the full production files.
   • - Small changes in the data format have been made to make them fully FITS compliant. e.g. floating point numbers previously written as xx.xxxe-10 are now written as xx.xxxE-10 These changes will also be made to other FITS products at a later date.
   With these changes several of the additional steps previously required to analyze quick pseudo-production level data described in this recipe are no longer needed. These are steps:
   1. It is no longer necessary to insert RA and dec. into the index file.
   3. You no longer need to determine the "real" ObsID.

   ---

   Introduction

   Realtime data are "quick look" products, useful for general analysis of an RXTE observation before the fully-processed data become available. (Fully-processed data are usually available 1-2 weeks after observation.)

   Although much data analysis can be performed with Realtime Data, you should always redo your analysis with the fully-processed production data, when it becomes available.

   All RXTE data are available at three different stages of processing;

   1. "Rapid" (same-day) Realtime Data: available within a few hours of observation, from the XTE SOF's realtime data archive. This is the first-look data, and may possibly contain telemetry gaps. These data are available for 7 days, or until the Production-quality Realtime Data become available.

   2. "Production-quality" Realtime Data ("Pseudo-Production" Data): available 2-3 days after observation, from the XTE SOF's pseudo-production data archive. These data are the complete production-quality data, that will later be run through full processing (as such, they may be slighly more complete than the same-day realtime data). Data remain on this site for roughly 30 days.

   3. Fully-Processed Production Data: these are the processed data delivered to PIs by the XTE GOF via e-mail notification, and to the RXTE Public Data Archive. They are generally available 8 days after observation (delivery to the data pick-up area occurs Monday-Friday, in the late afternoon, US East Coast time). Note that this means Pseudo-Production and Fully-Processed Production data may be offered simultaneously, for a limited period of time.

   Realtime data are in the same FITS format as production data and can be reduced and analyzed in the same way. However, there are some general differences, explained in the following section, of which you should be aware. The recipe itself describes the additional steps needed to prepare your realtime data for analysis.

   ---

   General Differences between Realtime and Production data

   As the name implies, realtime data are telemetred when the spacecraft is in contact with the ground station. Production data, on the other hand, comprise daily batches of realtime data put together by the data capture facility. These data have been time-sorted and with duplicate telemetry removed. Consequently, users should be aware of the following caveats concerning realtime data:

   1. Start and stop times of these observations are estimated to the best of our (planning) knowledge, but may not be entirely accurate.

   2. There may be gaps in the real-time data caused by telemetry interruptions which will not be present in the production data.

   3. Orbit ephemerides are not yet available for these observations; one will have to wait for the definitive products.

   4. Complete filter files can be generated from realtime data, but only with a few additional steps (see recipe below).

   ---

   Preparing your Realtime Data

   This recipe boils down to making a filter file and PCA response. First the filter file:

   1. Insert the target RA & Dec into the stdprod index file: The program that generates the filter file, xtefilt, needs the target position in order to calculate the instantaneous pointing offset (the OFFSET column in the filter file). It gets this information from the RA_OBJ and DEC_OBJ keywords in the primary header of the stdprod index file. In realtime data, the RA and Dec are both zero and should be replaced with their correct values:
      1. Find the stdprod index file. This is a file beginning with FIST at the ObsId level, e.g. 20077-01-13-00/FIST_5e47bf5-5e488d9.

      2. Create an ASCII file, containing the correct RA and Dec in decimal degrees, like this example:

            RA_OBJ 266.137909
            DEC_OBJ -28.741100
         

         We'll call it position.

      3. Run the ftool fmodhead to insert the new values into the primary header of the stdprod index file:

            fmodhead FIST_5e47bf5-5e488d9+0 position
         

   2. Create a list of AppIds for filtering: Xtefilt requires an ASCII list of application IDs and columns from which it obtains data to make the filter file. The standard list can be found be typing fhelp xtefilt. We'll call the AppId list realtime_applist.

   3. Determine the "real" ObsId from the FMI file: Xtefilt needs to know the location of the files containing the listed AppIds. It gets this information first by looking at the FMI file which contains pointers to the subsystem index files which in turn point to the files themselves. Since xtefilt makes one filter file per ObsId, and since an FMI file can sometimes point to more than one ObsId, xtefilt also needs the ObsId.

      However, the ObsId specified in the FMI is not the same as the one implied by the name of the tar file (which is also the ObsId the production data will be known by). Rather, realtime ObsIds are denoted with a slightly different convention. For example, the ObsId that appears to be 20077-01-13-00 is listed in the FMI file as D1144-05-07-01R. To get the realtime ObsId, either run XDF and look in the "Observations" window or run fdump on the FMI file, like this:

      
         fdump mode=h prhead=no infile=20077-01-13-00/FMI columns=ObsId
      
                 ObsId
      
               1 D1144-05-07-01R
      

   4. Create the filter file: Here, we'll create the filter file pulburst_13.xfl in the current directory using the files we created above. Default binning of 16 seconds is used.

      
         xtefilt -f pulburst_13
      
         Running XTEFILT v1.3
         ##########################################
      
         Observation ID: [] D1144-05-07-01R
         Path to XTE FITS Database: [] /data/realtime/20077-01-13-00
         File containing AppID list: [] realtime_applist
         Timestep: [16.0]
      

      The resulting file has the same columns (bar the HEXTE particle monitors) as one generated from production data. But it will likely have gaps corresponding to periods when the spacecraft was not in contact with the ground station.

   And now for the response matrix:
   1. Extract your spectrum: Follow the usual procedure which, for PCA data, is described in the recipe Reduction and Analysis of PCA Spectra. However, please be aware of the following caveats:
      • Realtime datafiles have only one GTI extension, containing those times when telemetered data were present. These GTI are applied by answering "APPLY" to the saextrct prompt Input GTI files to be OR'd.

      • Realtime slew data are not separated into their own ObsIds. To ensure that your data are entirely on source, we recommend creating a GTI file from your filter file with the ftool maketime. Like this:

        
           maketime
           Name of FITS file and [ext#][] pulburst_13.xfl
           Name of output FITS file[] on_target.gti
           Selection Expression[] OFFSET.lt.0.02
           Column containing HK parameter names[NAME]
           Column containing HK parameter values[VALUE]
           Column containing HK parameter times[TIME] Time
           Flag, yes if HK format is compact[no]
        

        The criterion used here is that pointing offset be less than 0.02 degrees. The file should be input at the saextrct prompt Input GTI file to be AND'd with INFILE.

   2. Insert the target RA & Dec into the PHA file: The program that generates the response, pcarsp, needs the target position in order to calculate the effective area. It gets this information from the RA_OBJ and DEC_OBJ keywords in the first extension header of the PHA file. In realtime data, the RA and Dec are both zero and should be replaced with their correct values. The procedure is the same as for inserting the target RA & Dec into the stdprod index file, as described above. Indeed, the same ASCII file can be used. Note, however, that the "+0" should be omitted, e.g.:

         fmodhead flare.pha position
      

      [This step is not strictly necessary if the previous step is followed: Selecting only on-target data means, in effect, that the target is at the center of the field of view, in which case "NONE" can be given to pcarsp as the attitude file.]

   3. Run pcarsp: This is described in the recipe Creating PCA Responses.

   ---

   If you have a question about RXTE, please send email to one of our help desks.

   This page is maintained by the RXTE GOF and was last modified on Wednesday, 20-Aug-2008 15:11:40 EDT.