next up previous contents FITSIO Home
Next: 5. Basic CFITSIO Interface Up: 4. Programming Guidelines Previous: 4.12 When the Final   Contents

4.13 CFITSIO Size Limitations

CFITSIO places very few restrictions on the size of FITS files that it reads or writes. There are a few limits, however, that may affect some extreme cases:

1. The maximum number of FITS files that may be simultaneously opened by CFITSIO is set by NMAXFILES as defined in fitsio2.h. It is currently set = 300 by default. CFITSIO will allocate about 80 * NMAXFILES bytes of memory for internal use. Note that the underlying C compiler or operating system, may have a smaller limit on the number of opened files. The C symbolic constant FOPEN_MAX is intended to define the maximum number of files that may open at once (including any other text or binary files that may be open, not just FITS files). On some systems it has been found that gcc supports a maximum of 255 opened files.

Note that opening and operating on many FITS files simultaneously in parallel may be less efficient than operating on smaller groups of files in series. CFITSIO only has NIOBUF number of internal buffers (set = 40 by default) that are used for temporary storage of the most recent data records that have been read or written in the FITS files. If the number of opened files is greater than NIOBUF, then CFITSIO may waste more time flushing and re-reading or re-writing the same records in the FITS files.

2. It used to be common for computer systems to only support disk files up to 2**31 bytes = 2.1 GB in size, but this barrier is gradually being broken. CFITSIO can optionally read and write these so-called 'large files' that are greater than 2.1 GB on platforms where they are supported, but this usually requires that special compiler option flags be specified to turn on this option. On linux and solaris systems the compiler flags are '-D_LARGEFILE_SOURCE' and `-D_FILE_OFFSET_BITS=64'. These flags may also work on other platforms but this has not been tested. Starting with version 3.0 of CFITSIO, the default Makefile that is distributed with CFITSIO will include these 2 compiler flags when building on Solaris and Linux PC systems. Users on other platforms will need to add these compiler flags manually if they want to support large files. In most cases it appears that it is not necessary to include these compiler flags when compiling application code that call the CFITSIO library routines.

When CFITSIO is built with large file support (e.g., on Solaris and Linux PC system by default) then it can read and write FITS data files on disk that have any of these conditions:

The current maximum FITS file size supported by CFITSIO is about 6 terabytes (containing 2**31 FITS blocks, each 2880 bytes in size). Currently, support for large files in CFITSIO has been tested on the Linux, Solaris, and IBM AIX operating systems.

Note that when writing application programs that are intended to support large files it is important to use 64-bit integer variables to store quantities such as the dimensions of images, or the number of rows in a table. These programs must also call the special versions of some of the CFITSIO routines that have been adapted to support 64-bit integers. The names of these routines end in 'll' ('el' 'el') to distinquish them from the 32-bit integer version (e.g., fits_get_num_rowsll).


next up previous contents FITSIO Home
Next: 5. Basic CFITSIO Interface Up: 4. Programming Guidelines Previous: 4.12 When the Final   Contents