Sustainability of Digital Formats
 Planning for Library of Congress Collections

Scope
The factors discussed here apply to still images that convey their meaning in visual terms, e.g. pictorial images, photographs, posters, graphs, diagrams, documentary architectural drawings.  Formats for such images may be bitmapped (sometimes called raster), vector, or some combination of the two.  A bitmapped image is an array of dots (usually called pixels, from picture elements, when referring to screen display), the type of image produced by a digital camera or a scanner.  Vector images are made up of scalable objects—lines, curves, and shapes—defined in mathematical terms, often with typographic insertions.  Graphic design software may be primarily of the paint variety and intended to produce bitmapped images or primarily of the draw variety and intended to produce vector images.  Formats may combine vector and bitmapped data, either in layers that will be combined at display or output, or in a single flat representation (for example, allowing a vector-defined shape to be "filled" with bitmapped image data). Some combination formats have been termed metafile formats.

This discussion concerns a variety of media-independent still image formats—bitmapped and vector—and their implementations. Some formats, e.g., JPEG 2000, allow for many different implementations compared to, say, GIF_89a, a format whose uses are relatively more constrained. Within the overall range, many implementations are for end-user applications, intended for home or classroom use, Web sites, and the like. At the other end of the range, are implementations for specialized professional applications,¹ some of which are relevant to preservation undertakings carried out by archives.

Not covered here are factors that might be significant for special classes of images in a particular context, such as geospatial representations, medical imagery, or images of pages that are primarily of text.  One omitted category, for example, is that of "drawings" produced by the design systems that support a building or manufacturing process, e.g., computer-aided design (CAD) or manufacturing (CAM). In some cases, the Library will prefer to collect examples of CAD-CAM drawings in "flattened" form, e.g., rendered as a bitmap. In other cases, however, the Library will wish to possess copies that retain process-support functionality, e.g., the ability to move or change lines. This desire, or an analogous desire regarding other special categories, may lead us to identify quality and functionality factors beyond those listed here at a later time or in other parts of this Web site. 

Albeit indirectly, this discussion does provide information pertaining to compound documents, which integrate still images with other forms of expression. For example, many PDF (Portable Document Format) files combine images with character-based (non-bitmapped) text.  The factors outlined here regarding still images also pertain to the pictorial or illustrative elements in a PDF file. Meanwhile, the factors that pertain to text in a compound document are described in Text: Quality and Functionality Factors and in individual format descriptions like the one for PDF. Most aspects of this discussion of still images also apply to images embedded in multi-page formats; see Functionality Beyond Normal Rendering below.

Normal rendering
Normal rendering for still images is associated with end-user implementations, and consists of on-screen viewing and printing to paper for personal or classroom use.  For many classes of images, user communities will demand the ability to zoom in to study detail, to scale images to display at different sizes or on devices of different resolution, and/or the ability to produce publication quality output.  Freezing of vector images as bitmaps of adequate resolution can still support normal rendering, as can flattening a layered image by combining the layers.

For formats implemented in specialized professional applications, the same type of normal rendering may not obtain, although most professional image editing systems facilitate display or printing in a manner comparable to that described in the preceding paragraph. The underlying images in these professional applications, however, may be in a format, e.g., camera raw, for which specialized output or display devices are required. For many professional-image-format implementations, normal rendering will be afforded by a service file that has been produced from a master image in a specialized format.

Clarity (support for high image resolution)
Clarity refers to the degree to which "high resolution" content may be represented within this format.  In this context, the term clarity is meant broadly, referring to the factors that will influence a careful (even expert) viewing experience.  Generally speaking, this factor pertains to bitmapped representations and not to vector-based images like Scalable Vector Graphics SVG_1_1 files, which are inherently scalable and often employ color in a fully managed way. (However, see comment about "clean edges," below.) Clarity is tested in a practical sense when the reproduction is repurposed, e.g., used as the basis for a new printed publication, or when an online user enlarges an images to see the fine detail. 

For bitmapped images, the two characteristics most often associated with clarity are pixels per linear unit (often colloquially expressed as "dots per inch") and bit depth ("bits per pixel" or "bits per color channel").  As an example of the latter, an 8-bit indexed-color file (the GIF format limit) offers less clarity than a 24-bit "true-color" image (supported by many other formats), at the same spatial resolution (dots per inch).  This is due to the contouring or posterization that often occurs when bitmapped images are represented with low bit depth.

For specialized professional applications, clarity is often of great importance. In some cases, clarity is—to stretch a photographic analogy—developed and visible: an image in a given format successfully represents an extended dynamic range or color gamut. In other cases, clarity is latent: a file contains data that can be processed to yield one or more image outputs that provide desired values. Examples of the latter include camera raw images, extended-data-range images, images with "linear gamma," and so on. Meanwhile, professional image applications may exploit specialized color encodings like CIELAB, CIE XYZ, scRBG, RIMM and ROMM RGB, Adobe RGB, and what is variously called multispectral or hyperspectral imagery. The latter is generally associated with remote sensing (e.g., images captured by satellites) but is sometimes employed in other contexts, including the reproduction of fine art.

Clarity in a broad sense will also be adversely affected by the presence of the visible artifacts that may result from the application of lossy compression or watermarking.  (Note that compression as it detracts from the sustainability evaluation factor transparency is cited in the definition of that factor in Sustainability Factors.)  If lossy compression is applied, the quality of the particular codec (compression/decompression algorithm and software) becomes significant to clarity.

The scalability of vector images means that clarity is generally not an issue; they are designed to be enlarged, although the enlargement of combination images or metafiles may be limited by the resolution of their bitmapped elements.  In some vector formats, the creator can influence the look of a rescaled on-screen presentation, for example, by specifying whether display software should favor "clean edges" (which may entail the slight repositioning of lines or the edges of shapes) or "geometric precision." 

Some image formats allow a single image to be stored as a set of independent layers that are super-imposed to create the final image.  Image-element layering is particularly important to creators, and support for layering is mentioned below in Functionality Beyond Normal Rendering. In many cases, images are "flattened" into a single layer at the end of the creative process.  In other cases, however, the creator will transmit a layered image for use.  For example, prepress images used to create printing plates may employ layers tailored to characteristics of the printing process, such as higher spatial resolution for line art layers than for continuous tone images or the representation of monochrome color layers, e.g., what printers call "spot color," independent of four-color layers.  Retention of the layered structure in such cases could contribute to clarity.

The importance of clarity for a particular subclass of images should be considered in relation to the creator's intent or the context of creation or original use.  The subtle color differentiation offered by, say, 24-bit-deep color is unlikely to be important in a graph in which the intention is distinguish a red from a blue column.  Images intended only for online display are likely to have lower spatial resolution than those intended for printing.

Color maintenance
Color maintenance refers to the degree to which the color gamut represented in a given image can be managed and maintained through various outputs or migrations.  This requirement calls for a format that can support color encoding in different colorspaces and the metadata needed by color management systems, e.g., the inclusion of a color map for indexed-color files or an ICC profile for the capture device, e.g., the digital camera or scanner, or for the artist's decisions about color, dating from the time of image creation or final edit. The management of multispectral or hyperspectral data adds requirements to both formats and applications.

During the last few years, imaging specialists have refined the concept of image states and, in 2004, this concept was described and defined in the international standard Photography and graphic technology—Extended Colour encodings for digital image storage, manipulation, and interchange (ISO 22028-1). At the highest level, there are two states: scene-referred, which "represents estimates of the color-space coordinates of the elements of a scene," and picture-referred, which "represents the color-space coordinates of the elements of a hardcopy or softcopy image." The latter state has two subcategories: original-referred, which is "typically produced by scanning artwork," and output-referred, an image "that has undergone color-rendering appropriate for a specified real or virtual output device and viewing conditions." For some photographers and users of photography, image-state information will play a role in a production workflow, e.g., when a fresh image moves from the field to the printed page. There do not appear to be well-established metadata conventions for expressing these states, nor do there appear to be file headers (or other structures) in which to inscribe such data. Comments welcome.

The terms in the preceding paragraph are defined in terms of color and this text appears in a section headed Color maintenance. Nevertheless, one can extend the concept to grayscale images and perhaps to such things as the "native" pixel count of an image when it is initially demosaiced,² e.g., in a camera. Thus it may be that we will come to consider more than color maintenance when we examine a given format's accommodation of image-state metadata.

Support for graphic effects and typography
For vector graphics, this factor refers to the degree to which the format supports the use of shadows, filters or other effects as applied to fill areas and text, offers levels of transparency, and manages the specification of fonts and patterns. 

Functionality beyond normal rendering
Various still image formats support functions beyond those mentioned above in Normal Rendering, e.g., the manipulations and outputs possible with extended data like that found in camera raw or layered images. Some formats provide multi-resolution functionality by storing the image as a series of independent arrays, each representing the image at a different spatial resolution (e.g., the Kodak ImagePak associated with the PhotoCD, or the FlashPix fpx format). More recently, multi-resolution capability has been supported much more flexibly by formats that employ wavelet compression, as in MrSID_MG3 or JP2_FF (JPEG 2000) files, as described in the following paragraph. Some formats display images progressively, revealing full clarity in stages. Another beyond-normal functionality concerns multipage images (e.g., multipage TIFF files) or simple animations (e.g., animated GIFs). Animations of only a few frames are included in this web site's still image category; longer animated works are covered in the moving image category. In some cases, the Library of Congress may select beyond-normal-rendering images for its collections, while in other cases, such images may be flattened into more typical bit-mapped form for permanent retrnation.

Additional features are associated with emerging formats like JP2_FF (JPEG 2000), and it is not clear how widely these features will be implemented in software, how broadly they will be adopted, and how significant the aspects of the viewing experience that they support will be to users.  Examples include the ability to select regions of interest (ROI) to receive higher-resolution treatment during compression and the ability to attach metadata to a point or region of an image. JPEG 2000 allows those compressing images to control the progression in which aspects of a large image file can be displayed most efficiently to a user, for example, starting with the whole image at full size but low quality or expecting the user to pick a relevant area using a small-size image and zoom in to high spatial resolution and quality for that area.  In JPEG 2000 and some other formats, ROI , quality layers, and progression order are part of a family of features that can be used for sophisticated interactive access in a client-server mode.  This interactivity is reinforced by the continuing development of associated functional specification like the JPEG 2000 Interactive Protocol standard (JPIP).  This web site will continue to monitor these emerging features and will discuss their applicability with Library staff, updating this document as needed.