halem Programming Environment

Overview of compilers and development tools

halem provides a full spectrum of programming development tools including assemblers, compilers, parallelizing compilers, and programming utilities. Tutorials are also available on many aspects of high-performance computing. The following tables show these elements of halem's programming environment.

Available compilers and compiler tools

Useful Fortran compiler options

| Top of Page |

Fortran compilers

Compaq Fortran for Tru64 UNIX Alpha Systems conforms to ANSI Fortran 95 and ANSI Fortran 90. The current version of the Fortran compiler on halem as of September 25, 2002 is Compaq Fortran Compiler V5.5. To find the installed version, Issue the command

f90 -version

The release note for the current version can be found in /usr/lib/cmplrs/fort90/relnotes90. Note that not all items in the release note are valid for the HP/Compaq SC system. For example, the -numa flag is only for the Compaq GS systems. For valid options and flags, always refer to the man page.

Usually the best performance can be obtained from

% f90 -O5 -fast -arch ev67 -tune ev67 a.f90 b.f90 c.f90

You may also try omitting the -O5 (yielding the default -O4), since loop transforms can sometimes reduce performance.

Default loaded compiler

The default loaded compiler is Compaq Fortran V5.5A-3548, Compaq Fortran Compiler X5.5A-41201-41201-48E1K.

Module commands

Module commands are available on halem for users who wish to use different versions of the Fortran compiler. However, these commands are not bundled into the standard release of halem's operating system, so some simple steps are required to enable their use.

Sourceable files that enable the use of module commands are stored in the following directory on halem:

/usr/local/Modules/3.1.6/init

To source the file that corresponds to your current shell, issue a command such as:

source /usr/local/Modules/3.1.6/init/csh

or

. /usr/local/Modules/3.1.6/init/sh

To adjust your MANPATH variable and view man page documentation, issue the command:

setenv MANPATH ${MANPATH}:/usr/local/Modules/3.1.6/man

or

MANPATH="${MANPATH}:/usr/local/Modules/3.1.6/man"
export MANPATH

The following is a description of the versions of the underlying components mapped to the names used in modules for the Fortran compiler (all items in the list below are controlled by modules). The compiler consists of:

• run time library
• hpf library
• library with extended math functions (includes cxml) (o3a)
• compiler front end
• compiler back end
• for_main.o, header files, and other generic parts of the Fortran code

The table below shows the relationships between the various versions of Fortran, their module labels, and their release numbers:

| Top of Page |

C and C++ compilers

The Compaq C compiler, formerly DEC C, is a standards-compliant, multidialect and feature rich implementation of the C language. It contains a highly optimizing code generator specifically designed to exploit the 64-bit Alpha architecture. It is particularly well suited and contains extended support for systems programming, parallel programming, and mathematical computing. The Compaq C compiler supports language extensions including:

• OpenMP parallel decomposition directives
• 32-bit pointers (64-bit pointers are the default) to reduce memory consumption and facilitate porting
• User-defined assembly language sequences

The Compaq C compiler also supports enhancements for mathematical computing such as:

• IEEE floating point
• Fast math mode
• Quad-precision (128-bit) floating point representation for long double.

The Compaq Programmer's Guide for Tru64 UNIX describes the programming development environment of the Tru64 UNIX operating system, with emphasis on the C programming language. The Compaq C for Tru64 UNIX Language Reference Manual provides reference information for using the Compaq C language on Compaq systems.

Compaq C++ for Tru64 UNIX is a native programming language product that generates optimized object code without employing an intermediate translation to C. Other C++ user documents are available in HTML format in the /usr/share/doclib/cplusplus directory on halem.

| Top of Page |

Mathematical and statistical libraries

The Compaq Extended Math Library (CXML) (formerly DXML) is a set of scientific subroutines optimized for Alpha systems, organized into the following libraries:

• BLAS - Basic Linear Algebra Subprograms
• LAPACK - An extensive library of Linear System and Eigenproblem Solvers
• Sparse Linear System Solvers - A library of direct and iterative sparse solvers
• Signal Processing - FFTs, sine/cosine transforms, convolution, correlation and filters
• SCIPORT - A library of routines compatible with Cray's SCILIB V7.

The library treats REAL data as 32-bit single precision quantity. To link to the serial library use -lcxml. To link to the parallel library, which includes SMP parallelism for many routines, use -lcxmlp. For details on CXML and a list of routines, refer to man dxml. The available man pages for each routine are at /usr/opt/XMDMAN510/man. The flag -ldxml is linked to -lcxml. Some example codes can be found in /usr/examples/cxml. For an online version of CXML, LAPACK, and Sciport routines, check the Compaq Extended Math Library Index.

The SCIPORT library is HP/Compaq's implementation of version 7 of the Cray Research scientific numerical library SCILIB. SCIPORT provides 64-bit single-precision and 64-bit integer interfaces to underlying CXML routines for Cray users porting programs to HP/Compaq systems. To aid in porting applications from Cray, the Compaq Fortran compiler supports switches to override default behavior and automatically treat all REAL, COMPLEX, and INTEGER data as the 64, 128, and 64 bits, respectively. The SCIPORT library is intended to be used in conjunction with these compiler switches to port Cray applications with minimal source changes. SCIPORT also provides an equivalent version of almost all Cray Math Library and CF77 (Cray Fortran 77) Math intrinsic routines. To use the SCIPORT library, link to either the serial or parallel CXML library after libsciport.a by issuing the following command:

% f90 -i8 -r8 -double_size 128 myprog.f -lsciport -lcxmlp

In order to be completely source code compatible with SCILIB, the SCIPORT library calling sequence supports 64-bit integers passed by reference. However, SCIPORT uses 32-bit integers internally. Consequently, some run-time uses of SciLib are not supported by SCIPORT.

Use extra caution in using the FFT routines. For performance reasons, SCIPORT FFT routines are implemented as calls to CXML FFT routines. FFT routines in CXML use different conventions from Cray SciLib in their directions (analysis/synthesis), normalization, and storage. For details, refer to man sciport.

| Top of Page |

Porting issues

Several issues need to be addressed when you are attempting to port programs from one of the other NCCS supercomputers to the HP/Compaq SC45 system environment.

Data Format

Big-Endian or Little-Endian Byte Order. This issue refers to which bytes are most significant in multi-byte data types. In big-endian architectures, the leftmost bytes (those with a lower address) are most significant. In little-endian architectures, the rightmost bytes are most significant. The terms "big-endian" and "little-endian" are derived from the Lilliputians of Gulliver's Travels, whose major political issue was whether soft-boiled eggs should be opened on the big side or the little side. Our issue is that the HP/Compaq SC45, which uses Alpha chips, has a little-endian architecture. Therefore, all in-file data and legacy binary files in big-endian order must be converted to little-endian order before halem can process them. One example is that the integer and real arrays in the big-header record of MM5 model MMOUT_DOMAIN files are not correctly byte ordered. There are several options for rectifying this problem:

• Rewrite the export program that is providing the imported file. It might export directly in little-endian binary.
• Use a seperate translator program such as byteswap. Type 'byteswap' for usage.
• Read the data as bytes, and rearrange them on the fly. This can be done through the compiler switch -convert big_endian or -convert cray (for Cray binary). Cray binary data written with assign flag '-N ieee -F f77' can be read this way on halem. Instead of the compiler switch, the OPEN statement can be used in the program with an option CONVERT.
• The NCARU library provides subroutines that can be called from Fortran program to read Cray blocked or unblocked data directly.
• Write out the data as machine-independent such as HDF or netCDF.

Lengths of Real and Integer. Based on what type of processor the computer is built upon, the lengths of supported reals and integers will vary from computer to computer, and the kind type parameters associated with a given length will also vary. Variation is troublesome when programs have to be ported to computers with different types of processors. Fortran 90/95 programming practice recommends intrinsic functions SELECTED_REAL_KIND and SELECTED_INT_KIND to automatically select the proper kind of real and integer to use. Two tables are provided here to compare Kind/precision/range values for real and Kind/range values for integer variables for NCCS systems.

KIND/precision/range for real values in NCCS Fortran 90 compilers

KIND/range for integer values in NCCS Fortran 90 compilers

Typical porting pitfalls stem from assuming the default size of integer is long (64 bits) and the same as that of the pointer (64 bits). Refer to the Compaq porting tips page for the details on 64-bit considerations.

Standard Fortran

Non-standard Fortran codes are difficult to port. To find out where you must replace your code with standard code, use the Fortran compiler option on halem, -std, -std90, or -std95. In making your code standard, you may need to modify the input/output (I/O); that is, switch from BUFFER IN/OUT to Fortran READ/WRITE. There are no equivalents for BUFFER IN/OUT in AlphaServer SC systems.

Libraries

When porting code from one computer to another, you must identify your math library needs. To do this, you should make a full load map to see exactly which routines are coming from which libraries. To make a full loadmap, you can type your usual compile commands and library options and add the -Mfile,f option to segldr. For example:

% f90 -L/usr/local/lib -l mathpack -Wl"-Mmycode.l,f" mycode.f

The loadmap will be contained in the file mycode.l. Once you identify the basic building blocks for math software (for example, fast Fourier transforms and linear algebra routines), the next step would be to select the most efficient software for your program, such as CXML on halem.

| Top of Page |