FAQs Debugging

What is the limitation for the MPI-2 one-sided program deadlock?

There is a limitation in the MPI library that only affects programs using MPI-2 one-sided communication in vn mode. If MPI collectives are intermingled with the one-sided communication, an optimization in the collective may cause deadlock on the other communication. In order to avoid this, you must disable the optimized tree collectives via the environment setting DCMF_COLLECTIVES=NOTREE. Alternatively, MPI_Barrier() calls may be inserted to ensure that all one-sided synchronization is complete prior to a collective.

What is the program environment length limit?

There is a system limitation (of mpirun itself) where the total length of environment variables may not be longer than 2048 characters.

What is the program argument length limit?

There is a system limitation (of mpirun itself) that the total length of arguments to your program may not be longer than 1024 characters. If you exceed this you may see a message like this in your job's stderr file:

 <Oct 16 13:46:59.549770> FE_MPI (ERROR): Arguments string is longer than 1025

How do I flush the file output from Fortran programs?

To flush the output to a file from an XL Fortran program, use "CALL FLUSH(UNIT_NUMBER)" and compile with "-qextname=flush". Follow this link for example code.

My Job completed with no .stdout or .stderr produced, only .cobaltlog, what do I do?

If your executable is in /scratch, that file system is local to a particular login node. Since it is not mounted by the BG/P I/O nodes, an executable there cannot be run on the back-end. Copy your executable to a gpfs file system such as your home directory.

My I/O output runner thread terminated, what do I do?

A message in the stderr file like this:

<Jun 24 10:40:56.697742> BE_MPI (Info) : I/O output runner thread terminated

is a normal part of job startup and is one of the last messages in the stderr before the job actually starts running. It is not an error.