Blob Blame History Raw
HPL has a large number of tunable parameters.
These configuration files should be considered a starting point
and guideline for the problem size and parameters to start with.
From there additional tuning of HPL and MPI parameters can be done
to get the best performance for a given cluster.

For each cluster size there are multiple configs:
	t = a very small test config
	s = on the low side of a 1G/process cluster
	m = in the middle of a 1G/process cluster
	l = on the high side of a 1G/process cluster
Main difference is problem size (N)

The config_hpl2 script can be used to transfer the HPL.dat file to all
nodes specified in mpi_hosts

A very critical HPL parameter is the problem size (N).
These files assume 1G of memory per processor in the cluster
(hence a 2G dual proc box would be typical).  For alternate
memory configurations the problem size should be approximated by
	SQRT(total Mem/8)*0.80
	Where total mem is the number of bytes of memory in the fabric.
	8 is the size of a double floating point number
	and 0.80 allows 80% for application data and 20% for OS, fabric, code, etc

Another important value is NB.  For the goto library supplied an NB value of
104 or 112 is recommended.

While running hpl monitor a few systems with top.  There should be no swapping
if you observe swapping reduce the problem size or reduce the memory
for MPI.

A few critical parameters in MPI (/etc/sysconfig/mpi.param) to tune:
VIADEV_SQ_SIZE - values in the 100-200 range are best.  If too low additional
	code overheads occur, if too high excessive memory is used.
VIADEV_MEM_REG_MAX - controls how much memory can be dynamically locked for
	RPUT/RDMA Write transfers, a value of 5-10 works best
VIADEV_NUM_RDMA_BUFFER - how many buffers for RDMA fast path, this consumes
	extra memory and can hurt scalability, for MPI runs over 16 processes,
	set this to 0
VIADEV_INITIAL_PREPOST_DEPTH - how many receive buffers to prepost on each QP,
	since HPL uses very few of the possible paths a value of 15 is best.

For additional information see hpl/TUNING and hpl/www/faq.html