DTPools Release 0.0
DTPools is a datatype library used to test MPI communication routines with
different datatype combinations. DTPools' interface is used to create pools
of datatypes, each having a specified signature (i.e., native type + count).
Every pool supports different datatype layouts (defined internally by the
library). For a list of the available layouts, go to section: "4. Supported
Derived Datatype layouts".
This README is organized as follows:
1. DTPools API
2. Testing with DTPools
3. Supported Derived Datatypes
4. Supported Derived Datatype layouts
5. Extending DTPools
6. TODOs
----------------------------------------------------------------------------
1. DTPools API
==============
Follows a list of DTPools interfaces used for datatype testing:
* int DTP_pool_create(MPI_Datatype basic_type, int basic_count, DTP_t *dtp)
Create a new basic pool with defined datatype signature.
- basic_type: native datatype part of signature
- basic_count: native datatype count part of signature
- dtp: datatype pool object
* int DTP_pool_create_struct(int basic_type_count, MPI_Datatype *basic_types, int *basic_counts, DTP_t *dtp)
Create a new struct pool with defined signature.
- basic_type_count: number of native datatypes in struct
- basic_type: array of native datatypes
- basic_counts: array of native datatype counts
- dtp: datatype pool object
* int DTP_pool_free(DTP_t dtp)
Free a previously created datatype pool.
- dtp: datatype pool object
* int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)
Create a datatype object (at index obj_idx) inside the specified pool. Also initialize
the buffer elements using start, stride and count.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be created
- val_start: start of initialization value for buffer at index obj_idx
- val_stride: increment for next element in buffer
- val_count: total number of elements to be initialized in buffer
* int DTP_obj_free(DTP_t dtp, int obj_idx)
Free a previously created datatype object inside the specified pool.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be freed
* int DTP_obj_buf_check(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)
Checks whether the received buffer (used in communication routine) matches the sent buffer.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be checked
- val_start: start of checking value for buffer at index obj_idx
- val_stride: increment for next checked element in buffer
- val_count: total number of elements to be checked in buffer
----------------------------------------------------------------------------
2. Testing with DTPools
=======================
Follows a simple test application that uses DTPools:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "mpi.h"
#include "dtpools.h"
#define BASIC_TYPE_MAX_COUNT (1024)
int main(int argc, char *argv[]) {
int i, j, err;
int num_err = 0;
int basic_type_count;
int myrank = 0;
MPI_Request req;
DTP_t send_dtp, recv_dtp;
MPI_Init(NULL, NULL);
basic_type_count = BASIC_TYPE_MAX_COUNT;
err = DTP_pool_create(MPI_INT, basic_type_count, &send_dtp);
if (err != DTP_SUCCESS) {
/* error hanling */;
}
err = DTP_pool_create(MPI_INT, basic_type_count * 2, &recv_dtp);
if (err != DTP_SUCCESS) {
/* error hanling */;
}
for (i = 0; i < send_dtp->DTP_num_objs; i++) {
err = DTP_obj_create(send_dtp, i, 0, 2, basic_type_count);
if (err != DTP_SUCCESS) {
/* error handling */;
}
for (j = 0; j < recv_dtp->DTP_num_objs; j++) {
err = DTP_obj_create(recv_dtp, j, 0, 0, 0);
if (err != DTP_SUCCESS) {
/* error handling */;
}
MPI_Irecv(recv_dtp->DTP_obj_array[j].DTP_obj_buf,
recv_dtp->DTP_obj_array[j].DTP_obj_count,
recv_dtp->DTP_obj_array[j].DTP_obj_type,
myrank, 0, MPI_COMM_WORLD, &req);
MPI_Send(send_dtp->DTP_obj_array[i].DTP_obj_buf,
send_dtp->DTP_obj_array[i].DTP_obj_count,
send_dtp->DTP_obj_array[i].DTP_obj_type,
myrank, 0, MPI_COMM_WORLD);
MPI_Wait(&req, MPI_STATUS_IGNORE);
if (DTP_obj_buf_check(recv_dtp, j, 0, 2, basic_type_count) != DTP_SUCCESS) {
num_err++;
}
DTP_obj_free(recv_dtp, j);
}
DTP_obj_free(send_dtp, i);
}
DTP_pool_free(send_dtp);
DTP_pool_free(recv_dtp);
if (num_err > 0) {
fprintf(stdout, " No Errors\n");
fflush(stdout);
}
MPI_Finalize();
return err;
}
----------------------------------------------------------------------------
3. Supported Derived Datatypes
==============================
Currently the following derived datatype are supported:
* MPI_Type_contiguous
* MPI_Type_vector
* MPI_Type_create_hvector
* MPI_Type_indexed
* MPI_Type_create_hindexed
* MPI_Type_create_indexed_block
* MPI_Type_create_hindexed_block
* MPI_Type_create_subarray
* MPI_Type_create_struct
The following native datatypes are also supported:
* MPI_CHAR,
* MPI_WCHAR,
* MPI_SHORT,
* MPI_INT,
* MPI_LONG,
* MPI_LONG_LONG_INT,
* MPI_UNSIGNED_CHAR,
* MPI_UNSIGNED_SHORT,
* MPI_UNSIGNED,
* MPI_UNSIGNED_LONG,
* MPI_UNSIGNED_LONG_LONG,
* MPI_FLOAT,
* MPI_DOUBLE,
* MPI_LONG_DOUBLE,
* MPI_INT8_T,
* MPI_INT16_T,
* MPI_INT32_T,
* MPI_INT64_T,
* MPI_UINT8_T,
* MPI_UINT16_T,
* MPI_UINT32_T,
* MPI_UINT64_T,
* MPI_C_COMPLEX,
* MPI_C_FLOAT_COMPLEX,
* MPI_C_DOUBLE_COMPLEX,
* MPI_C_LONG_DOUBLE_COMPLEX,
* MPI_FLOAT_INT,
* MPI_DOUBLE_INT,
* MPI_LONG_INT,
* MPI_2INT,
* MPI_SHORT_INT,
* MPI_LONG_DOUBLE_INT
----------------------------------------------------------------------------
4. Supported Derived Datatype layouts
=====================================
The following layouts for derived datatypes are currently supported:
* Simple layout
- DTPI_OBJ_LAYOUT_SIMPLE__CONTIG: type_count = 1; type_stride = -; type_blklen = basic_type_count;
- DTPI_OBJ_LAYOUT_SIMPLE__VECTOR: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__HVECTOR: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__INDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__HINDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_INDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_HINDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
* Complex layout
- DTPI_OBJ_LAYOUT_LARGE_BLK__VECTOR: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__HVECTOR: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_C: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_F: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__VECTOR: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__HVECTOR: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__INDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__HINDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_INDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_HINDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_C: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_F: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__VECTOR: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HVECTOR: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_C: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_F: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__VECTOR: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HVECTOR: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__INDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HINDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_INDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_HINDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_C: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_F: type_count = large; type_blklen = small; type_stride = type_count * 2;
----------------------------------------------------------------------------
5. Extending DTPools
====================
Extending DTPools with new datatype layouts is as simple as adding the
type descriptor in `test/mpi/dtpools/include/dtpools_internal.h`, the
corresponding type create and buf check functions in
`test/mpi/dtpools/src/dtpools_internal.c`, and including the new layout to
the pool create function in `test/mpi/dtpools/src/dtpools.c`.
Additionally the type create function should also be added to creators
function vector `DTPI_Init_creators`.
Example:
/* dtpools_internal.h */
enum {
...,
DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR,
...
};
int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp);
int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp);
/* dtpools_internal.c */
void DTPI_Init_creators(DTPI_Creator * creators) {
...
creators[DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR] = DTPI_Nested_vector_create;
}
int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp) {
...
}
int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp) {
...
}
/* dtpools.c */
int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int var_stride, int val_count) {
...
switch(obj_idx) {
case XXX:
...
break;
case DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR:
/* set up parameters for create function */
par.core.type_count = X(count); /* signature count */
par.core.type_blklen = Y(count);
par.core.type_stride = Z(count);
break;
default:;
}
...
}
----------------------------------------------------------------------------
6. TODOs
========
Follows a list of known issues that should be fixed in a future release:
1. Resized datatypes (using MPI_TYPE_CREATE_RESIZED) are not currently supported.
2. The framework does not provide an interface to reset the type buffer:
`DTP_obj_reset(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)`
3. The interface should return an object handle that can be used to directly
reference the created datatype, its count, and buffer instead of accessing
directly the object array:
`DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count, DTP_Obj_t *obj)`
4. Currently datatypes in the pools have count of 1. The framework should be extended
to support counts > 1.