DTPools Release 0.0

DTPools is a datatype library used to test MPI communication routines with

different datatype combinations. DTPools' interface is used to create pools

of datatypes, each having a specified signature (i.e., native type + count).

Every pool supports different datatype layouts (defined internally by the

library). For a list of the available layouts, go to section: "4. Supported

Derived Datatype layouts".

This README is organized as follows:

1. DTPools API

2. Testing with DTPools

3. Supported Derived Datatypes

4. Supported Derived Datatype layouts

5. Extending DTPools

6. TODOs

----------------------------------------------------------------------------

1. DTPools API

==============

Follows a list of DTPools interfaces used for datatype testing:

* int DTP_pool_create(MPI_Datatype basic_type, int basic_count, DTP_t *dtp)

  Create a new basic pool with defined datatype signature.

  - basic_type:   native datatype part of signature

  - basic_count:  native datatype count part of signature

  - dtp:          datatype pool object

* int DTP_pool_create_struct(int basic_type_count, MPI_Datatype *basic_types, int *basic_counts, DTP_t *dtp)

  Create a new struct pool with defined signature.

  - basic_type_count:  number of native datatypes in struct

  - basic_type:        array of native datatypes

  - basic_counts:      array of native datatype counts

  - dtp:               datatype pool object

* int DTP_pool_free(DTP_t dtp)

  Free a previously created datatype pool.

  - dtp:  datatype pool object

* int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)

  Create a datatype object (at index obj_idx) inside the specified pool. Also initialize

  the buffer elements using start, stride and count.

  - dtp:         datatype pool object

  - obj_idx:     number of datatype inside the pool to be created

  - val_start:   start of initialization value for buffer at index obj_idx

  - val_stride:  increment for next element in buffer

  - val_count:   total number of elements to be initialized in buffer

* int DTP_obj_free(DTP_t dtp, int obj_idx)

  Free a previously created datatype object inside the specified pool.

  - dtp:      datatype pool object

  - obj_idx:  number of datatype inside the pool to be freed

* int DTP_obj_buf_check(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)

  Checks whether the received buffer (used in communication routine) matches the sent buffer.

  - dtp:         datatype pool object

  - obj_idx:     number of datatype inside the pool to be checked

  - val_start:   start of checking value for buffer at index obj_idx

  - val_stride:  increment for next checked element in buffer

  - val_count:   total number of elements to be checked in buffer

----------------------------------------------------------------------------

2. Testing with DTPools

=======================

Follows a simple test application that uses DTPools:

#include <stdio.h>

#include <stdlib.h>

#include <unistd.h>

#include "mpi.h"

#include "dtpools.h"

#define BASIC_TYPE_MAX_COUNT (1024)

int main(int argc, char *argv[]) {

    int i, j, err;

    int num_err = 0;

    int basic_type_count;

    int myrank = 0;

    MPI_Request req;

    DTP_t send_dtp, recv_dtp;

    MPI_Init(NULL, NULL);

    basic_type_count = BASIC_TYPE_MAX_COUNT;

    err = DTP_pool_create(MPI_INT, basic_type_count, &send_dtp);

    if (err != DTP_SUCCESS) {

        /* error hanling */;

    }

    err = DTP_pool_create(MPI_INT, basic_type_count * 2, &recv_dtp);

    if (err != DTP_SUCCESS) {

        /* error hanling */;

    }

    for (i = 0; i < send_dtp->DTP_num_objs; i++) {

        err = DTP_obj_create(send_dtp, i, 0, 2, basic_type_count);

        if (err != DTP_SUCCESS) {

            /* error handling */;

        }

        for (j = 0; j < recv_dtp->DTP_num_objs; j++) {

            err = DTP_obj_create(recv_dtp, j, 0, 0, 0);

            if (err != DTP_SUCCESS) {

                /* error handling */;

            }

            MPI_Irecv(recv_dtp->DTP_obj_array[j].DTP_obj_buf,

                      recv_dtp->DTP_obj_array[j].DTP_obj_count,

                      recv_dtp->DTP_obj_array[j].DTP_obj_type,

                      myrank, 0, MPI_COMM_WORLD, &req;;

            MPI_Send(send_dtp->DTP_obj_array[i].DTP_obj_buf,

                     send_dtp->DTP_obj_array[i].DTP_obj_count,

                     send_dtp->DTP_obj_array[i].DTP_obj_type,

                     myrank, 0, MPI_COMM_WORLD);

            MPI_Wait(&req, MPI_STATUS_IGNORE);

            if (DTP_obj_buf_check(recv_dtp, j, 0, 2, basic_type_count) != DTP_SUCCESS) {

                num_err++;

            }

            DTP_obj_free(recv_dtp, j);

        }

        DTP_obj_free(send_dtp, i);

    }

    DTP_pool_free(send_dtp);

    DTP_pool_free(recv_dtp);

    if (num_err > 0) {

        fprintf(stdout, " No Errors\n");

        fflush(stdout);

    }

    MPI_Finalize();

    return err;

}

----------------------------------------------------------------------------

3. Supported Derived Datatypes

==============================

Currently the following derived datatype are supported:

* MPI_Type_contiguous

* MPI_Type_vector

* MPI_Type_create_hvector

* MPI_Type_indexed

* MPI_Type_create_hindexed

* MPI_Type_create_indexed_block

* MPI_Type_create_hindexed_block

* MPI_Type_create_subarray

* MPI_Type_create_struct

The following native datatypes are also supported:

* MPI_CHAR,

* MPI_WCHAR,

* MPI_SHORT,

* MPI_INT,

* MPI_LONG,

* MPI_LONG_LONG_INT,

* MPI_UNSIGNED_CHAR,

* MPI_UNSIGNED_SHORT,

* MPI_UNSIGNED,

* MPI_UNSIGNED_LONG,

* MPI_UNSIGNED_LONG_LONG,

* MPI_FLOAT,

* MPI_DOUBLE,

* MPI_LONG_DOUBLE,

* MPI_INT8_T,

* MPI_INT16_T,

* MPI_INT32_T,

* MPI_INT64_T,

* MPI_UINT8_T,

* MPI_UINT16_T,

* MPI_UINT32_T,

* MPI_UINT64_T,

* MPI_C_COMPLEX,

* MPI_C_FLOAT_COMPLEX,

* MPI_C_DOUBLE_COMPLEX,

* MPI_C_LONG_DOUBLE_COMPLEX,

* MPI_FLOAT_INT,

* MPI_DOUBLE_INT,

* MPI_LONG_INT,

* MPI_2INT,

* MPI_SHORT_INT,

* MPI_LONG_DOUBLE_INT

----------------------------------------------------------------------------

4. Supported Derived Datatype layouts

=====================================

The following layouts for derived datatypes are currently supported:

* Simple layout

  - DTPI_OBJ_LAYOUT_SIMPLE__CONTIG:                  type_count = 1; type_stride = -; type_blklen = basic_type_count;

  - DTPI_OBJ_LAYOUT_SIMPLE__VECTOR:                  type_count = basic_type_count; type_stride = 2; type_blklen = 1;

  - DTPI_OBJ_LAYOUT_SIMPLE__HVECTOR:                 type_count = basic_type_count; type_stride = 2; type_blklen = 1;

  - DTPI_OBJ_LAYOUT_SIMPLE__INDEXED:                 type_count = basic_type_count; type_stride = 2; type_blklen = 1;

  - DTPI_OBJ_LAYOUT_SIMPLE__HINDEXED:                type_count = basic_type_count; type_stride = 2; type_blklen = 1;

  - DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_INDEXED:           type_count = basic_type_count; type_stride = 2; type_blklen = 1;

  - DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_HINDEXED:          type_count = basic_type_count; type_stride = 2; type_blklen = 1;

* Complex layout

  - DTPI_OBJ_LAYOUT_LARGE_BLK__VECTOR:               type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__HVECTOR:              type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__INDEXED:              type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__HINDEXED:             type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_INDEXED:        type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_HINDEXED:       type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_C:           type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_F:           type_count = small; type_blklen = large; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__VECTOR:               type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__HVECTOR:              type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__INDEXED:              type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__HINDEXED:             type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_INDEXED:        type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_HINDEXED:       type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_C:           type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_F:           type_count = large; type_blklen = small; type_stride = type_blklen + 1;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__VECTOR:          type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HVECTOR:         type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__INDEXED:         type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HINDEXED:        type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_INDEXED:   type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_HINDEXED:  type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_C:      type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_F:      type_count = small; type_blklen = large; type_stride = type_blklen * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__VECTOR:          type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HVECTOR:         type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__INDEXED:         type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HINDEXED:        type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_INDEXED:   type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_HINDEXED:  type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_C:      type_count = large; type_blklen = small; type_stride = type_count  * 2;

  - DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_F:      type_count = large; type_blklen = small; type_stride = type_count  * 2;

----------------------------------------------------------------------------

5. Extending DTPools

====================

Extending DTPools with new datatype layouts is as simple as adding the

type descriptor in `test/mpi/dtpools/include/dtpools_internal.h`, the

corresponding type create and buf check functions in

`test/mpi/dtpools/src/dtpools_internal.c`, and including the new layout to

the pool create function in `test/mpi/dtpools/src/dtpools.c`.

Additionally the type create function should also be added to creators

function vector `DTPI_Init_creators`.

Example:

/* dtpools_internal.h */

enum {

    ...,

    DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR,

    ...

};

int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp);

int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp);

/* dtpools_internal.c */

void DTPI_Init_creators(DTPI_Creator * creators) {

    ...

    creators[DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR] = DTPI_Nested_vector_create;

}

int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp) {

    ...

}

int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp) {

    ...

}

/* dtpools.c */

int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int var_stride, int val_count) {

    ...

    switch(obj_idx) {

        case XXX:

            ...

            break;

        case DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR:

            /* set up parameters for create function */

            par.core.type_count  = X(count); /* signature count */

            par.core.type_blklen = Y(count);

            par.core.type_stride = Z(count);

            break;

        default:;

    }

    ...

}

----------------------------------------------------------------------------

6. TODOs

========

Follows a list of known issues that should be fixed in a future release:

1. Resized datatypes (using MPI_TYPE_CREATE_RESIZED) are not currently supported.

2. The framework does not provide an interface to reset the type buffer:

   `DTP_obj_reset(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)`

3. The interface should return an object handle that can be used to directly

   reference the created datatype, its count, and buffer instead of accessing

   directly the object array:

   `DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count, DTP_Obj_t *obj)`

4. Currently datatypes in the pools have count of 1. The framework should be extended

   to support counts > 1.