<div align="center">
<a href="http://www.openucx.org/"><img src="./docs/doxygen/UCX_Logo_930x933.png" width="200"></a>
<br>
<a href="https://twitter.com/intent/follow?screen_name=openucx"> <img src="https://img.shields.io/twitter/follow/openucx?style=social&logo=twitter" alt="follow on Twitter"></a>
<a href="https://openucx.github.io/ucx/api/latest/html/"><img src="docs/doxygen/api.svg"></a>
<a href='https://openucx.readthedocs.io/en/master/?badge=master'><img src='https://readthedocs.org/projects/openucx/badge/?version=master' alt='Documentation Status' />
<a href="https://github.com/openucx/ucx/releases/latest"><img src="docs/doxygen/release.svg"></a>
</div>
<!-- TOC generated by https://github.com/ekalinin/github-markdown-toc -->
<hr>
* [Unified Communication X](#unified-communication-x)
* [Using UCX](#using-ucx)
* [Building and Running Internal Unit Tests](#building-and-running-internal-unit-tests)
* [UCX Performance Test](#ucx-performance-test)
* [Our Community](#our-community)
* [Licenses](#licenses)
* [Contributor Agreement and Guidelines](#contributor-agreement-and-guidelines)
* [UCX Publications](#ucx-publications)
* [UCX Architecture](#ucx-architecture)
* [Supported Transports](#supported-transports)
* [Supported CPU Architectures](#supported-cpu-architectures)
<hr>
# Unified Communication X
Unified Communication X (UCX) provides an optimized communication
layer for Message Passing ([MPI](https://www.mpi-forum.org/)),
[PGAS](http://www.pgas.org/)/[OpenSHMEM](http://www.openshmem.org/)
libraries and RPC/data-centric applications.
UCX utilizes high-speed networks for inter-node communication, and
shared memory mechanisms for efficient intra-node communication.
## Using UCX
### Release Builds
Building UCX is typically a combination of running "configure" and "make".
Execute the following commands to install the UCX system from within the
directory at the top of the tree:
```sh
$ ./autogen.sh
$ ./contrib/configure-release --prefix=/where/to/install
$ make -j8
$ make install
```
NOTE: Compiling support for various networks or other specific hardware may
require additional command line flags when running configure.
### Developer Builds
```bash
$ ./autogen.sh
$ ./contrib/configure-devel --prefix=$PWD/install-debug
```
*** NOTE: Developer builds of UCX typically include a large performance
penalty at run-time because of extra debugging code.
### Running internal unit tests
```sh
$ make -C test/gtest test
```
### Build RPM package
```bash
$ contrib/buildrpm.sh -s -b
```
### Build DEB package
```bash
$ dpkg-buildpackage -us -uc
```
### Build Doxygen documentation
```bash
$ make docs
```
### OpenMPI and OpenSHMEM installation with UCX
[Wiki page](http://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX)
### MPICH installation with UCX
[Wiki page](http://github.com/openucx/ucx/wiki/MPICH-installation-with-UCX)
### UCX Performance Test
Start server:
```sh
$ ./src/tools/perf/ucx_perftest -c 0
```
Connect client:
```sh
$ ./src/tools/perf/ucx_perftest <server-hostname> -t tag_lat -c 1
```
Note: the `-c` flag sets CPU affinity. If running both commands on same host, make sure you set the affinity to different CPU cores.
## Our Community
* [Project Website](http://www.openucx.org/)
* [ReadTheDocs](https://openucx.readthedocs.io/en/master/)
* [Github](http://www.github.com/openucx/ucx/)
* [Software Releases](http://www.github.com/openucx/ucx/releases)
* [Mailing List](https://elist.ornl.gov/mailman/listinfo/ucx-group)
* [Twitter](https://twitter.com/openucx)
## Licenses
UCX is licensed as:
* [BSD3](LICENSE)
## Contributor Agreement and Guidelines
In order to contribute to UCX, please sign up with an appropriate
[Contributor Agreement](http://www.openucx.org/license/).
Follow these
[instructions](https://github.com/openucx/ucx/wiki/Guidance-for-contributors)
when submitting contributions and changes.
## UCX Publications
To reference UCX in a publication, please use the following entry:
```bibtex
@inproceedings{shamis2015ucx,
title={UCX: an open source framework for HPC network APIs and beyond},
author={Shamis, Pavel and Venkata, Manjunath Gorentla and Lopez, M Graham and Baker, Matthew B and Hernandez, Oscar and Itigin, Yossi and Dubman, Mike and Shainer, Gilad and Graham, Richard L and Liss, Liran and others},
booktitle={2015 IEEE 23rd Annual Symposium on High-Performance Interconnects},
pages={40--43},
year={2015},
organization={IEEE}
}
```
To reference the UCX website:
```bibtex
@misc{openucx-website,
title = {{The Unified Communication X Library}},
key = {{{The Unified Communication X Library}},
howpublished = {{\url{http://www.openucx.org}}}
}
```
## UCX Architecture
| Component | Role | Description |
| :---: | :---: | --- |
| UCP | Protocol | Implements high-level abstractions such as tag-matching, streams, connection negotiation and establishment, multi-rail, and handling different memory types |
| UCT | Transport | Implements low-level communication primitives such as active messages, remote memory access, and atomic operations |
| UCS | Services | A collection of data structures, algorithms, and system utilities for common use |
| UCM | Memory | Intercepts memory allocation and release events, used by the memory registration cache |
## Supported Transports
* [Infiniband](https://www.infinibandta.org/)
* [Omni-Path](https://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-driving-exascale-computing.html)
* [RoCE](http://www.roceinitiative.org/)
* [Cray Gemini and Aries](https://www.cray.com/)
* [CUDA](https://developer.nvidia.com/cuda-zone)
* [ROCm](https://rocm.github.io/)
* Shared Memory
* posix, sysv, [cma](https://dl.acm.org/citation.cfm?id=2616532), [knem](http://knem.gforge.inria.fr/), and [xpmem](https://github.com/hjelmn/xpmem)
* TCP/IP
## Supported CPU Architectures
* [x86_64](https://en.wikipedia.org/wiki/X86-64)
* [Power8/9](https://www.ibm.com/support/knowledgecenter/en/POWER9/p9hdx/POWER9welcome.htm)
* [Arm v8](https://www.arm.com/products/silicon-ip-cpu)