# Plugin Developer's Guide #
```eval_rst
.. toctree::
```
## Overview ##
Beginning with OPAE C library version 1.2.0, OPAE implements a plugin-centric
model. This guide serves as a reference to define the makeup of an OPAE C API
plugin and to describe a sequence of steps that one may follow when constructing
an OPAE C API plugin.
## Plugin Required Functions ##
An OPAE C API plugin is a runtime-loadable shared object library, also known as
a module. On Linux systems, the *dl* family of APIs from libdl are used to
interact with shared objects. Refer to "man dlopen" and "man dlsym" for examples
of using the libdl API.
An OPAE C API plugin implements one required function. This function is required
to have C linkage, so that its name is not mangled.
```c
int opae_plugin_configure(opae_api_adapter_table *table, const char *config);
```
During initialization, the OPAE plugin manager component loads each plugin,
searching for its `opae_plugin_configure` function. If none is found, then
the plugin manager rejects that plugin. When it is found, `opae_plugin_configure`
is called passing a pointer to a freshly-created `opae_api_adapter_table` and
a buffer consisting of configuration data for the plugin.
The job of the `opae_plugin_configure` function is to populate the given adapter
table with each of the plugin's API entry points and to consume and comprehend
the given configuration data in preparation for initialization.
## OPAE API Adapter Table ##
The adapter table is a data structure that contains function pointer entry points
for each of the OPAE APIs implemented by a plugin. In this way, it adapts the
plugin-specific behavior to the more general case of a flat C API. Note that
OPAE applications are only required to link with opae-c. In other words, the
name of the plugin library should not appear on the linker command line. In this
way, plugins are truly decoupled from the OPAE C API, and they are required to
adapt to the strict API specification by populating the adapter table only. No
other linkage is required nor recommended.
`adapter.h` contains the definition of the `opae_api_adapter_table`. An abbreviated
version is depicted below, along with supporting type `opae_plugin`:
```c
typedef struct _opae_plugin {
char *path;
void *dl_handle;
} opae_plugin;
typedef struct _opae_api_adapter_table {
struct _opae_api_adapater_table *next;
opae_plugin plugin;
fpga_result (*fpgaOpen)(fpga_token token, fpga_handle *handle,
int flags);
fpga_result (*fpgaClose)(fpga_handle handle);
...
fpga_result (*fpgaEnumerate)(const fpga_properties *filters,
uint32_t num_filters, fpga_token *tokens,
uint32_t max_tokens,
uint32_t *num_matches);
...
// configuration functions
int (*initialize)(void);
int (*finalize)(void);
// first-level query
bool (*supports_device)(const char *device_type);
bool (*supports_host)(const char *hostname);
} opae_api_adapter_table;
```
Some points worth noting are that the adapter tables are organized in memory by
adding them to a linked list data structure. This is the use of the `next`
structure member. (The list management is handled by the plugin manager.)
The `plugin` structure member contains the handle to the shared object instance,
as created by `dlopen`. This handle is used in the plugin's `opae_plugin_configure`
to load plugin entry points. A plugin need only implement the portion of the
OPAE C API that a target application needs. Any API entry points that are not
supported should be left as NULL pointers (the default) in the adapter table.
When an OPAE API that has no associated entry point in the adapter table is
called, the result for objects associated with that plugin will be
`FPGA_NOT_SUPPORTED`.
The following code illustrates a portion of the `opae_plugin_configure` for
a theoretical OPAE C API plugin libfoo.so:
```c
/* foo_plugin.c */
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
adapter->fpgaOpen = dlsym(adapter->plugin.dl_handle, "foo_fpgaOpen");
adapter->fpgaClose =
dlsym(adapter->plugin.dl_handle, "foo_fpgaClose");
...
adapter->fpgaEnumerate =
dlsym(adapter->plugin.dl_handle, "foo_fpgaEnumerate");
...
return 0;
}
```
Notice that the implementations of the API entry points for plugin libfoo.so
are prefixed with `foo_`. This is the recommended practice to avoid name
collisions and to enhance the debugability of the application. Upon successful
configuration, `opae_plugin_configure` returns 0 to indicate success. A
non-zero return value indicates failure and causes the plugin manager to
reject the plugin from futher consideration.
## Plugin Optional Functions ##
Once the plugin manager loads and configures each plugin, it uses the adapter
table to call back into the plugin so that it can be made ready for runtime.
This is the job of the `opae_plugin_initialize` entry point, whose signature
is defined as:
```c
int opae_plugin_initialize(void);
```
The function takes no parameters, as the configuration data was already given
to the plugin by `opae_plugin_configure`. `opae_plugin_initialize` returns 0
if no errors were encountered during initialization. A non-zero return code
indicates that plugin initialization failed. A plugin makes its
`opae_plugin_initialize` available to the plugin manager by populating the
adapter table's `initialize` entry point as shown:
```c
/* foo_plugin.c */
int foo_plugin_initialize(void)
{
...
return 0; /* success */
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->initialize =
dlsym(adapter->plugin.dl_handle, "foo_plugin_initialize");
...
return 0;
}
```
If a plugin does not implement an `opae_plugin_initialize` entry point, then
the `initialize` member of the adapter table should be left uninitialized.
During plugin initialization, if a plugin has no `opae_plugin_initialize`
entry in its adapter table, the plugin initialization step will be skipped,
and the plugin will be considered to have initialized successfully.
Once plugin initialization is complete for all loaded plugins, the system
is considered to be running and fully functional.
During teardown, the plugin manager uses the adapter table to call into each
plugin's `opae_plugin_finalize` entry point, whose signature is defined as:
```c
int opae_plugin_finalize(void);
```
`opae_plugin_finalize` returns 0 if no errors were encountered during teardown.
A non-zero return code indicates that plugin teardown failed. A plugin makes
its `opae_plugin_finalize` available to the plugin manager by populating the
adapter table's `finalize` entry point as shown:
```c
/* foo_plugin.c */
int foo_plugin_finalize(void)
{
...
return 0; /* success */
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->finalize =
dlsym(adapter->plugin.dl_handle, "foo_plugin_finalize");
...
return 0;
}
```
If a plugin does not implement an `opae_plugin_finalize` entry point, then
the `finalize` member of the adapter table should be left uninitialized.
During plugin cleanup, if a plugin has no `opae_plugin_finalize` entry
point in its adapter table, the plugin finalize step will be skipped, and
the plugin will be considered to have finalized successfully.
In addition to `initialize` and `finalize`, an OPAE C API plugin has two
further optional entry points that relate to device enumeration. During
enumeration, when a plugin is being considered for a type of device, the
plugin may provide input on that decision by exporting an
`opae_plugin_supports_device` entry point in the adapter table:
```c
bool opae_plugin_supports_device(const char *device_type);
```
`opae_plugin_supports_device` returns true if the given device type is
supported and false if it is not. A false return value from
`opae_plugin_supports_device` causes device enumeration to skip the
plugin.
Populating the `opae_plugin_supports_device` is done as:
```c
/* foo_plugin.c */
bool foo_plugin_supports_device(const char *device_type)
{
if (/* device_type is supported */)
return true;
...
return false;
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->supports_device =
dlsym(adapter->plugin.dl_handle, "foo_plugin_supports_device");
...
return 0;
}
```
```eval_rst
.. note::
The `opae_plugin_supports_device` mechanism serves as a placeholder only.
It is not implemented in the current version of the OPAE C API.
```
Similarly to determining whether a plugin supports a type of device, a plugin
may also answer questions about network host support by populating an
`opae_plugin_supports_host` entry point in the adapter table:
```c
bool opae_plugin_supports_host(const char *hostname);
```
`opae_plugin_supports_host` returns true if the given hostname is supported
and false if it is not. A false return value from `opae_plugin_supports_host`
causes device enumeration to skip the plugin.
Populating the `opae_plugin_supports_host` is done as:
```c
/* foo_plugin.c */
bool foo_plugin_supports_host(const char *hostname)
{
if (/* hostname is supported */)
return true;
...
return false;
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->supports_host =
dlsym(adapter->plugin.dl_handle, "foo_plugin_supports_host");
...
return 0;
}
```
```eval_rst
.. note::
The `opae_plugin_supports_host` mechanism serves as a placeholder only.
It is not implemented in the current version of the OPAE C API.
```
## Plugin Construction ##
The steps required to implement an OPAE C API plugin, libfoo.so, are:
* Create foo\_plugin.c: implements `opae_plugin_configure`,
`opae_plugin_initialize`, `opae_plugin_finalize`, `opae_plugin_supports_device`,
and `opae_plugin_supports_host` as described in the previous sections.
* Create foo\_plugin.h: implements function prototypes for each of the
plugin-specific OPAE C APIs.
```c
/* foo_plugin.h */
fpga_result foo_fpgaOpen(fpga_token token, fpga_handle *handle,
int flags);
fpga_result foo_fpgaClose(fpga_handle handle);
...
fpga_result foo_fpgaEnumerate(const fpga_properties *filters,
uint32_t num_filters, fpga_token *tokens,
uint32_t max_tokens,
uint32_t *num_matches);
...
```
* Create foo\_types.h: implements plugin-specific types for opaque data
structures.
```c
/* foo_types.h */
struct _foo_token {
...
};
struct _foo_handle {
...
};
struct _foo_event_handle {
...
};
struct _foo_object {
...
};
```
* Create foo\_enum.c: implements `foo_fpgaEnumerate`,
`foo_fpgaCloneToken`, and `foo_fpgaDestroyToken`.
* Create foo\_open.c: implements `foo_fpgaOpen`.
* Create foo\_close.c: implements `foo_fpgaClose`.
* Create foo\_props.c: implements `foo_fpgaGetProperties`,
`foo_fpgaGetPropertiesFromHandle`, `foo_fpgaUpdateProperties`
* Create foo\_mmio.c: implements `foo_fpgaMapMMIO`, `foo_fpgaUnmapMMIO`
`foo_fpgaWriteMMIO64`, `foo_fpgaReadMMIO64`, `foo_fpgaWriteMMIO32`,
`foo_fpgaReadMMIO32`.
* Create foo\_buff.c: implements `foo_fpgaPrepareBuffer`,
`foo_fpgaReleaseBuffer`, `foo_fpgaGetIOAddress`.
* Create foo\_error.c: implements `foo_fpgaReadError`, `foo_fpgaClearError`,
`foo_fpgaClearAllErrors`, `foo_fpgaGetErrorInfo`.
* Create foo\_event.c: implements `foo_fpgaCreateEventHandle`,
`foo_fpgaDestroyEventHandle`, `foo_fpgaGetOSObjectFromEventHandle`,
`foo_fpgaRegisterEvent`, `foo_fpgaUnregisterEvent`.
* Create foo\_reconf.c: implements `foo_fpgaReconfigureSlot`.
* Create foo\_obj.c: implements `foo_fpgaTokenGetObject`,
`foo_fpgaHandleGetObject`, `foo_fpgaObjectGetObject`,
`foo_fpgaDestroyObject`, `foo_fpgaObjectGetSize`, `foo_fpgaObjectRead`,
`foo_fpgaObjectRead64`, `foo_fpgaObjectWrite64`.
* Create foo\_clk.c: implements `foo_fpgaSetUserClock`,
`foo_fpgaGetUserClock`.