|
Packit |
e9ba0d |
What's New In Libevent 2.0 so far:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
1. Meta-issues
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
1.1. About this document
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
This document describes the key differences between Libevent 1.4 and
|
|
Packit |
e9ba0d |
Libevent 2.0, from a user's point of view. It was most recently
|
|
Packit |
e9ba0d |
updated based on features in git master as of August 2010.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
NOTE: I am very sure that I missed some thing on this list. Caveat
|
|
Packit |
e9ba0d |
haxxor.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
1.2. Better documentation
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There is now a book-in-progress that explains how to use Libevent and its
|
|
Packit |
e9ba0d |
growing pile of APIs. As of this writing, it covers everything except the
|
|
Packit |
e9ba0d |
http and rpc code. Check out the latest draft at
|
|
Packit |
e9ba0d |
http://www.wangafu.net/~nickm/libevent-book/ .
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2. New and Improved Event APIs
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Many APIs are improved, refactored, or deprecated in Libevent 2.0.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
COMPATIBILITY:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Nearly all existing code that worked with Libevent 1.4 should still
|
|
Packit |
e9ba0d |
work correctly with Libevent 2.0. However, if you are writing new code,
|
|
Packit |
e9ba0d |
or if you want to port old code, we strongly recommend using the new APIs
|
|
Packit |
e9ba0d |
and avoiding deprecated APIs as much as possible.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Binaries linked against Libevent 1.4 will need to be recompiled to link
|
|
Packit |
e9ba0d |
against Libevent 2.0. This is nothing new; we have never been good at
|
|
Packit |
e9ba0d |
preserving binary compatibility between releases. We'll try harder in the
|
|
Packit |
e9ba0d |
future, though: see 2.1 below.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.1. New header layout for improved forward-compatibility
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent 2.0 has a new header layout to make it easier for programmers to
|
|
Packit |
e9ba0d |
write good, well-supported libevent code. The new headers are divided
|
|
Packit |
e9ba0d |
into three types.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There are *regular headers*, like event2/event.h. These headers contain
|
|
Packit |
e9ba0d |
the functions that most programmers will want to use.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There are *backward compatibility headers*, like event2/event_compat.h.
|
|
Packit |
e9ba0d |
These headers contain declarations for deprecated functions from older
|
|
Packit |
e9ba0d |
versions of Libevent. Documentation in these headers should suggest what's
|
|
Packit |
e9ba0d |
wrong with the old functions, and what functions you want to start using
|
|
Packit |
e9ba0d |
instead of the old ones. Some of these functions might be removed in a
|
|
Packit |
e9ba0d |
future release. New programs should generally not include these headers.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Finally, there are *structure headers*, like event2/event_struct.h.
|
|
Packit |
e9ba0d |
These headers contain definitions of some structures that Libevent has
|
|
Packit |
e9ba0d |
historically exposed. Exposing them caused problems in the past,
|
|
Packit |
e9ba0d |
since programs that were compiled to work with one version of Libevent
|
|
Packit |
e9ba0d |
would often stop working with another version that changed the size or
|
|
Packit |
e9ba0d |
layout of some object. We've moving them into separate headers so
|
|
Packit |
e9ba0d |
that programmers can know that their code is not depending on any
|
|
Packit |
e9ba0d |
unstable aspect of the Libvent ABI. New programs should generally not
|
|
Packit |
e9ba0d |
include these headers unless they really know what they are doing, are
|
|
Packit |
e9ba0d |
willing to rebuild their software whenever they want to link it
|
|
Packit |
e9ba0d |
against a new version of Libevent, and are willing to risk their code
|
|
Packit |
e9ba0d |
breaking if and when data structures change.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Functionality that once was located in event.h is now more subdivided.
|
|
Packit |
e9ba0d |
The core event logic is now in event2/event.h. The "evbuffer" functions
|
|
Packit |
e9ba0d |
for low-level buffer manipulation are in event2/buffer.h. The
|
|
Packit |
e9ba0d |
"bufferevent" functions for higher-level buffered IO are in
|
|
Packit |
e9ba0d |
event2/bufferevent.h.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
COMPATIBILITY:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
All of the old headers (event.h, evdns.h, evhttp.h, evrpc.h, and
|
|
Packit |
e9ba0d |
evutil.h) will continue to work by including the corresponding new
|
|
Packit |
e9ba0d |
headers. Old code should not be broken by this change.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.2. New thread-safe, binary-compatible, harder-to-mess-up APIs
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Some aspects of the historical Libevent API have encouraged
|
|
Packit |
e9ba0d |
non-threadsafe code, or forced code built against one version of Libevent
|
|
Packit |
e9ba0d |
to no longer build with another. The problems with now-deprecated APIs
|
|
Packit |
e9ba0d |
fell into two categories:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
1) Dependence on the "current" event_base. In an application with
|
|
Packit |
e9ba0d |
multiple event_bases, Libevent previously had a notion of the
|
|
Packit |
e9ba0d |
"current" event_base. New events were linked to this base, and
|
|
Packit |
e9ba0d |
the caller needed to explicitly reattach them to another base.
|
|
Packit |
e9ba0d |
This was horribly error-prone.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Functions like "event_set" that worked with the "current" event_base
|
|
Packit |
e9ba0d |
are now deprecated but still available (see 2.1). There are new
|
|
Packit |
e9ba0d |
functions like "event_assign" that take an explicit event_base
|
|
Packit |
e9ba0d |
argument when setting up a structure. Using these functions will help
|
|
Packit |
e9ba0d |
prevent errors in your applications, and to be more threadsafe.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2) Structure dependence. Applications needed to allocate 'struct
|
|
Packit |
e9ba0d |
event' themselves, since there was no function in Libevent to do it
|
|
Packit |
e9ba0d |
for them. But since the size and contents of struct event can
|
|
Packit |
e9ba0d |
change between libevent versions, this created binary-compatibility
|
|
Packit |
e9ba0d |
nightmares. All structures of this kind are now isolated in
|
|
Packit |
e9ba0d |
_struct.h header (see 2.1), and there are new allocate-and-
|
|
Packit |
e9ba0d |
initialize functions you can use instead of the old initialize-only
|
|
Packit |
e9ba0d |
functions. For example, instead of malloc and event_set, you
|
|
Packit |
e9ba0d |
can use event_new().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
(For people who do really want to allocate a struct event on the
|
|
Packit |
e9ba0d |
stack, or put one inside another structure, you can still use
|
|
Packit |
e9ba0d |
event2/event_compat.h.)
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
So in the case where old code would look like this:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
#include <event.h>
|
|
Packit |
e9ba0d |
...
|
|
Packit |
e9ba0d |
struct event *ev = malloc(sizeof(struct event));
|
|
Packit |
e9ba0d |
/* This call will cause a buffer overrun if you compile with one version
|
|
Packit |
e9ba0d |
of Libevent and link dynamically against another. */
|
|
Packit |
e9ba0d |
event_set(ev, fd, EV_READ, cb, NULL);
|
|
Packit |
e9ba0d |
/* If you forget this call, your code will break in hard-to-diagnose
|
|
Packit |
e9ba0d |
ways in the presence of multiple event bases. */
|
|
Packit |
e9ba0d |
event_set_base(ev, base);
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
New code will look more like this:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
#include <event2/event.h>
|
|
Packit |
e9ba0d |
...
|
|
Packit |
e9ba0d |
struct event *ev;
|
|
Packit |
e9ba0d |
ev = event_new(base, fd, EV_READ, cb, NULL);
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.3. Overrideable allocation functions
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you want to override the allocation functions used by libevent
|
|
Packit |
e9ba0d |
(for example, to use a specialized allocator, or debug memory
|
|
Packit |
e9ba0d |
issues, or so on), you can replace them by calling
|
|
Packit |
e9ba0d |
event_set_mem_functions. It takes replacements for malloc(),
|
|
Packit |
e9ba0d |
free(), and realloc().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you're going to use this facility, you need to call it _before_
|
|
Packit |
e9ba0d |
Libevent does any memory allocation; otherwise, Libevent may allocate some
|
|
Packit |
e9ba0d |
memory with malloc(), and free it with the free() function you provide.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can disable this feature when you are building Libevent by passing
|
|
Packit |
e9ba0d |
the --disable-malloc-replacement argument to configure.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.4. Configurable event_base creation
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Older versions of Libevent would always got the fastest backend
|
|
Packit |
e9ba0d |
available, unless you reconfigured their behavior with the environment
|
|
Packit |
e9ba0d |
variables EVENT_NOSELECT, EVENT_NOPOLL, and so forth. This was annoying
|
|
Packit |
e9ba0d |
to programmers who wanted to pick a backend explicitly without messing
|
|
Packit |
e9ba0d |
with the environment.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Also, despite our best efforts, not every backend supports every
|
|
Packit |
e9ba0d |
operation we might like. Some features (like edge-triggered events, or
|
|
Packit |
e9ba0d |
working with non-socket file descriptors) only work with some operating
|
|
Packit |
e9ba0d |
systems' fast backends. Previously, programmers who cared about this
|
|
Packit |
e9ba0d |
needed to know which backends supported what. This tended to get quite
|
|
Packit |
e9ba0d |
ungainly.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There is now an API to choose backends, either by name or by feature.
|
|
Packit |
e9ba0d |
Here is an example:
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
struct event_config_t *config;
|
|
Packit |
e9ba0d |
struct event_base *base;
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
/* Create a new configuration object. */
|
|
Packit |
e9ba0d |
config = event_config_new();
|
|
Packit |
e9ba0d |
/* We don't want to use the "select" method. */
|
|
Packit |
e9ba0d |
event_config_avoid_method(config, "select");
|
|
Packit |
e9ba0d |
/* We want a method that can work with non-socket file descriptors */
|
|
Packit |
e9ba0d |
event_config_require_features(config, EV_FEATURE_FDS);
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
base = event_base_new_with_config(config);
|
|
Packit |
e9ba0d |
if (!base) {
|
|
Packit |
e9ba0d |
/* There is no backend method that does what we want. */
|
|
Packit |
e9ba0d |
exit(1);
|
|
Packit |
e9ba0d |
}
|
|
Packit |
e9ba0d |
event_config_free(config);
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Supported features are documented in event2/event.h
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.5. Socket is now an abstract type
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
All APIs that formerly accepted int as a socket type now accept
|
|
Packit |
e9ba0d |
"evutil_socket_t". On Unix, this is just an alias for "int" as
|
|
Packit |
e9ba0d |
before. On Windows, however, it's an alias for SOCKET, which can
|
|
Packit |
e9ba0d |
be wider than int on 64-bit platforms.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.6. Timeouts and persistent events work together.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Previously, it wasn't useful to set a timeout on a persistent event:
|
|
Packit |
e9ba0d |
the timeout would trigger once, and never again. This is not what
|
|
Packit |
e9ba0d |
applications tend to want. Instead, applications tend to want every
|
|
Packit |
e9ba0d |
triggering of the event to re-set the timeout. So now, if you set
|
|
Packit |
e9ba0d |
up an event like this:
|
|
Packit |
e9ba0d |
struct event *ev;
|
|
Packit |
e9ba0d |
struct timeval tv;
|
|
Packit |
e9ba0d |
ev = event_new(base, fd, EV_READ|EV_PERSIST, cb, NULL);
|
|
Packit |
e9ba0d |
tv.tv_sec = 1;
|
|
Packit |
e9ba0d |
tv.tv_usec = 0;
|
|
Packit |
e9ba0d |
event_add(ev, &tv;;
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The callback 'cb' will be invoked whenever fd is ready to read, OR whenever
|
|
Packit |
e9ba0d |
a second has passed since the last invocation of cb.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.7. Multiple events allowed per fd
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Older versions of Libevent allowed at most one EV_READ event and at most
|
|
Packit |
e9ba0d |
one EV_WRITE event per socket, per event base. This restriction is no
|
|
Packit |
e9ba0d |
longer present.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.8. evthread_* functions for thread-safe structures.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent structures can now be built with locking support. This code
|
|
Packit |
e9ba0d |
makes it safe to add, remove, and activate events on an event base from a
|
|
Packit |
e9ba0d |
different thread. (Previously, if you wanted to write multithreaded code
|
|
Packit |
e9ba0d |
with Libevent, you could only an event_base or its events in one thread at
|
|
Packit |
e9ba0d |
a time.)
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you want threading support and you're using pthreads, you can just
|
|
Packit |
e9ba0d |
call evthread_use_pthreads(). (You'll need to link against the
|
|
Packit |
e9ba0d |
libevent_pthreads library in addition to libevent_core. These functions are
|
|
Packit |
e9ba0d |
not in libevent_core.)
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you want threading support and you're using Windows, you can just
|
|
Packit |
e9ba0d |
call evthread_use_windows_threads().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you are using some locking system besides Windows and pthreads, You
|
|
Packit |
e9ba0d |
can enable this on a per-event-base level by writing functions to
|
|
Packit |
e9ba0d |
implement mutexes, conditions, and thread IDs, and passing them to
|
|
Packit |
e9ba0d |
evthread_set_lock_callbacks and related functions in event2/thread.h.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Once locking functions are enabled, every new event_base is created with a
|
|
Packit |
e9ba0d |
lock. You can prevent a single event_base from being built with a lock
|
|
Packit |
e9ba0d |
disabled by using the EVENT_BASE_FLAG_NOLOCK flag in its
|
|
Packit |
e9ba0d |
event_config. If an event_base is created with a lock, it is safe to call
|
|
Packit |
e9ba0d |
event_del, event_add, and event_active on its events from any thread. The
|
|
Packit |
e9ba0d |
event callbacks themselves are still all executed from the thread running
|
|
Packit |
e9ba0d |
the event loop.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
To make an evbuffer or a bufferevent object threadsafe, call its
|
|
Packit |
e9ba0d |
*_enable_locking() function.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The HTTP api is not currently threadsafe.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
To build Libevent with threading support disabled, pass
|
|
Packit |
e9ba0d |
--disable-thread-support to the configure script.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.9. Edge-triggered events on some backends.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
With some backends, it's now possible to add the EV_ET flag to an event
|
|
Packit |
e9ba0d |
in order to request that the event's semantics be edge-triggered. Right
|
|
Packit |
e9ba0d |
now, epoll and kqueue support this.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The corresponding event_config feature is EV_FEATURE_ET; see 2.4 for more
|
|
Packit |
e9ba0d |
information.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.10. Better support for huge numbers of timeouts
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The heap-based priority queue timer implementation for Libevent 1.4 is good
|
|
Packit |
e9ba0d |
for randomly distributed timeouts, but suboptimal if you have huge numbers
|
|
Packit |
e9ba0d |
of timeouts that all expire in the same amount of time after their
|
|
Packit |
e9ba0d |
creation. The new event_base_init_common_timeout() logic lets you signal
|
|
Packit |
e9ba0d |
that a given timeout interval will be very common, and should use a linked
|
|
Packit |
e9ba0d |
list implementation instead of a priority queue.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.11. Improved debugging support
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
It's been pretty easy to forget to delete all your events before you
|
|
Packit |
e9ba0d |
re-initialize them, or otherwise put Libevent in an internally inconsistent
|
|
Packit |
e9ba0d |
state. You can tell libevent to catch these and other common errors with
|
|
Packit |
e9ba0d |
the new event_enable_debug_mode() call. Just invoke it before you do
|
|
Packit |
e9ba0d |
any calls to other libevent functions, and it'll catch many common
|
|
Packit |
e9ba0d |
event-level errors in your code.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
2.12. Functions to access all event fields
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
So that you don't have to access the struct event fields directly, Libevent
|
|
Packit |
e9ba0d |
now provides accessor functions to retrieve everything from an event that
|
|
Packit |
e9ba0d |
you set during event_new() or event_assign().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
3. Backend-specific and performance improvements.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
3.1. Change-minimization on O(1) backends
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
With previous versions of Libevent, if you called event_del() and
|
|
Packit |
e9ba0d |
event_add() repeatedly on a single event between trips to the backend's
|
|
Packit |
e9ba0d |
dispatch function, the backend might wind up making unnecessary calls or
|
|
Packit |
e9ba0d |
passing unnecessary data to the kernel. The new backend logic batches up
|
|
Packit |
e9ba0d |
redundant adds and deletes, and performs no more operations than necessary
|
|
Packit |
e9ba0d |
at the kernel level.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
This logic is on for the kqueue backend, and available (but off by
|
|
Packit |
e9ba0d |
default) for the epoll backend. To turn it on for the epoll backend,
|
|
Packit |
e9ba0d |
set the EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST flag in the
|
|
Packit |
e9ba0d |
event_base_cofig, or set the EVENT_EPOLL_USE_CHANGELIST environment
|
|
Packit |
e9ba0d |
variable. Doing this with epoll may result in weird bugs if you give
|
|
Packit |
e9ba0d |
any fds closed by dup() or its variants.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
3.2. Improved notification on Linux
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
When we need to wake the event loop up from another thread, we use
|
|
Packit |
e9ba0d |
an epollfd to do so, instead of a socketpair. This is supposed to be
|
|
Packit |
e9ba0d |
faster.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
3.3. Windows: better support for everything
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Bufferevents on Windows can use a new mechanism (off-by-default; see below)
|
|
Packit |
e9ba0d |
to send their data via Windows overlapped IO and get their notifications
|
|
Packit |
e9ba0d |
via the IOCP API. This should be much faster than using event-based
|
|
Packit |
e9ba0d |
notification.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Other functions throughout the code have been fixed to work more
|
|
Packit |
e9ba0d |
consistently with Windows. Libevent now builds on Windows using either
|
|
Packit |
e9ba0d |
mingw, or using MSVC (with nmake). Libevent works fine with UNICODE
|
|
Packit |
e9ba0d |
defined, or not.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Data structures are a little smarter: our lookups from socket to pending
|
|
Packit |
e9ba0d |
event are now done with O(1) hash tables rather than O(lg n) red-black
|
|
Packit |
e9ba0d |
trees.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Unfortunately, the main Windows backend is still select()-based: from
|
|
Packit |
e9ba0d |
testing the IOCP backends on the mailing list, it seems that there isn't
|
|
Packit |
e9ba0d |
actually a way to tell for certain whether a socket is writable with IOCP.
|
|
Packit |
e9ba0d |
Libevent 2.1 may add a multithreaded WaitForMultipleEvents-based
|
|
Packit |
e9ba0d |
backend for better performance with many inactive sockets and better
|
|
Packit |
e9ba0d |
integration with Windows events.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4. Improvements to evbuffers
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent has long had an "evbuffer" implementation to wrap access to an
|
|
Packit |
e9ba0d |
input or output memory buffer. In previous versions, the implementation
|
|
Packit |
e9ba0d |
was very inefficient and lacked some desirable features. We've made many
|
|
Packit |
e9ba0d |
improvements in Libevent 2.0.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.1. Chunked-memory internal representation
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Previously, each evbuffer was a huge chunk of memory. When we ran out of
|
|
Packit |
e9ba0d |
space in an evbuffer, we used realloc() to grow the chunk of memory. When
|
|
Packit |
e9ba0d |
data was misaligned, we used memmove to move the data back to the front
|
|
Packit |
e9ba0d |
of the buffer.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Needless to say, this is a terrible interface for networked IO.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Now, evbuffers are implemented as a linked list of memory chunks, like
|
|
Packit |
e9ba0d |
most Unix kernels use for network IO. (See Linux's skbuf interfaces,
|
|
Packit |
e9ba0d |
or *BSD's mbufs). Data is added at the end of the linked list and
|
|
Packit |
e9ba0d |
removed from the front, so that we don't ever need realloc huge chunks
|
|
Packit |
e9ba0d |
or memmove the whole buffer contents.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
To avoid excessive calls to read and write, we use the readv/writev
|
|
Packit |
e9ba0d |
interfaces (or WSASend/WSARecv on Windows) to do IO on multiple chunks at
|
|
Packit |
e9ba0d |
once with a single system call.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
COMPATIBILITY NOTE:
|
|
Packit |
e9ba0d |
The evbuffer struct is no longer exposed in a header. The code here is
|
|
Packit |
e9ba0d |
too volatile to expose an official evbuffer structure, and there was never
|
|
Packit |
e9ba0d |
any means provided to create an evbuffer except via evbuffer_new which
|
|
Packit |
e9ba0d |
heap-allocated the buffer.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you need access to the whole bufer as a linear chunk of memory, the
|
|
Packit |
e9ba0d |
EVBUFFER_DATA() function still works. Watch out, though: it needs to copy
|
|
Packit |
e9ba0d |
the buffer's contents in a linear chunk before you can use it.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.2. More flexible readline support
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The old evbuffer_readline() function (which accepted any sequence of
|
|
Packit |
e9ba0d |
CR and LF characters as a newline, and which couldn't handle lines
|
|
Packit |
e9ba0d |
containing NUL characters), is now deprecated. The preferred
|
|
Packit |
e9ba0d |
function is evbuffer_readln(), which supports a variety of
|
|
Packit |
e9ba0d |
line-ending styles, and which can return the number of characters in
|
|
Packit |
e9ba0d |
the line returned.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can also call evbuffer_search_eol() to find the end of a line
|
|
Packit |
e9ba0d |
in an evbuffer without ever extracting the line.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.3. Support for file-based IO in evbuffers.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can now add chunks of a file into a evbuffer, and Libevent will have
|
|
Packit |
e9ba0d |
your OS use mapped-memory functionality, sendfile, or splice to transfer
|
|
Packit |
e9ba0d |
the data without ever copying it to userspace. On OSs where this is not
|
|
Packit |
e9ba0d |
supported, Libevent just loads the data.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There are probably some bugs remaining in this code. On some platforms
|
|
Packit |
e9ba0d |
(like Windows), it just reads the relevant parts of the file into RAM.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.4. Support for zero-copy ("scatter/gather") writes in evbuffers.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can add a piece of memory to an evbuffer without copying it.
|
|
Packit |
e9ba0d |
Instead, Libevent adds a new element to the evbuffer's linked list of
|
|
Packit |
e9ba0d |
chunks with a pointer to the memory you supplied. You can do this
|
|
Packit |
e9ba0d |
either with a reference-counted chunk (via evbuffer_add_reference), or
|
|
Packit |
e9ba0d |
by asking Libevent for a pointer to its internal vectors (via
|
|
Packit |
e9ba0d |
evbuffer_reserve_space or evbuffer_peek()).
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.5. Multiple callbacks per evbuffer
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Previously, you could only have one callback active on an evbuffer at a
|
|
Packit |
e9ba0d |
time. In practice, this meant that if one part of Libevent was using an
|
|
Packit |
e9ba0d |
evbuffer callback to notice when an internal evbuffer was reading or
|
|
Packit |
e9ba0d |
writing data, you couldn't have your own callback on that evbuffer.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Now, you can now use the evbuffer_add_cb() function to add a callback that
|
|
Packit |
e9ba0d |
does not interfere with any other callbacks.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The evbuffer_setcb() function is now deprecated.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.6. New callback interface
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Previously, evbuffer callbacks were invoked with the old size of the
|
|
Packit |
e9ba0d |
buffer and the new size of the buffer. This interface could not capture
|
|
Packit |
e9ba0d |
operations that simultaneously filled _and_ drained a buffer, or handle
|
|
Packit |
e9ba0d |
cases where we needed to postpone callbacks until multiple operations were
|
|
Packit |
e9ba0d |
complete.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Callbacks that are set with evbuffer_setcb still use the old API.
|
|
Packit |
e9ba0d |
Callbacks added with evbuffer_add_cb() use a new interface that takes a
|
|
Packit |
e9ba0d |
pointer to a struct holding the total number of bytes drained read and the
|
|
Packit |
e9ba0d |
total number of bytes written. See event2/buffer.h for full details.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
4.7. Misc new evbuffer features
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can use evbuffer_remove() to move a given number of bytes from one
|
|
Packit |
e9ba0d |
buffer to another.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The evbuffer_search() function lets you search for repeated instances of
|
|
Packit |
e9ba0d |
a pattern inside an evbuffer.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can use evbuffer_freeze() to temporarily suspend drains from or adds
|
|
Packit |
e9ba0d |
to a given evbuffer. This is useful for code that exposes an evbuffer as
|
|
Packit |
e9ba0d |
part of its public API, but wants users to treat it as a pure source or
|
|
Packit |
e9ba0d |
sink.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There's an evbuffer_copyout() that looks at the data at the start of an
|
|
Packit |
e9ba0d |
evbuffer without doing a drain.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can have an evbuffer defer all of its callbacks, so that rather than
|
|
Packit |
e9ba0d |
being invoked immediately when the evbuffer's length changes, they are
|
|
Packit |
e9ba0d |
invoked from within the event_loop. This is useful when you have a
|
|
Packit |
e9ba0d |
complex set of callbacks that can change the length of other evbuffers,
|
|
Packit |
e9ba0d |
and you want to avoid having them recurse and overflow your stack.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5. Bufferevents improvements
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent has long included a "bufferevents" structure and related
|
|
Packit |
e9ba0d |
functions that were useful for generic buffered IO on a TCP connection.
|
|
Packit |
e9ba0d |
This is what Libevent uses for its HTTP implementation. In addition to
|
|
Packit |
e9ba0d |
the improvements that they get for free from the underlying evbuffer
|
|
Packit |
e9ba0d |
implementation above, there are many new features in Libevent 2.0's
|
|
Packit |
e9ba0d |
evbuffers.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.1. New OO implementations
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The "bufferevent" structure is now an abstract base type with multiple
|
|
Packit |
e9ba0d |
implementations. This should not break existing code, which always
|
|
Packit |
e9ba0d |
allocated bufferevents with bufferevent_new().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Current implementations of the bufferevent interface are described below.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.2. bufferevent_socket_new() replaces bufferevent_new()
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Since bufferevents that use a socket are not the only kind,
|
|
Packit |
e9ba0d |
bufferevent_new() is now deprecated. Use bufferevent_socket_new()
|
|
Packit |
e9ba0d |
instead.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.3. Filtered bufferevent IO
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can use bufferevent_filter_new() to create a bufferevent that wraps
|
|
Packit |
e9ba0d |
around another bufferevent and transforms data it is sending and
|
|
Packit |
e9ba0d |
receiving. See test/regress_zlib.c for a toy example that uses zlib to
|
|
Packit |
e9ba0d |
compress data before sending it over a bufferevent.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.3. Linked pairs of bufferevents
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can use bufferevent_pair_new() to produce two linked
|
|
Packit |
e9ba0d |
bufferevents. This is like using socketpair, but doesn't require
|
|
Packit |
e9ba0d |
system-calls.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.4. SSL support for bufferevents with OpenSSL
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There is now a bufferevent type that supports SSL/TLS using the
|
|
Packit |
e9ba0d |
OpenSSL library. The code for this is build in a separate
|
|
Packit |
e9ba0d |
library, libevent_openssl, so that your programs don't need to
|
|
Packit |
e9ba0d |
link against OpenSSL unless they actually want SSL support.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There are two ways to construct one of these bufferevents, both
|
|
Packit |
e9ba0d |
declared in <event2/bufferevent_ssl.h>. If you want to wrap an
|
|
Packit |
e9ba0d |
SSL layer around an existing bufferevent, you would call the
|
|
Packit |
e9ba0d |
bufferevent_openssl_filter_new() function. If you want to do SSL
|
|
Packit |
e9ba0d |
on a socket directly, call bufferevent_openssl_socket_new().
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.5. IOCP support for bufferevents on Windows
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There is now a bufferevents backend that supports IOCP on Windows.
|
|
Packit |
e9ba0d |
Supposedly, this will eventually make Windows IO much faster for
|
|
Packit |
e9ba0d |
programs using bufferevents. We'll have to see; the code is not
|
|
Packit |
e9ba0d |
currently optimized at all. To try it out, call the
|
|
Packit |
e9ba0d |
event_base_start_iocp() method on an event_base before contructing
|
|
Packit |
e9ba0d |
bufferevents.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
This is tricky code; there are probably some bugs hiding here.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.6. Improved connect support for bufferevents.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can now create a bufferevent that is not yet connected to any
|
|
Packit |
e9ba0d |
host, and tell it to connect, either by address or by hostname.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The functions to do this are bufferevent_socket_connect and
|
|
Packit |
e9ba0d |
bufferevent_socket_connect_hostname.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
5.7. Rate-limiting for bufferevents
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
If you need to limit the number of bytes read/written by a single
|
|
Packit |
e9ba0d |
bufferevent, or by a group of them, you can do this with a new set of
|
|
Packit |
e9ba0d |
bufferevent rate-limiting calls.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6. Other improvements
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.1. DNS improvements
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.1.1. DNS: IPv6 nameservers
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The evdns code now lets you have nameservers whose addresses are IPv6.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.1.2. DNS: Better security
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent 2.0 tries harder to resist DNS answer-sniping attacks than
|
|
Packit |
e9ba0d |
earlier versions of evdns. See comments in the code for full details.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Notably, evdns now supports the "0x20 hack" to make it harder to
|
|
Packit |
e9ba0d |
impersonate a DNS server. Additionally, Libevent now uses a strong
|
|
Packit |
e9ba0d |
internal RNG to generate DNS transaction IDs, so you don't need to supply
|
|
Packit |
e9ba0d |
your own.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.1.3. DNS: Getaddrinfo support
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There's now an asynchronous getaddrinfo clone, evdns_getaddrinfo(),
|
|
Packit |
e9ba0d |
to make the results of the evdns functions more usable. It doesn't
|
|
Packit |
e9ba0d |
support every feature of a typical platform getaddrinfo() yet, but it
|
|
Packit |
e9ba0d |
is quite close.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There is also a blocking evutil_getaddrinfo() declared in
|
|
Packit |
e9ba0d |
event2/util.h, to provide a getaddrinfo() implementation for
|
|
Packit |
e9ba0d |
platforms that don't have one, and smooth over the differences in
|
|
Packit |
e9ba0d |
various platforms implementations of RFC3493.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Bufferevents provide bufferevent_connect_hostname(), which combines
|
|
Packit |
e9ba0d |
the name lookup and connect operations.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.1.4. DNS: No more evdns globals
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Like an event base, evdns operations are now supposed to use an evdns_base
|
|
Packit |
e9ba0d |
argument. This makes them easier to wrap for other (more OO) languages,
|
|
Packit |
e9ba0d |
and easier to control the lifetime of. The old evdns functions will
|
|
Packit |
e9ba0d |
still, of course, continue working.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.2. Listener support
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
You can now more easily automate setting up a bound socket to listen for
|
|
Packit |
e9ba0d |
TCP connections. Just use the evconnlistener_*() functions in the
|
|
Packit |
e9ba0d |
event2/listener.h header.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The listener code supports IOCP on Windows if available.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.3. Secure RNG support
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Network code very frequently needs a secure, hard-to-predict random number
|
|
Packit |
e9ba0d |
generator. Some operating systems provide a good C implementation of one;
|
|
Packit |
e9ba0d |
others do not. Libevent 2.0 now provides a consistent implementation
|
|
Packit |
e9ba0d |
based on the arc4random code originally from OpenBSD. Libevent (and you)
|
|
Packit |
e9ba0d |
can use the evutil_secure_rng_*() functions to access a fairly secure
|
|
Packit |
e9ba0d |
random stream of bytes.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
6.4. HTTP
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The evhttp uriencoding and uridecoding APIs have updated versions
|
|
Packit |
e9ba0d |
that behave more correctly, and can handle strings with internal NULs.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
The evhttp query parsing and URI parsing logic can now detect errors
|
|
Packit |
e9ba0d |
more usefully. Moreover, we include an actual URI parsing function
|
|
Packit |
e9ba0d |
(evhttp_uri_parse()) to correctly parse URIs, so as to discourage
|
|
Packit |
e9ba0d |
people from rolling their own ad-hoc parsing functions.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
There are now accessor functions for the useful fields of struct http
|
|
Packit |
e9ba0d |
and friends; it shouldn't be necessary to access them directly any
|
|
Packit |
e9ba0d |
more.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Libevent now lets you declare support for all specified HTTP methods,
|
|
Packit |
e9ba0d |
including OPTIONS, PATCH, and so on. The default list is unchanged.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Numerous evhttp bugs also got fixed.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
7. Infrastructure improvements
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
7.1. Better unit test framework
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
We now use a unit test framework that Nick wrote called "tinytest".
|
|
Packit |
e9ba0d |
The main benefit from Libevent's point of view is that tests which
|
|
Packit |
e9ba0d |
might mess with global state can all run each in their own
|
|
Packit |
e9ba0d |
subprocess. This way, when there's a bug that makes one unit test
|
|
Packit |
e9ba0d |
crash or mess up global state, it doesn't affect any others.
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
7.2. Better unit tests
|
|
Packit |
e9ba0d |
|
|
Packit |
e9ba0d |
Despite all the code we've added, our unit tests are much better than
|
|
Packit |
e9ba0d |
before. Right now, iterating over the different backends on various
|
|
Packit |
e9ba0d |
platforms, I'm getting between 78% and 81% test coverage, compared
|
|
Packit |
e9ba0d |
with less than 45% test coverage in Libevent 1.4.
|
|
Packit |
e9ba0d |
|