As of version 0.4, Cap’n Proto’s C++ RPC implementation is a Level 1 implementation. Persistent capabilities, three-way introductions, and distributed equality are not yet implemented.
The Calculator example implements a fully-functional Cap’n Proto client and server.
KJ Concurrency Framework
RPC naturally requires a notion of concurrency. Unfortunately, all concurrency models suck.
Cap’n Proto’s RPC is based on the KJ library’s event-driven concurrency
framework. The core of the KJ asynchronous framework (events, promises, callbacks) is defined in
kj/async.h, with I/O interfaces (streams, sockets, networks) defined in
Event Loop Concurrency
KJ’s concurrency model is based on event loops. While multiple threads are allowed, each thread must have its own event loop. KJ discourages fine-grained interaction between threads as synchronization is expensive and error-prone. Instead, threads are encouraged to communicate through Cap’n Proto RPC.
As of version 0.4, the only supported way to communicate between threads is over pipes or socketpairs. This will be improved in future versions. For now, just set up an RPC connection over that socketpair. :)
Function calls that do I/O must do so asynchronously, and must return a “promise” for the result. Promises – also known as “futures” in some systems – are placeholders for the results of operations that have not yet completed. When the operation completes, we say that the promise “resolves” to a value, or is “fulfilled”. A promise can also be “rejected”, which means an exception occurred.
If you want to do something with the result of a promise, you must first wait for it to complete. This is normally done by registering a callback to execute on completion. Luckily, C++11 just introduced lambdas, which makes this far more pleasant than it would have been a few years ago!
The callback passed to
then() takes the promised result as its parameter and returns a new value.
then() itself returns a new promise for that value which the callback will eventually return.
If the callback itself returns a promise, then
then() actually returns a promise for the
resolution of the latter promise – that is,
Promise<Promise<T>> is automatically reduced to
then() consumes the original promise: you can only call
then() once. This is true
of all of the methods of
Promise. The only way to consume a promise in multiple places is to
first “fork” it with the
fork() method, which we don’t get into here. Relatedly, promises
are linear types, which means they have move constructors but not copy constructors.
then() takes an optional second parameter for handling errors. Think of this like a
Note that the KJ framework coerces all exceptions to
kj::Exception – the exception’s description
(as returned by
what()) will be retained, but any type-specific information is lost. Under KJ
exception philosophy, exceptions always represent an error that should not occur under normal
operation, and the only purpose of exceptions is to make software fault-tolerant. In particular,
the only reasonable ways to handle an exception are to try again, tell a human, and/or propagate
to the caller. To that end,
kj::Exception contains information useful for reporting purposes
and to help decide if trying again is reasonable, but typed exception hierarchies are not useful
and not supported.
It is recommended that Cap’n Proto code use the assertion macros in
kj/debug.h to throw
exceptions rather than use the C++
throw keyword. These macros make it easy to add useful
debug information to an exception and generally play nicely with the KJ framework. In fact, you
can even use these macros – and propagate exceptions through promises – if you compile your code
with exceptions disabled. See the headers for more information.
It is illegal for code running in an event callback to wait, since this would stall the event loop. However, if you are the one responsible for starting the event loop in the first place, then KJ makes it easy to say “run the event loop until this promise resolves, then return the result”.
wait() is common in high-level client-side code. On the other hand, it is almost never
used in servers.
If you discard a
Promise without calling any of its methods, the operation it was waiting for
is canceled, because the
Promise itself owns that operation. This means than any pending
callbacks simply won’t be executed. If you need explicit notification when a promise is canceled,
you can use its
attach() method to attach an object with a destructor – the destructor will be
called when the promise either completes or is canceled.
Callbacks registered with
.then() which aren’t themselves asynchronous (i.e. they return a value,
not a promise) by default won’t execute unless the result is actually used – they are executed
“lazily”. This allows the runtime to optimize by combining a series of .then() callbacks into one.
To force a
.then() callback to execute as soon as its input is available, do one of the
- Add it to a
kj::TaskSet– this is usually the best choice. You can cancel all tasks in the set by destroying the
.wait()on it – but this only works in a top-level wait scope, typically your program’s main function.
.eagerlyEvaluate()on it. This returns a new
Promise. You can cancel the task by destroying this
Promise(without otherwise consuming it).
.detach()is dangerous because there is no way to cancel a promise once it has been detached. This can make it impossible to safely tear down the execution environment, e.g. if the callback has captured references to other objects. It is therefore recommended to avoid
.detach()except in carefully-controlled circumstances.
KJ supports a number of primitive operations that can be performed on promises. The complete API
is documented directly in the
kj/async.h header. Additionally, see the
for APIs for performing basic network I/O – although Cap’n Proto RPC users typically won’t need
to use these APIs directly.
Imagine the following interface:
capnp compile will generate code that looks like this (edited for readability):
Client type represents a reference to a remote
pass-by-value types that use reference counting under the hood. (Warning: For performance
reasons, the reference counting used by
Clients is not thread-safe, so you must not copy a
Client to another thread, unless you do it by means of an inter-thread RPC.)
Client can be implicitly constructed from any of:
kj::Own<Server>, which takes ownership of the server object and creates a client that calls it. (You can get a
kj::Own<T>to a newly-allocated heap object using
kj::Promise<Client>, which creates a client whose methods first wait for the promise to resolve, then forward the call to the resulting client.
kj::Exception, which creates a client whose methods always throw that exception.
nullptr, which creates a client whose methods always throw. This is meant to be used to initialize variables that will be initialized to a real value later on.
For each interface method
Client has a method
fooRequest() which creates a new
request to call
foo(). The returned
capnp::Request object has methods equivalent to a
Builder for the parameter struct (
FooParams), with the addition of a method
send() sends the RPC and returns a
RemotePromise is equivalent to
kj::Promise<capnp::Response<FooResults>>, but also has
methods that allow pipelining. Namely:
- For each interface-typed result, it has a getter method which returns a
Clientof that type. Calling this client will send a pipelined call to the server.
- For each struct-typed result, it has a getter method which returns an object containing pipeline getters for that struct’s fields.
In other words, the
RemotePromise effectively implements a subset of the eventual results’
Reader interface – one that only allows access to interfaces and sub-structs.
RemotePromise eventually resolves to
capnp::Response<FooResults>, which behaves like a
Reader for the result struct except that it also owns the result message.
For generic methods, the
fooRequest() method will be a template;
you must explicitly specify type parameters.
Server type is an abstract interface which may be subclassed to implement a
capability. Each method takes a
context argument and returns a
resolves when the call is finished. The parameter and result structures are accessed through the
context.getParams() returns a
Reader for the parameters, and
Builder for the results. The context also has methods for controlling RPC logistics,
such as cancellation – see
capnp/capability.h for details.
Accessing the results through the context (rather than by returning them) is unintuitive, but
necessary because the underlying RPC transport needs to have control over where the results are
allocated. For example, a zero-copy shared memory transport would need to allocate the results in
the shared memory segment. Hence, the method implementation cannot just create its own
On the server side, generic methods are NOT templates. Instead,
the generated code is exactly as if all of the generic parameters were bound to
server generally does not get to know exactly what type the client requested; it must be designed
to be correct for any parameterization.
Cap’n Proto makes it easy to start up an RPC client or server using the “EZ RPC” classes,
capnp/ez-rpc.h. These classes get you up and running quickly, but they hide a lot
of details that power users will likely want to manipulate. Check out the comments in
to understand exactly what you get and what you miss. For the purpose of this overview, we’ll
show you how to use EZ RPC to get started.
Starting a client
A client should typically look like this:
Note that for the connect address, Cap’n Proto supports DNS host names as well as IPv4 and IPv6
addresses. Additionally, a Unix domain socket can be specified as
unix: followed by a path name,
and an abstract Unix domain socket can be specified as
unix-abstract: followed by an identifier.
For a more complete example, see the calculator client sample.
Starting a server
A server might look something like this:
Note that for the bind address, Cap’n Proto supports DNS host names as well as IPv4 and IPv6
addresses. The special address
* can be used to bind to the same port on all local IPv4 and
IPv6 interfaces. Additionally, a Unix domain socket can be specified as
unix: followed by a
path name, and an abstract Unix domain socket can be specified as
unix-abstract: followed by
For a more complete example, see the calculator server sample.
If you’ve written a server and you want to connect to it to issue some calls for debugging, perhaps
interactively, the easiest way to do it is to use pycapnp.
We have decided not to add RPC functionality to the
capnp command-line tool because pycapnp is
better than anything we might provide.