Sending data with rings

Fast, zero-copy data between programs.

Preview interface

The app developer kit (SDK) is not public yet. The snippets here show the shape of the interface described in the v0.1 specification. Names, numbers and signatures can change before release. For early access, write to [email protected].

A ring is a pair of queues in shared memory. You add requests to the submission queue. The other side adds results to the completion queue. You never copy the payload: each entry names a page of memory that both sides can see.

The entries

ring entries (from libipc)

#[repr(C)]
pub struct SqEntry {
    pub opcode: u16,     // READ, WRITE, FLUSH, IRQ_ACK, ...
    pub flags: u16,
    pub frame_cptr: u32, // capability slot of the data page (0 if inline)
    pub offset: u32,     // byte offset inside that page
    pub len: u32,        // number of bytes
    pub user_data: u64,  // yours: echoed back on the matching result
}

#[repr(C)]
pub struct CqEntry {
    pub result: i32,     // negative means an error, like -errno
    pub flags: u16,
    _pad: u16,
    pub user_data: u64,
}

Submitting a request

submit and notify

use libipc::ring::{RingHeader, SqEntry};

const OP_WRITE: u16 = 1; // placeholder: real values come with the SDK

fn send_write(ring: &RingHeader, entries: &mut [SqEntry], ring_cptr: u64, frame_cptr: u32, len: u32) {
    // Was the ring empty before we push? Only then must the receiver be woken.
    let was_empty = ring.is_empty();

    let entry = SqEntry {
        opcode: OP_WRITE,
        flags: 0,
        frame_cptr,
        offset: 0,
        len,
        user_data: 42,
    };

    if ring.try_push(entries, entry).is_err() {
        // Ring full: wait for the receiver to catch up.
        return;
    }

    if was_empty {
        // Wake the receiver. This call never blocks.
        unsafe { syscall6(SYS_RING_NOTIFY, ring_cptr, 0, 0, 0, 0, 0) };
    }
}

Note

is_empty is shown as a convenience for readability. It is equivalent to comparing the head and tail counters.

Good to know

The ring size is always a power of two.
Several threads in one program can push at the same time. The push uses an atomic compare-and-swap, so no lock is needed.
A busy driver can poll the queue and skip SYS_RING_NOTIFY completely.
Large data goes in a shared page. Never copy it into the ring.

See the ring in action on How programs talk.