cel's hyperboria site

Notes on meshchat

meshchat tries to do many things at once and doesn't do a great job of any of them.

Here are my notes on what I would like to do with meshchat.

Restructure into three parts:

  1. Broadcast over cjdns

    Use cjdns's admin port and routing table to get IPs. Ping them discover to peers. This is implemented.

    Decide on how to handle channel (un)reliability. Use heartbeats? The join messages in (2) don't need in-order delivery or at-most-once delivery, but they should probably have guaranteed to nodes that are online.

  2. Multicast

    Use the broadcast layer (1) to implement named multicast groups. To join a group, a node sends a broadcast message that it wants to join a group of a given name. Peers receiving it note that that peer is in that group and that ack with whether or not they are also in the group. Messages sent to the group should have guaranteed (in-order?) delivery, so they would need sequence numbers, acks, and retransmissions. Use NACK suppression as in Scalable Reliable Multicast [1] to reduce sender flooding in case of negative acknowledgments, and also find new peers.

  3. Embedded IRC server

    This is implemented but is buggy. It should be restructured to be one of potentially many services on top of the multicast layer (2).

Multiple multicast services

Applications other than IRC servers could benefit from the multicast platform (2). If (1) and (2) are packaged as a library, then each application using them would have to do its own peer discovery and duplicate the work needed to form the network. Therefore, separate out (1) and (2) into a system service and implement a client library to communicate with it over e.g. named pipes.

Example servers to build on (2) alongside the IRC server

Requirements for multicast channels

Each of these multicast services may have different requirements for the channel. For example, in an audio/video server, dropping frames may be acceptable rather than buffering, so guaranteed packet delivery might not be necessary. However, for chat or file sharing guaranted delivery is desirable. In-order delivery may be desirable for file transfer but not as important for real-time chat. Consider allowing a multicast peer to send datagrams as well as communicate over a reliable stream, according to the needs of the application.

Event loop

Consider using the event loop in openwrt's libubox instead of libuv, so as to have a smaller footprint.


A tree-based multicast protocol would be more scalable, as nodes would then be able to send a message to a multicast group by sending it to a subset of the members of the group. However, if nodes relay messages sent by other nodes, then we lose the identity guarantees of cjdns packets. This could be handled by exposing a message signing API from cjdns, or by adding another layer of keypair identities so that packets could be signed, forwarded, and verified. Because of the complexity and overhead of this, let's just do many-to-many multicast.


  1. A Reliable Multicast Framework for Lightweight Sessions and Application Level Framing - Floyd, Jacobson, et al. - 1995
  2. An API for Scalable Reliable Multicast - Jim Gemmell, Jörg Liebeherr, Dave Bassett - 1997
  3. A negative acknowledgement with periodic polling protocol for multicast over lans - Ramakrishnan, Jain - 1987

Last updated