Skip to content
vivek pal

wiki / systems

gossipsub on one page

How libp2p's gossipsub keeps pubsub traffic bounded with a sparse mesh, lazy gossip, and peer scoring.

updated · 4 min read

Floodsub, the simplest pubsub router, forwards every message to every known topic peer. Outbound degree is unbounded, so densely connected nodes drown in bandwidth. Gossipsub forwards full messages to a small, bounded set of peers per topic and sends only metadata to everyone else.

the mesh

For each topic it subscribes to, a peer keeps a mesh: a random subset of topic peers that receive every full message. Joining a topic means picking up to D known topic peers and sending each a GRAFT. Leaving means sending PRUNE to every mesh member.

Mesh links are bidirectional. If A grafts B, B adds A to its own mesh as long as it is still subscribed. In steady state, if A is in B’s mesh, B is in A’s.

            full messages (mesh, ~D peers)
   [peer] ====================> [mesh peers]
      |
      |  IHAVE(ids) every heartbeat (lazy gossip)
      +--------------------->   [other topic peers]
                                   |
                IWANT(ids) <-------+  "send me the ones I haven't seen"

the parameters

From the v1.0 spec’s table of reasonable defaults:

parametermeaningdefault
Ddesired outbound degree6
D_lowlower bound for mesh size4
D_highupper bound for mesh size12
D_lazypeers to emit gossip to (optional)D
heartbeat_intervaltime between heartbeats1 second
fanout_ttllifetime of a topic’s fanout state60 seconds
mcache_lenhistory windows in the message cache5
mcache_gossipwindows used when emitting gossip3
seen_ttlexpiry of the seen-message cache2 minutes

The v1.1 spec writes the bounds as D_lo and D_hi. Same knobs.

the heartbeat

Every heartbeat_interval, each peer does three things:

  1. Mesh maintenance. If a topic mesh has fewer than D_low peers, graft enough non-mesh topic peers to get back to D. If it has more than D_high, prune random peers down to D.
  2. Fanout maintenance. Drop fanout state for topics we haven’t published to within fanout_ttl, and top each remaining fanout set up to D.
  3. Gossip emission. For each topic, take the message IDs from the last mcache_gossip windows of the message cache and send them in an IHAVE to a random selection of topic peers that are not in the mesh or fanout. Then shift the cache.

The message cache (mcache) is a ring of history windows, one per heartbeat. Messages older than mcache_len windows fall out.

lazy gossip: IHAVE and IWANT

IHAVE carries IDs, not payloads. A peer receiving it checks its seen cache and replies with IWANT for any IDs it hasn’t seen. The sender answers from its mcache. This is how a message reaches peers outside the mesh, at the cost of an extra round trip.

fanout

You can publish to a topic you are not subscribed to. The router keeps a separate fanout map of up to D peers for that topic and sends to them. If you later subscribe, those fanout peers become the start of your mesh.

message IDs and the seen cache

Deduplication needs a message ID. The pubsub spec’s default is origin stamping: the concatenation of the from and seqno fields. You can supply your own message_id_fn, for example a hash of data, which gives content-addressed messaging. Two things to know:

  • Every peer in a topic must use the same ID function, or the topic malfunctions.
  • With hash-based IDs, identical payloads (keepalives, say) sent within the seen-cache window are treated as duplicates and dropped.

The seen cache is a timed cache of recent IDs, kept for seen_ttl. It stops re-forwarding and keeps IWANT requests to messages you don’t already have.

v1.1: hardening against attackers

v1.0 assumes honest peers. v1.1 adds defenses against sybils and eclipse attacks:

  • Flood publishing (default on). Your own messages go to every connected topic peer above a publish score threshold, not just your mesh. An honest publisher can’t be eclipsed by a captured mesh.
  • PRUNE backoff and peer exchange. A pruned peer must wait before re-grafting; the recommended backoff is 1 minute, and the PRUNE carries the backoff in seconds. When pruning for oversubscription, a peer can include other peers (PX) so the pruned one can find new links.
  • Adaptive gossip. Instead of a fixed D_lazy, gossip goes to a fraction of peers (GossipFactor, recommended 0.25) with D_lazy as the floor. Over 3 rounds, a peer misses gossip about a fresh message with probability (3/4)³ ≈ 0.42.
  • Outbound mesh quotas. Keep at least D_out outbound connections in the mesh (2 for a D of 6; must be below D_lo and at most D/2), so inbound-only attackers can’t take over the mesh.
  • Peer scoring. Each peer scores others locally from per-topic terms (time in mesh, first message deliveries, mesh delivery rate, mesh delivery failures, invalid messages) and global ones (an application-specific score, IP colocation, a behavioural penalty). Peers with negative scores get pruned. Below GossipThreshold, gossip to and from them stops; below PublishThreshold, you stop flood-publishing to them; below GraylistThreshold, their RPCs are ignored.
  • Opportunistic grafting. If the median mesh score drops below OpportunisticGraftThreshold, graft a couple of better-scoring peers. The spec recommends checking every minute and grafting 2.

Scores are never shared, and they persist for a while after disconnect so a bad peer can’t reset by reconnecting.

sources

← all notes