April 18, 2024 4 min read A to Z Series: G Home

Gossip Protocols: Small Messages, Shared State

Gossip spreads state by having nodes periodically share what they know with a few peers.

I like gossip protocols when the system can tolerate knowledge spreading gradually. I recommend them for membership and soft state problems where simple repeated messages beat one fragile central source.

It is good for awareness, not instant agreement

When I talk about "It is good for awareness, not instant agreement", I am really asking what Gossip Protocols does when a message is late, duplicated, or missing.

A gossip protocol can spread membership, load, feature flags, or cache hints. It is not the same as consensus. Gossip helps nodes become aware. Consensus helps nodes agree on one committed decision.

Cluster membership example

I use "Cluster membership example" to keep Gossip Protocols grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.

Node A learns that Node D is unhealthy. Instead of calling every node directly, it tells Node B and Node C. They pass that fact along during their next gossip round.

Gossip merge code

When I show "Gossip merge code", I want the code in Gossip Protocols to reveal the production decision, not just the syntax.

Each node merges what a peer knows and passes the merged view along:

void gossip(Node self, Node peer) {
  ClusterState merged = self.state().merge(peer.state());
  self.update(merged);
  peer.update(merged);
}

Each peer merges the other peer's state and updates both sides. Repeating this cheap exchange spreads membership or health information through the cluster over time.

Sequence diagram: a fact spreads through the cluster

a fact spreads through the cluster The fact spreads gradually. The design accepts short disagreement while the cluster catches up. sd a fact spreads through the cluster Node A Node A Node B Node B Node C Node C Node E Node E Node F Node F Share that Node D missedheartbeats. Share the same observation. Pass the observation in the next round. Pass the observation until the clusterconverges.

The fact spreads gradually. The design accepts short disagreement while the cluster catches up.

The propagation choice

Continue with membership

Written by Arunkumar Ganesan.

What I learnt is that gossip is not about instant truth; it is about steady convergence under messy network conditions.

#DistributedSystems #GossipProtocol #ClusterManagement #Architecture