I think about network partitions as the moment both sides may still look alive. I recommend writing explicit rules for reads, writes, leadership, and reconciliation before the network proves it can split.
The danger is split truth
I am looking at the messy part here: late messages, duplicates, missing messages, and the decision the system makes.
If isolated nodes make independent decisions about the same data, the system may later discover conflicts. A good design names which side can accept writes, which side can serve reads, and how recovery works.
Regional outage example
I include the practical workflow because the weak spots show up faster when the system has a real user, deadline, and failure mode.
A regional inventory service loses the link to the primary region. It can still show the last known inventory, but it should not promise a scarce item until it can confirm ownership.
Partition response code
The goal of the snippet is to show the behavior a team would need to own.
A partition rule lets safe reads continue while blocking scarce writes:
Response handleInventory(Command command, boolean partitioned) {
if (partitioned && command.reservesLastItem()) {
return Response.pending("Reservation waits for primary region");
}
return inventory.apply(command);
}The code allows ordinary inventory work but holds the dangerous last-item reservation during a partition. That protects correctness where overselling would hurt.
Sequence diagram: serve reads and block risky writes
The system remains useful for safe reads while protecting scarce write decisions.
The survival choice
- Decide which operations are safe when isolated.
- Expose degraded behavior instead of hiding uncertainty.
- Test partition behavior before an incident.