October 21, 2024 4 min read A to Z Series: W Home

Write Ahead Log: Write Intent Before State

A write ahead log records the intended change before the system applies it to the main state.

I use the write ahead log idea whenever recovery matters. Before state changes are trusted, I want a durable intent record the system can replay after a crash.

The log is the recovery truth

When I talk about "The log is the recovery truth", I am really asking what Write Ahead Log does when a message is late, duplicated, or missing.

A database can append a log entry, flush it safely, then update indexes or pages later. If the process crashes halfway through, recovery reads the log and finishes or rolls back the work correctly.

Database recovery example

I use "Database recovery example" to keep Write Ahead Log grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.

A broker receives a message. It writes the message to the log before acknowledging the producer. After a crash, the broker replays the log and does not lose the acknowledged message.

Append before apply code

When I show "Append before apply code", I want the code in Write Ahead Log to reveal the production decision, not just the syntax.

Acknowledge only after the intent is durable:

void handle(Message message) {
  log.append(message);
  log.flush();
  state.apply(message);
  producer.ack(message.id());
}

The handler flushes the log before applying state and acknowledging the message. If the process dies, the log is the source used to replay safely.

Sequence diagram: log before acknowledge

log before acknowledge The acknowledgement is safe because recovery can find the message in the log. sd log before acknowledge Producer Producer Message Broker Message Broker Write Ahead Log Write Ahead Log Broker Restart Broker Restart Send message M7. Append M7 and flush it. Acknowledge only after thelog is durable. Replay M7 after a crash.

The acknowledgement is safe because recovery can find the message in the log.

The durability rule

Related storage topics

Written by Arunkumar Ganesan.

What I learnt is that recovery is designed before the crash, and the log is the part that lets the system remember what it promised.

#DistributedSystems #WriteAheadLog #Databases #Reliability