---
title: "Kill it, fill its disk, freeze it, cut its power: what a Semurg node gives back"
slug: what-a-semurg-node-gives-back
date: 2026-10-30
author: Semurg
section: engineering
tags: [durability, reliability, self-hosted, testing]
summary: We break a running node the ways real machines break, then read back every write it acknowledged. A killed process, a full disk, a slow disk and a frozen process lost nothing; a power cut, the last few seconds.
status: published
constants: []
evidence:
  - release:r11.138.20261029.4109aec
---
A database earns trust by what it gives back after something goes wrong. So we break a running node the ways real machines break,
then read back every write the node acknowledged. These are the results on the release published on 29 October 2026, and each one
sits on our [durability page](/durability) beside the release it was measured on.[^1]

## A killed process

Thirty-two writers kept writing while we killed the node with `kill -9` 21 times in 140 seconds, restarting it each time.[^1]
Afterwards we read back all 32.2 million acknowledged writes, each at its newest acknowledged version. None was lost.[^1]

## A full disk

We kept writing until the data volume filled. As it neared full, the node refused new writes by name (HTTP 507, "insufficient
storage") and stayed healthy, and all 2,183,488 writes it had acknowledged read back.[^1] We then grew the volume while the node kept
running: writes were accepted again with no restart, and all 6,190,464 acknowledged writes read back.[^1]

## A slow disk

We throttled the disk to 200 MB/s, then 40 MB/s, then 8 MB/s, under write load.[^1] The node got slower and never wrong: every
acknowledged write read back at every level.

## A frozen process

We paused the node 5 times for 15 seconds each while it was under load, and all 15.2 million acknowledged writes read back.[^1]
During the pauses, 80 client calls timed out.[^1] None of those was an acknowledged write that went missing: a call that times out was
never acknowledged, so the client sends it again, and every call sent again was stored.

## A power cut

We replayed sudden power loss at the block device, on a datacenter drive with power-loss protection, 21 cuts in each of two data
shapes.[^1] Everything acknowledged before the final seconds survived every cut, and nothing that read back was ever damaged. Writes
acknowledged in the last seconds before a cut can be lost on ext4: at most 9.4 seconds in one shape and 4.7 seconds in the other,
inside the 10-second bound we test against.[^1] Our goal there is zero.

## What it means if you run it

Your data stays on your own server, and the node keeps what it told you it kept: through a crash, a full disk, a slow disk or a
frozen process, with the power-cut limit above. When it cannot keep a write, it refuses that write by name at the moment you send
it, so nothing is accepted now and lost later.

The tests run again on new releases, and the [durability page](/durability) names the release behind every result. To try a node
yourself, [install one on Linux](/install).

*This post was written with AI assistance. Every result in it is checked against a published release or record before the post goes out.*