Blog / How-to guides

Semurg as a document store: store, fetch, find, and the limits

12 October 2026 · Semurg · 4 min read · Markdown

Verified: every command below ran on a fresh node of r11.78.20261012.bc0d94f (how)

documentdatabasehow-to

Most reads from a document store fetch one document by its id: the order, the profile, the event. Semurg keeps each document as an opaque value under your own id, so a read by id is one lookup and a stream of the bytes, with no parsing on the way out.

What it does not do yet is look inside a document: there is no query by a field and no secondary index, so this is a store for reading documents by id, with a word index beside it, not a replacement for every MongoDB query.

A JSON document is stored as bytes under your id, in one record when small or as a record and pieces on every disk when large; a read by id is one lookup and a stream with no parsing, and finding by word goes through the search door under the same id.A JSON documentStored under your id,as bytesSmall: one recordLarge: a record andpieces on every diskRead by id: onelookup, no parsingFind by word: thesearch door, same id

Store documents under your ids

A document is any bytes you choose, so JSON works as it is: send it base64 encoded, under a whole-number id. One call can carry many:

bash
export SEMURG_API_TOKEN=<your token>
curl -sS http://localhost:4000/v1/query \
  -H "Authorization: Bearer $SEMURG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"op": "object_put",
       "objects": [
         {"id": 900000000401,
          "value_b64": "eyJ0eXBlIjoiUHVzaEV2ZW50IiwicmVwbyI6ImFjbWUvYmlsbGluZyIsImFjdG9yIjoiZGFuYSIsImNvbW1pdHMiOjN9"},
         {"id": 900000000402,
          "value_b64": "eyJ0eXBlIjoiSXNzdWVzRXZlbnQiLCJyZXBvIjoiYWNtZS9iaWxsaW5nIiwiYWN0b3IiOiJsZWUiLCJhY3Rpb24iOiJvcGVuZWQifQ=="}]}'

Expected: both documents are written

The two values are {"type":"PushEvent","repo":"acme/billing","actor":"dana","commits":3} and an IssuesEvent for the same repository. A small document is stored inside a single record; a large one becomes one record that names it plus a run of fixed-size pieces spread across every disk, so reading it back is one sweep. Writing to an id again stores a new version, and a read returns the newest. Add "content": true to skip any id that already holds a live value.

Read them back

bash
export SEMURG_API_TOKEN=<your token>
curl -sS http://localhost:4000/v1/query \
  -H "Authorization: Bearer $SEMURG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"op": "object_get", "ids": [900000000401, 900000000402, 900000000499]}'

Expected: the first two documents are present and the third id is absent

Every id answers with a state, in the order you asked: present with its value, absent, deleted, or a named state when a value could not be read or is larger than the call allows, never a silent gap. object_delete with the same ids removes them; a later read answers deleted.

Find documents by word

The store reads documents by id. To find them by what they say, index their text with the search door, under the same ids. Here, a line of text for each of the two events:

bash
export SEMURG_API_TOKEN=<your token>
curl -sS http://localhost:4000/v1/query \
  -H "Authorization: Bearer $SEMURG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"op": "index_documents",
       "docs": [{"id": 900000000401, "body": "dana pushed three commits to acme billing"},
                {"id": 900000000402, "body": "lee opened an issue on acme billing"}]}'

Expected: two documents indexed under the same ids as the stored events

Then ask which documents hold a word, and read those documents back by id:

bash
export SEMURG_API_TOKEN=<your token>
curl -sS http://localhost:4000/v1/query \
  -H "Authorization: Bearer $SEMURG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"op": "term_postings", "word": "opened"}'

Expected: the one document that mentions opened

The ids it returns are the documents' own ids, so one object_get with them brings back the events themselves, here both of them:

bash
export SEMURG_API_TOKEN=<your token>
curl -sS http://localhost:4000/v1/query \
  -H "Authorization: Bearer $SEMURG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"op": "object_get", "ids": [900000000401, 900000000402]}'

Expected: both events come back present

The search index post covers the search door in full, and its limits: single words, no ranking yet.

Where it stands

We have taken our published benchmark rows down while every domain is re-measured on new hardware, with a dataset in RAM and one bigger than RAM running at the same time, as a stress test of Semurg itself. The re-measured document results will be on the benchmark page, wins and losses alike, with the release and the machine they ran on. Until then this post states no comparison.

What it does not do yet

  • Query by a field inside a document, or keep a secondary index: a document is opaque bytes to the store.
  • Update part of a document: write the whole new version under the same id.
  • Use ids other than whole numbers.
  • Return more than a bounded number of bytes in one read call: a call that would exceed it answers each oversized document by name.
  • Load straight from a running MongoDB: there is no connector for it in the published release yet.

Next

How these commands were checked

Our test farm ran each of the 5 command blocks marked runnable, exactly as written, with the answer checked against the expectation printed under it, against a fresh node of the published release r11.78.20261012.bc0d94f. A post is published only when every one of them passed there.

Related