--- title: "Semurg as a document store: store, fetch, find, and the limits" slug: semurg-as-a-document-store date: 2026-10-12 author: Semurg section: how-to tags: [document, database, how-to] summary: Store JSON documents under your own ids, read them back one at a time or many at once and find them by word, with what the store does not do yet. status: published covers: [route:POST /v1/query, op:object_put, op:object_get, op:object_delete] --- Most reads from a document store fetch one document by its id: the order, the profile, the event. Semurg keeps each document as an opaque value under your own id, so a read by id is one lookup and a stream of the bytes, with no parsing on the way out. What it does not do yet is look inside a document: there is no query by a field and no secondary index, so this is a store for reading documents by id, with a word index beside it, not a replacement for every MongoDB query. > Diagram doc-by-id: A JSON document is stored as bytes under your id, in one record when small or as a record and pieces on every disk when large; a read by id is one lookup and a stream with no parsing, and finding by word goes through the search door under the same id. ## Store documents under your ids A document is any bytes you choose, so JSON works as it is: send it base64 encoded, under a whole-number id. One call can carry many: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "object_put", "objects": [ {"id": 900000000401, "value_b64": "eyJ0eXBlIjoiUHVzaEV2ZW50IiwicmVwbyI6ImFjbWUvYmlsbGluZyIsImFjdG9yIjoiZGFuYSIsImNvbW1pdHMiOjN9"}, {"id": 900000000402, "value_b64": "eyJ0eXBlIjoiSXNzdWVzRXZlbnQiLCJyZXBvIjoiYWNtZS9iaWxsaW5nIiwiYWN0b3IiOiJsZWUiLCJhY3Rpb24iOiJvcGVuZWQifQ=="}]}' ``` ```expect expect: both documents are written assert: json:.ok==true and .written==2 ``` The two values are `{"type":"PushEvent","repo":"acme/billing","actor":"dana","commits":3}` and an `IssuesEvent` for the same repository. A small document is stored inside a single record; a large one becomes one record that names it plus a run of fixed-size pieces spread across every disk, so reading it back is one sweep. Writing to an id again stores a new version, and a read returns the newest. Add `"content": true` to skip any id that already holds a live value. ## Read them back ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "object_get", "ids": [900000000401, 900000000402, 900000000499]}' ``` ```expect expect: the first two documents are present and the third id is absent assert: json:.objects[0].state=="present" and .objects[1].state=="present" and .objects[2].state=="absent" ``` Every id answers with a state, in the order you asked: `present` with its value, `absent`, `deleted`, or a named state when a value could not be read or is larger than the call allows, never a silent gap. `object_delete` with the same `ids` removes them; a later read answers `deleted`. ## Find documents by word The store reads documents by id. To find them by what they say, index their text with the search door, under the same ids. Here, a line of text for each of the two events: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "index_documents", "docs": [{"id": 900000000401, "body": "dana pushed three commits to acme billing"}, {"id": 900000000402, "body": "lee opened an issue on acme billing"}]}' ``` ```expect expect: two documents indexed under the same ids as the stored events assert: json:.ok==true and .docs==2 ``` Then ask which documents hold a word, and read those documents back by id: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "term_postings", "word": "opened"}' ``` ```expect expect: the one document that mentions opened assert: json:.ok==true and .ids==[900000000402] ``` The ids it returns are the documents' own ids, so one `object_get` with them brings back the events themselves, here both of them: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "object_get", "ids": [900000000401, 900000000402]}' ``` ```expect expect: both events come back present assert: json:.objects[0].state=="present" and .objects[1].state=="present" ``` The [search index post](/blog/semurg-as-a-search-index) covers the search door in full, and its limits: single words, no ranking yet. ## Where it stands We have taken our published benchmark rows down while every domain is re-measured on new hardware, with a dataset in RAM and one bigger than RAM running at the same time, as a stress test of Semurg itself. The re-measured document results will be on the [benchmark page](/benchmark), wins and losses alike, with the release and the machine they ran on. Until then this post states no comparison. ## What it does not do yet - Query by a field inside a document, or keep a secondary index: a document is opaque bytes to the store. - Update part of a document: write the whole new version under the same id. - Use ids other than whole numbers. - Return more than a bounded number of bytes in one read call: a call that would exceed it answers each oversized document by name. - Load straight from a running MongoDB: there is no connector for it in the published release yet. ## Next - [Four kinds of data, one route](/blog/objects-streams-search-and-places): objects, streams, search and places, one call each. - [Talk to a Semurg node over plain HTTP](/blog/connect-over-rest): the token and every route.