--- title: "Four kinds of data, one route: objects, streams, search and places" slug: objects-streams-search-and-places date: 2026-10-12 author: Semurg section: how-to tags: [objects, streaming, search, geo, how-to] summary: Semurg keeps objects, message streams, searchable documents and points on a map in the same store as its graph. Each is one operation on the same HTTP route. Here is how to write and read each one. status: published covers: [route:POST /v1/query, op:object_put, op:object_get, op:object_delete, op:stream_produce, op:stream_read, op:stream_count, op:index_documents, op:delete_documents, op:term_docs, op:term_postings, op:geo_put, op:geo_within] --- Most applications need more than one kind of data. Files and blobs, a stream of events, text that has to be searchable, places on a map. Usually each is its own system. In Semurg they are the same store, written and read through the same route, `POST /v1/query`, with an `op` that says which kind of data the call is about. This guide shows one write and one read for each of four kinds. > Diagram one-route-four-kinds: One query route takes an op that names the kind of data: objects by id, messages on a stream in order, documents found by word and points found near a center, all in the one store where the ids connect. Each block puts your token in the shell first; replace `` with yours. ## Objects: store a value of any size under an id An object is a value of any size, from a few bytes to a large file, stored under a numeric id. Values travel as base64 over HTTP. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"object_put","objects":[{"id":900000000001,"value_b64":"aGVsbG8gc2VtdXJn"}]}' ``` ```expect expect: the object is written (or skipped, if this node already holds exactly that value) assert: json:(.written + .skipped)==1 ``` Read it back by id. Each object in the answer says its state: `present` with its value, `absent` if the id holds nothing, `deleted` if it was deleted, and a few more states that name exactly why a value could not be returned. None of them is a silent empty value. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"object_get","ids":[900000000001]}' ``` ```expect expect: the object, present, with the same base64 value that was stored assert: json:.objects[0].state=="present" and .objects[0].value_b64=="aGVsbG8gc2VtdXJn" ``` To delete, send `{"op":"object_delete","ids":[...]}`; the answer says how many were deleted, and a later read answers `deleted` with the generation it happened at. One call over HTTP returns at most `64` MiB of values; a value larger than that answers `too_large` rather than arriving cut short. ## Streams: append messages, read them in order A stream is a named topic that messages are appended to and read from in order. A message is acknowledged only once it is readable. > Diagram stream-cursor: A producer appends messages to a topic, each acknowledged once readable; a consumer reads everything after its cursor and, with a wait, waits for the next message instead of polling. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"stream_produce","topic":"orders","values":["order placed","order paid"]}' ``` ```expect expect: the first and last sequence numbers of the two messages, and the topic's cursor assert: json:(.last - .first)==1 ``` Read everything after a cursor. A consumer keeps the cursor it was given and asks again from there; with `wait_ms` it waits for the next message instead of polling. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"stream_read","topic":"orders","cursor":0,"max":10}' ``` ```expect expect: the messages after the cursor, each with its sequence number and its value in base64, and the new cursor assert: json:.count>=1 and (.messages|length)==.count ``` And how many messages a topic holds, for one topic or many in one call: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"stream_count","topics":["orders"]}' ``` ```expect expect: the count for the orders topic assert: json:.counts[0]>=2 ``` ## Document search: index text, find it by word Index documents by id, then ask which documents contain a word. Words are lower-case runs of letters and digits. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "index_documents", "docs": [{"id": 900000000101, "body": "the submarine surfaced at dawn"}, {"id": 900000000102, "body": "a yellow submarine song"}]}' ``` ```expect expect: the number of documents and words indexed assert: json:.docs==2 ``` The documents that contain a word, as ids in ascending order, with how many there are: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"term_postings","word":"submarine"}' ``` ```expect expect: both documents' ids for the word submarine assert: json:.df>=2 and ([.ids[]|select(.==900000000101 or .==900000000102)]|length)==2 ``` For counts only, `{"op":"term_docs","words":["submarine","dawn"]}` answers how many documents hold each word, in the order you asked. To remove documents from the index, send `delete_documents` with the same documents you indexed. A word that is not a single word is refused by name, and a store that has never been indexed says so; neither is answered with a zero. ## Places: points, and which are near a center Store a point for an entity. Coordinates are signed integers in millionths of a degree: latitude, then longitude. ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op": "geo_put", "points": [[900000000201, -33856800, 151215300], [900000000202, -33873300, 151206900], [900000000203, 51507400, -127800]]}' ``` ```expect expect: three points written assert: json:.written==3 ``` Then ask which of a set of candidates lie within a radius of any of a set of centers. Centers are ids too, so only ids travel. Here the center is a point in Sydney, and the candidates are a second point in Sydney and one in London: ```bash runnable kind=shell-node needs=token export SEMURG_API_TOKEN= curl -sS http://localhost:4000/v1/query \ -H "Authorization: Bearer $SEMURG_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"op":"geo_within","candidates":[900000000202,900000000203],"centers":[900000000201],"radius":20000}' ``` ```expect expect: only the second Sydney point assert: json:.ids==[900000000202] ``` Be precise about what the radius means: it is in millionths of a degree, tested as a straight distance on the latitude and longitude grid, not in metres along the earth's surface. A degree of longitude is shorter than a degree of latitude away from the equator, so choose the radius for your own latitudes. The test only looks at the candidates and centers you send, never the whole store, so its cost grows with your lists, not with your data. ## What these have in common All four are the same store, so the ids are yours to connect. A document's id can be an entity in the knowledge graph, an object can be the file a passage came from, a point can belong to a customer the memory already knows. Each answer names its state, so an empty or partial answer always says why. ## When something is refused - `422`: the body is malformed, and the message names the field: a bad id list, value, topic, cursor, point or radius. - `413`: the call is past a pack's bounds; send less per call. - `501`: this release does not carry that kind of data. - `503` on a stream write: nothing was acknowledged; send it again. - `401`: the token is missing or wrong. ## Next - [Hybrid records](/blog/hybrid-records-vector-filter-walk): a vector, attributes, text and links in one record, searched and walked in one call. - [Look after your node](/blog/look-after-your-node): the calls that tell you how it is doing.