Troubleshooting and FAQ

Something not working? Start here.

Every problem below is a real one people have hit, written in plain language, each with the exact command that fixes it. If you are brand new, the Start here walkthrough is the happy path. This page is for when a step does not go as expected.

Is it healthy? Running in a container Install errors Cannot reach it Data and queries FAQ
First check

Is the node healthy?

Before anything else, ask the node directly. This one command answers most questions:

curl -s http://localhost:4000/api/health
What you get backWhat it means
{"ok":true, ...}Healthy. It is up, serving, and reading from disk. You are done here.
Connection refused or nothingThe service is not running or not reachable. See Cannot reach it.
a page of errorsSomething is not armed. Check the logs (below) and the container section if you are in Docker.

To see what the service is doing, read its logs:

journalctl -u semurg -n 50 --no-pager

Is it even running? On a normal server:

systemctl status semurg

The full install log, with every step it took, is saved at /var/log/semurg-install.log.

The big one

Running inside a container (Docker, and similar).

On a normal Ubuntu server, the installer sets all of this up and you never see it. Inside a container, three of the container's defaults get in Semurg's way, because Semurg talks to the disk the fast way and locks a block of memory to do it. You do not have to remember any of them: the one command below brings up a fully armed node and supplies all three settings for you.

Install Docker first (a bare server does not have it), then run this one line on an x86_64 host. It downloads the bundle plus the prebuilt image, verifies both, loads the image, and starts the node — no registry pull, no local build:

curl -fsSLO https://one.semurg.io/dl/semurg-docker.tar.gz && curl -fsSLO https://one.semurg.io/dl/semurg-substrate-r11-amd64.tar.gz && curl -fsSL https://one.semurg.io/dl/SHA256SUMS | grep -E ' (semurg-docker\.tar\.gz|semurg-substrate-r11-amd64\.tar\.gz)$' | sha256sum -c - && tar xzf semurg-docker.tar.gz && docker load -i semurg-substrate-r11-amd64.tar.gz && cd semurg-docker && docker compose up
On Windows? These are Linux/bash commands — they do not run in PowerShell or CMD (curl, sha256sum, tar, grep and && are absent or different there). Install WSL2 once with wsl --install in an admin PowerShell, reboot, open Ubuntu, install Docker inside it, and run the line there. ARM / Apple Silicon is not supported (an Intel Mac is fine); use a cloud x86_64 Linux VM instead.

Leave it running. It starts in seconds (no build, no registry pull). When it is up you will see a green line:

[semurg-prove] ARMED: cold-op self-test PASSED, fast disk path + background compaction VERIFIED-ARMED-LIVE.

That is the node proving, on your own hardware, that the fast disk path is live. Now open http://127.0.0.1:4100 in your browser. To stop it: press Ctrl-C in that window, or run docker compose down from the semurg-docker folder (add -v to also delete the stored data).

If a container setting is ever missing, the node refuses to start and prints a red box that names the exact flag and the exact command to paste, it never starts a slow, silently degraded node:

============================================================================
  SEMURG CANNOT START, MISSING CONTAINER CAPABILITY
============================================================================

The docker compose up line above already sets all three, so you should never see that box. Here is what each of the three settings the compose file supplies is for, and what would break without it:

FlagWhyWithout it you see
--ulimit memlock=-1Semurg locks a block of memory so it stays put and stays fast. A container's default locked-memory limit is tiny.the engine fails to start, or a "cannot allocate locked memory" error
--security-opt seccomp=unconfinedDocker's default security profile blocks the modern Linux disk interface Semurg reads through.an operation-not-permitted error when the disk path arms
-v semurg-data:/var/lib/semurgThe store must sit on a real filesystem (ext4 or xfs). Use a named volume (as shown); a host-folder bind-mount lands on the container's layered filesystem, which cannot do direct disk reads.the store fails to open, or direct I/O errors on the data directory

Prefer explicit docker run? Load the prebuilt image once, then run it with exactly those three flags plus a loopback port publish. Nothing else is needed (no --privileged, no --device, no large-page mount):

docker load -i semurg-substrate-r11-amd64.tar.gz
docker run --ulimit memlock=-1 --security-opt seccomp=unconfined -v semurg-data:/var/lib/semurg -p 127.0.0.1:4100:4000 semurg/substrate:r11-test

Both docker compose up and this docker run publish the node on host loopback 4100 (the container listens on 4000 inside), so it is at http://127.0.0.1:4100 either way. Keep the 127.0.0.1: prefix so the port stays private.

Minimal images are missing tools. A slim base image often has no curl and no sudo. A real Ubuntu server ships both. Inside a bare container, install them first:
apt-get update && apt-get install -y curl ca-certificates sudo

Podman, LXC, or another runtime? Same three needs: unlimited locked memory, the modern Linux disk syscalls allowed, and the data directory on a real ext4 or xfs filesystem. The flag names differ by tool, the requirements do not.

Install errors

The installer stopped. What it is telling you.

The installer checks your machine before it touches anything and stops with one plain reason. Find yours here.

The checksum did not say OK

You saw: semurg-r11-installer.tar.gz: FAILED, or no OK line.

The download did not arrive whole. Do not run it. Delete both files and pull them again:

rm -f semurg-r11-installer.tar.gz SHA256SUMS && curl -fsSLO https://one.semurg.io/dl/semurg-r11-installer.tar.gz && curl -fsSLO https://one.semurg.io/dl/SHA256SUMS && sha256sum --ignore-missing -c SHA256SUMS

curl: command not found, or sudo: command not found

You saw: the shell cannot find curl or sudo.

You are on a minimal image. Install them (you may already be root, in which case drop the sudo):

apt-get update && apt-get install -y curl ca-certificates sudo

Too few cores, not enough RAM, or not enough disk

You saw: a preflight message naming cores, memory, or free space.

The minimums are 4 CPU cores, 8 GB of RAM, and 20 GB of free disk. Give the machine (or the container) more of whatever it named, then run the installer again. Check what you have with nproc (cores), free -h (RAM), and df -h (disk).

Wrong OS or version

You saw: a message about Ubuntu version or glibc.

The build targets Ubuntu 24.04 (64-bit x86). Ubuntu 22.04 has an older system library and needs a separate build. Confirm your machine with lsb_release -a and uname -m (you want x86_64).

Apple Silicon & ARM: not supported

You saw: rosetta error: failed to open elf, a Trace/breakpoint trap, exit code 133, or the new SEMURG CANNOT BUILD ON THIS ARCHITECTURE message.

Semurg is x86-64 (amd64) only. The engine uses AVX CPU instructions that do not exist on ARM / Apple Silicon. On an Apple-Silicon Mac, Docker Desktop emulates the amd64 image under Rosetta, which cannot run those instructions — so it is not supported, even emulated. Check with uname -m: arm64 or aarch64 means this machine cannot run Semurg directly.

What to do instead: run Semurg on a cloud x86_64 Linux VM (any provider), and reach it from your Mac in the browser. An Intel Mac (uname -m prints x86_64) can run the Docker path normally.

Windows: the commands do not run in PowerShell

You saw: errors on &&, or curl / sha256sum / tar / grep behaving oddly or "not recognized".

The install commands are Linux/bash, not PowerShell or CMD. In PowerShell curl is an alias for a different tool and sha256sum/tar/grep/sudo are absent, so the line fails immediately.

What to do: install WSL2 once — in an administrator PowerShell run wsl --install, reboot, then open the Ubuntu app from the Start menu. Inside that Ubuntu shell (a real x86_64 Linux) run the same install or Docker commands unchanged.

It says permission denied

You saw: permission denied while installing.

The installer sets up a background service, so it needs administrator rights. Run it with sudo: sudo ./semurg-install.sh.

Port 4000 is already in use

You saw: an address-in-use error, or health never comes up.

Something else is on port 4000. Either stop it, or tell Semurg to use a different port before you install:

sudo SEMURG_PORT=8080 ./semurg-install.sh

Then use that port everywhere in place of 4000.

Re-running the installer

It is safe to run again. The installer is idempotent, so a second run repairs a half-finished install rather than breaking a good one.

Cannot reach it

The install worked, but the page will not open.

Connection refused on localhost:4000

Check the service is running and start it if not:

systemctl status semurg || sudo systemctl start semurg

Inside a container without a service manager, the installer prints the exact command to start the node in the background. Re-read its final lines, or the install log at /var/log/semurg-install.log.

It works on the box, but not from my laptop

The node serves on localhost by design, so it is not exposed to the network by default. That is deliberate: reaching the port is full access to the node. To view the console from your own machine, forward the port over SSH rather than opening a firewall:

ssh -L 4000:localhost:4000 you@your-server

Then open http://localhost:4000 on your laptop. If you do expose the port, put it behind your own reverse proxy or private network. See Access and roles.

Where is the admin cluster console (and its one-time password)?

The admin cluster console is separate from the :4000 graph/query console above. It is a fail-closed, loopback-only HTTPS listener on 127.0.0.1:4610 (self-signed certificate — trust it once), provisioned automatically at first install. The installer prints a one-time admin secret and a TOTP secret in its SUCCESS banner — save them; the secret is shown once (also written to /opt/semurg/tmp/.admin_bootstrap_banner).

Reach it from your laptop by tunnelling the admin port (note: 4610, not 4000):

ssh -N -L 4610:127.0.0.1:4610 root@your-server

Then open https://localhost:4610/cluster, enter the one-time admin secret, and complete TOTP with your authenticator app. Lost the secret? Re-mint it by re-running the install with SEMURG_ADMIN_REPROVISION=1.

Docker: the log says ARMED but http://127.0.0.1:4100 will not open

The engine is up inside the container but the published port is not reaching it. Almost always this is a stale image from before the current release. Remove it, re-load the shipped image, and start again:

docker compose down && docker image rm semurg/substrate:r11-test 2>/dev/null; docker load -i semurg-substrate-r11-amd64.tar.gz && cd semurg-docker && docker compose up

The current image binds all interfaces inside the container so Docker's loopback publish can reach it. Confirm the mapping with docker compose ps (you should see 127.0.0.1:4100->4000/tcp), then curl -s http://127.0.0.1:4100/api/health.

The console loads but the graph is empty

An empty store draws an empty graph. Bring some data in first: the quick three-record proof, or load a large public graph, both on the Start here page.

Data and queries

Loading and reading data.

I cannot find my admin console credential

Your node GENERATED it and printed it once, on first start, under CLUSTER ADMIN CONSOLE (first-run bootstrap) — together with the console URL and the otpauth:// link for your authenticator app. You did not choose it. To read it back out of the boot log:

docker logs $(docker compose ps -q) 2>&1 | grep -A8 'first-run bootstrap'
sudo journalctl -u semurg --no-pager | grep -A8 'first-run bootstrap'

How to open the console (on the box, or over an SSH tunnel from your laptop) is on the cluster admin page.

Every /v1 call returns 401 "this /v1 instance requires an API token"

This is the one wall most people hit, and it is not a fault — your node is secure by default. /v1 is token-gated for reads AND writes, so a published port can never be an open database. You just need the token, which your node printed when it started, under YOUR NODE IS READY.

To get it again, run the line that matches how you installed:

docker exec $(docker compose ps -q) cat /var/lib/semurg/.api_token
sudo grep SEMURG_API_TOKEN /etc/semurg/semurg.env | cut -d= -f2

Then send it on every call:

export TOKEN='paste-your-token-here'; curl -s -X POST http://localhost:4000/v1/query -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json' -d '{"op":"traverse","seeds":[1],"hops":3}'

Prefer a fixed token? Pass SEMURG_API_TOKEN=your-own-secret into the container (or set it in /etc/semurg/semurg.env) and restart — the node will use yours instead of generating one.

A write or load returns 403 on one.semurg.io

That is on purpose. The public reference box at one.semurg.io is read-only, so nobody can write to the shared demo. On your own node, writes work fully. Run write and load commands against http://localhost:4000, not the public site.

A benchmark says the dataset is not staged, or is warming

A fresh node does not ship the large public graphs, so load one before you benchmark it. A dataset that was just loaded reports warming for a moment while it settles, then answers. Benchmarks run from the demo pages (/benchmarks, /demo/graph, /demo/kv, …), not an API endpoint — /api is health and version only.

Loading a large graph ran out of disk

A full public graph such as Wikidata is hundreds of gigabytes raw and more once ingested, so it can exhaust a small disk. Free up space, or point the store at a bigger disk, then load again. Smaller public SNAP graphs, or a redacted slice of your own data, are lighter starting points. See the docs operate section.

Do not run two copies against one store

One writer per store. Starting a second process on the same data directory is unsafe. Always restart through the service manager (sudo systemctl restart semurg), never by launching a second instance by hand.

FAQ

Common questions.

Do I need a GPU?

No. Semurg runs on your CPU. There is no GPU anywhere in the path.

Does it phone home, or need an account?

No. There is no sign-up, no telemetry, and no network call it depends on. You can run it fully offline, air-gapped, and it works the same.

Is it really free?

A single node is free and complete: full writes, no row cap, no time limit, no feature gates. A cluster of several nodes is a paid licence. One node needs no licence at all.

Is my data safe if the process crashes?

An acknowledged write survives a process crash and comes back off the disk on restart. That is tested by killing the process mid-write. Surviving a power cut depends on your drive's power-loss protection, which is why enterprise NVMe is the target. What a single node does not survive is losing the whole machine or its disk, which needs a second machine. The full, honest statement is in the docs durability section.

How do I back it up?

Stop the service and copy the store directory. That is the whole backup. Restoring works the same offline as online.

How do I change a setting, like the port or the disks?

Edit /etc/semurg/semurg.env and restart with sudo systemctl restart semurg. The docs list every setting.

How do I remove it?

Stop and disable the service, then delete the install directory and the data directory:

sudo systemctl stop semurg && sudo systemctl disable semurg && sudo rm -rf /opt/semurg /etc/semurg

Your store lives under the data directory shown by curl -s http://localhost:4000/v1/status before you stop it. Delete that too if you want the data gone.

Still stuck?

Read the logs (journalctl -u semurg -n 50 --no-pager) and the install log at /var/log/semurg-install.log. They name the step that failed in plain words. The docs cover operating the node in full.