Wins and losses in the same table.
Every domain has a lane script, including the domains with no result yet. A result appears here only after both engines answered the same question identically, three times, cold and warm, with the machine's utilisation recorded on both sides. Until a lane has been run that way on current hardware, this page says so instead of showing a number.
How a lane is run
The twelve domains
| Domain | What the lane asks | State |
|---|---|---|
| Graph | Follow relationships, several steps deep | Results pending |
| Vector | Nearest neighbours of a vector | Results pending |
| Time-series | Ranges and aggregates over a long history | Results pending |
| Relational | Point reads and writes against a schema | Results pending |
| Columnar | Grouped counts and sums over many rows | Results pending |
| Key-value | Get and put by key | Results pending |
| Object | Write and read whole objects | Results pending |
| Document | Fetch and filter documents | Results pending |
| Streaming | Append a stream and read it back | Results pending |
| Search | Full text over a corpus | Results pending |
| Universal | One question that spans several of the above | Results pending |
| Knowledge retrieval | Answer a question from a corpus | Results pending |
Every row here reads results pending: the lane ships and runs, and its figures publish here as each hardware configuration is measured, with the release and the machine printed beside them.
The four states a row can have
Measured
Both figures, the release, the three cycles, both sides' utilisation, and the command that reproduces it.
Results pending
The lane runs; its figures are being measured across hardware configurations. Where a lane would drive another engine through a slower path than the vendor's own tool, we say so rather than publish a number that is unfair to them.
Not published
That engine's licence forbids publishing benchmark results, so we publish none, whatever they show. No name, no figure, no direction.
Not run
No independent reference answer exists for this test yet, so a number here could not be checked.
All four render at the same size and weight. A loss is never smaller or greyer than a win.
About losses
Where another engine is faster, its row says so with the same detail as a row where we are faster.
An engine built for one job usually wins at that job. A page that hid that would make the rest of this worthless.
When results appear
Results publish domain by domain as each one clears the bar, not in one announcement at the end.
A domain's row changes state on the day its run is recorded, and the release it was measured on is printed beside it.
What every measured row must carry
- Both answers agreed, checked before anything was timed
- Three cycles: cold, warm, warm again
- Utilisation, both sides, so neither engine is quietly under-driven
- The release it was measured on, and the store width (nodes by disks)
- The command, so you can run the same lane yourself