---
title: "Why large mixture-of-experts models suit CPUs"
slug: why-large-moe-models-suit-cpus
date: 2026-11-17
author: Semurg
section: thought-leadership
tags: [moe, cpu, inference, models]
summary: The biggest language models are mixture-of-experts models: very large to store, light to use. Writing each word is limited by memory, not arithmetic, and memory is what a CPU server has most of.
status: published
constants: [896, 16]
evidence:
---
Many of today's largest open models are mixture-of-experts models. They are very large to store and light to use, so writing each word is limited by memory, not arithmetic. GPU servers hold them in small, expensive accelerator memory; CPU servers have more memory per dollar. That is the whole argument. The rest of this page is the reasoning behind it, in plain terms.

## Big to store, light to use

A mixture-of-experts model holds hundreds of small networks called experts, and for each word it writes it consults only a few of them. One current large mixture-of-experts model, for example, has 896 experts and uses 16 of them for each word. The whole model has to be kept somewhere. Each word touches a small part of it.

## One word is a read

To write one word the model reads the weights it uses once. For one person talking to the model, the time per word is set by how fast those bytes move out of memory, not by how fast the processor multiplies them. The arithmetic is quick. The wait is for the data.

## So the question is memory

How much memory can hold the model, how fast can it be read, and what does it cost? A CPU server holds a very large model in ordinary memory. The weights of one expert do not have to sit beside the weights of another.

## Many servers, one model

In principle, spreading the experts across several CPU servers would add memory and read speed together: each word would read the few experts it needs, and the machines could read at the same time.

## Where GPUs win

Reading a brand-new, very long prompt is different work. It is arithmetic, not memory, and GPUs are far stronger at it. For an agent that builds a conversation turn by turn, that cost is paid once. A provider that keeps an earlier part of a conversation charges each later turn only for what was added; Semurg publishes its figure for this on the performance page once a release measures it.

## What we publish, and when

Speeds appear on the [models page](/models) only when a published release has measured them, with the release named. Until then a row says that it has not been measured. The [performance page](/performance) shows what each release measured.

> **What Semurg is:** a memory that lasts for your AI agents, on a machine you run.
> **Why use it:** your agent recalls the facts it needs, each with its source, instead of reading the project again.
> **What it solves today:** agents that forget between sessions, and data that must stay on your own machines.
> **Try it:** `curl -fsSL https://one.semurg.io/install | sudo bash`, first node free forever.

*This post was written with AI assistance. Every result in it is checked against a published release or record before the post goes out.*
