← Back to Léo's Pages

Corpus v3 · 99 agentic cases · updated 18 Nov 2026

Where the household models stand

Every model we have put through our own agent benchmark, ranked by its best score. Green bars are models we can run at home; blue bars are cloud only. A small picture over a bar shows the machine we ran it on.

Bar chart of corpus v3 scores out of 99, highest first. Gemini 4 Neon 86, Gemini 4 Argon 83, GLM 5.3 82, Muse Wonder 80, then GLM 5.3 Flash on our DGX Sparks at 79, level with GPT-6.1 Sol, Kimi K3, Qwen3.8 Max Prime and the cloaked Crouton Alpha. Local models are green, cloud models blue.

What we test. Corpus v3 is 99 test cases drawn from the real work our household agents do every day. Each case gives a model a system prompt, a set of tools and a task, and a deterministic harness drives the session and grades what the model actually did: the tool calls it made, the files it wrote and the answer it gave, never its own description of the work. The cases cover exact tool-call formatting and knowing when not to call a tool; copying values exactly when decoys are nearby; calendar and mail chores; coordinating with other agents without inventing work or obeying second-hand authority; systems maintenance, where you diagnose before you edit and verify before you claim success; applying business rules from our ad-sales app to messy data; building a small web page; long multi-step sessions; picking up correctly after the conversation has been compressed; code and skills; and finding one planted instruction in 50,000 to 200,000 tokens of context.

Every model runs on the same harness at the same settings (low reasoning effort). We publish each model’s best run; one or two cases either way is noise. A separate 24-case security suite tests secret handling, prompt injection and fake authority, and we read it beside the corpus score, because a model we cannot trust is out however clever it is.

Best local model

GLM 5.3 Flash on the Sparks: 79

Our everyday model ties for fifth with four others, among them GPT-6.1 Sol, and sits seven points behind the top of the board, ahead of Claude Opus 5.5 (78) and GPT-6 Astra (77) on this corpus.

The top of the board

Gemini 4 Neon stands alone at 86

Neon scored 86 with 23 of 24 on security, both board highs (one run, on a Google subscription). Below it, thirteen models land between 77 and 83, where reliability and the security suite separate more than the corpus total.

The Mac Studio, first two days

Close, but not past the Sparks

Our 512 GB M5 Ultra ran five models. Best: the full GLM 5.3 at 4-bit, 76 with security 18, but it writes only 27 tokens a second. GLM 5.3 Flash at 6-bit and Muse Spark Open at 3-bit scored 75, DeepSeek V4.1 Flash 59. None beats GLM 5.3 Flash on the Sparks (79).

Mac Studio (ultron), oMLXBest /99Security /24Speed tok/s code / proseRuns
GLM 5.3 (full), 4-bit761827 / 271
GLM 5.3 Flash, 6-bit751972 / 523
GLM 5.3 Flash, 6-bit + reasoning restore751972 / 523
Muse Spark Open, 3-bit751827 / 271
GLM 5.3 Flash, 4-bit741591 / 653
DeepSeek V4.1 Flash, lossless591670 / 503
GLM 5.3 Flash on our Sparks (reference)791675 / 435

The full ranking

Best corpus score of 99, the number of runs behind it, and the 24-case security score where we have it.

#ModelBest /99RunsSecurity /24Speed tok/s
code / prose
Runs on
1Gemini 4 Neon (low)86123—Cloud only
2Gemini 4 Argon (low)83121—Cloud only
3GLM 5.3 (full, Z.ai cloud)82218—Fits our hardware (tested in the cloud)
4Kimi K3.1 Pro (Nous, cloud)80117—Cloud only
5Muse Wonder80117—Cloud only
6GLM 5.3 Flash EXL3 (Sparks, pinned)7951675 / 43DGX Sparks
7Crouton Alpha (stealth)79118—Cloud only
8GPT-6.1 Sol79120—Cloud only
9Kimi K379119—Cloud only
10Qwen3.8 Max Prime79116—Cloud only
11GLM 5.3 Flash FP4 (NVIDIA Flare, Sparks)7831836 / 31DGX Sparks
12Muse Spark Open (OpenRouter cloud)78219—Fits our hardware (tested in the cloud)
13Claude Opus 5.5 (low)78119—Cloud only
14GLM 5.3 Flash EXLR8 K4 (Sparks)7721772 / 39DGX Sparks
15GPT-6 Astra77119—Cloud only
16GLM 5.3 (full) 4-bit (oMLX, Mac Studio ultron)7611827 / 27Mac Studio M5 Ultra (measured)
17GLM 5.3 Flash 6-bit + reasoning restore (oMLX, Mac Studio ultron)75319—Mac Studio M5 Ultra (measured)
18GLM 5.3 Flash 6-bit oQ6e (oMLX, Mac Studio ultron)7531972 / 52Mac Studio M5 Ultra (measured)
19GLM 5.3 Flash NVFP4 QAD (Sparks)75216—DGX Sparks
20Muse Spark Open 3-bit oQ3e (oMLX, Mac Studio ultron)7511826 / 27Mac Studio M5 Ultra (measured)
21GLM 5.3 Flash 4-bit oQ4e (oMLX, Mac Studio ultron)7431591 / 64Mac Studio M5 Ultra (measured)
22Ling 3.1 Flash74214—Mac Studio (fits; not measured yet)
23Qwen3.8 Flash-Next NVFP4 QAD (Sparks)7425/12 (half suite)—DGX Sparks
24Qwen3.8 27B73120—Fits our hardware (tested in the cloud)
25Qwen 4 Max (Nous, cloud)73123—Cloud only
26GLM 5.3 Flash (Z.ai)72219—Fits our hardware (tested in the cloud)
27Inkling 2 Small72119—Mac Studio (fits; not measured yet)
28MissingNo (Nous stealth)72211—Cloud only
29Qwen3.8 Flash-Next IQ3_S (Mojo, pinned)71217—RTX 3090 workstation
30GLM 5.3 Flash FP4 (SparkGLM 0.3.5, Sparks)7031366 / 38DGX Sparks
31StepFun Step 5 Preview70216—Mac Studio (fits; not measured yet)
32Nemotron 3 Ultra6819—Mac Studio (fits; not measured yet)
33Nemotron 3.5 Super VL68213—Fits our hardware (tested in the cloud)
34Qwen3.6-35B-A3B Splash (Mac mini, pinned)68215—Mac mini
35DeepSeek V4.1 Flash 1017 (cloud)67213—Fits our hardware (tested in the cloud)
36GLM 5.3 Flash Zig 1104 (Sparks)6711966 / 36DGX Sparks
37MiniCPM-V 4.7 35B-A3B mix_q367112—Framework Desktop
38Gemini 3.8 Flash (low)67118—Cloud only
39DeepSeek V4.1 Flash64116—Fits our hardware (tested in the cloud)
40Qwen3.8 Flash-Next on Atlas (SparkQwen)64214—DGX Sparks
41DeepSeek V4.1 Flash 1017 (Allyn's BB8, full precision)62317462 / 238Cloud only
42Tencent Hy4 Flash61114—Mac Studio (fits; not measured yet)
43DeepSeek V4.1 Flash 1017 lossless (oMLX, Mac Studio ultron)5931670 / 50Mac Studio M5 Ultra (measured)
44DeepSeek V4.1 Flash 1017 4-bit oQ4e (oMLX, Mac Studio ultron)5731471 / 49Mac Studio M5 Ultra (measured)
45Nemotron 3 Super5616—Fits our hardware (tested in the cloud)
46Recursive Flint Q4_1 (Mojo, llama.cpp)553875 / 74RTX 3090 workstation
47LFM2.5 24B-A2B Q84525—Framework Desktop
48Tencent Hy439113—Mac Studio (fits; not measured yet)
49Naive N0.5 Flash (Sparks)322——DGX Sparks

Green squares mark models we can run on our own hardware; blue squares are cloud only. “Mac Studio” means the open weights would fit a 512 GB M5 Ultra Mac Studio, which we do not own yet.