Tag archive
#benchmarking
Benchmarking · 4 field reports
Chronological feed
Same Weights, Different Engine: XeStrata on the Arc Pro B70, and the Conversation That Survived a Restart
A second engine for the same Qwen3.8 model wins on speed, then learns to keep long conversations on the SSD. Also: the test harness that asked us to load the wrong model.
Approved for the main role. Same weights, tuned XeStrata: decode +6 to +15%, prompt reading +40 to +62%, quality 12/12 like production. Our fork resumes long conversations from SSD after a restart. Its SSD write volume needed one more fix, now being measured.
Squeezing Qwen3.8 Until It Cried: A 46.9% Speedup and One Optimization Too Far
A 46.9% Qwen3.8 speedup, a production qualification, and the tempting 64K shortcut that failed when the benchmark got serious.
Promote the full-vocabulary checkpoint: 46.88% faster decode, qualified behavior, and no support for the later 64K shortcut.
How ubatch=1024 Won the Benchmark and Lost the Ability to Speak
A 6% prefill improvement, five failed conversations, and one tiny silicon cult chanting the unused32 token.
Rejected — the narrow prefill win concealed catastrophic generation and tool-call failures.
Qwen3.8 Flash-Next on One Arc Pro B70
Fifteen tokens per second, one extremely narrow memory edge, and several acceleration ideas asked to leave the laboratory.
Promising isolated batch-worker profile, not a production route.