210K Context on a 32 GB Arc Pro B70

An AI model’s context window is its working notebook: the larger it is, the more of a long conversation, document, or codebase the model can keep in view at once. The awkward part is that the notebook has to share one 32 GB graphics card with the model itself. Marketing may promise a library; the hardware would still like somewhere to sit.

Our experiment asked how large that notebook could become in production, not merely whether the server could allocate an impressive number once. A useful result had to start repeatedly, remember an exact fact near the far end, handle images, reasoning, and tools, survive recovery checks, and leave us a clean rollback. Otherwise we had built a very large context window through which nothing dependable could pass.

Experiment summary

The short version

Question

What is the highest production context envelope that repeatedly starts, performs useful near-ceiling work, reports usage correctly, survives recovery checks, and rolls back cleanly?

Control

The admitted 138K Qwen3.8/vLLM XPU route at GPU utilization 0.88 and one sequence.

Candidate

A 210K server window at utilization 0.95, with OpenClaw active input capped at 200K and a 20K safeguard reserve.

Outcome

Three cold starts agreed on 212,957-token KV capacity, and the authenticated route passed the full suite at 179,525 reported prompt tokens.

The question

The model advertises a native context window of 262,144 tokens. The B70 has 32 GB of VRAM. These facts are individually true and, when placed next to each other in marketing copy, become dangerously sociable.

Our admitted production route was more conservative:

• Qwen3.8 27B GPTQ INT4 symmetric G128.
• BF16 MTP depth four.
• F16 vision projector.
• FP8 KV cache.
• vLLM XPU on one Intel Arc Pro B70.
• 138,000-token server and OpenClaw window.
• gpu-memory-utilization=0.88 and one concurrent sequence.

The question was not whether vLLM could allocate a large number once. It was:

What is the highest production context envelope that repeatedly starts, performs useful near-ceiling work, reports usage correctly, survives recovery checks, and can be rolled back cleanly?

Replication recipe

Prerequisites and immutable provenance

• One Intel Arc Pro B70 with a 30.3 GiB device-free baseline and a one-request concurrency policy.
• vLLM XPU image sha256:f01e24f6c7ff01f1e0662234255a1372297d1dbd89d003cf13c8fad3eab1ba4f.
• vLLM 0.27.2rc1.dev77+gac7509e2b at commit ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9, with kernels 0.1.12.3.
• SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16 at revision 9d189a60e4c0ad7f9f47cd94bfa393ca10b3924e.
• Cookbook provenance at commit 44e97e1927b1d66c6a26398bcfae6d67bce8aa50, including the family-specific MTP patches.
• The F16 vision processor overlay sourced from official Qwen/Qwen3.8-27B revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0; keep it separate from the immutable model snapshot.
• An authenticated OpenAI-compatible Router route and an OpenClaw model row that can be validated without exposing credentials, addresses, or private service names.

The historical evidence retains the exact image, runtime, model, and cookbook identities, but not the literal image-build command. Do not substitute a guessed build invocation. Verify the immutable image digest and model revision, or rebuild from the cited cookbook/runtime commits and treat that as a new provenance chain.

These are the exact retained vLLM argument fields relevant to the final route:

--max-model-len 210000
--gpu-memory-utilization 0.95
--max-num-seqs 1
--kv-cache-dtype fp8
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
--reasoning-parser qwen3

The route also used the model’s BF16 MTP head at speculative depth four and the full F16 vision path. The publication evidence does not retain the literal MTP CLI spelling from the original Compose command, so it is intentionally not reconstructed here. Native metrics must prove four-position MTP drafting before the run is accepted.

The exact recorded Router/OpenClaw contract was:

Router model id: qwen38-27b-b70-vllm-mtp4-vlm
Router advertised context: 210000
Router runtime selector: llamacpp
Effective backend implementation: vLLM XPU

{
  "id": "qwen38-27b-b70-vllm-mtp4-vlm",
  "contextWindow": 210000,
  "contextTokens": 200000,
  "maxTokens": 8192,
  "compat": {
    "thinkingFormat": "qwen-chat-template",
    "supportsUsageInStreaming": true
  }
}

That JSON is the copyable model-row fragment, not a complete OpenClaw configuration. Keep provider credentials, endpoint addresses, agent assignments, and fallback arrays out of the recipe.

Controlled variable and acceptance sequence

1. Hold image, model revision, processor overlay, MTP4, FP8 KV, vision/tool/reasoning parsers, and --max-num-seqs 1 fixed.
2. Establish the 138K/0.88 control and record its free-memory baseline, model load, fixed overhead, KV allocation, and reported capacity.
3. Probe GPU-memory utilization independently. Reject values that request more memory than the observed free baseline; in this experiment 0.96 and above failed, while 0.95 was the highest accepted value.
4. At 0.95, probe context lengths in fresh processes and retain both startup success and reported KV capacity. Do not infer the ceiling from arithmetic alone.
5. Before persistent mutation, require three fresh 210K starts. Every start must report at least 210,000 KV tokens; the recorded three all reported 212,957 and 1.010× maximum concurrency. If any start fails, select the already-passed 202K fallback.
6. Run the direct near-ceiling suite with the same chat template used for generation: exact marker recall near 179.5K prompt tokens, immediate short-request recovery, vision, structured reasoning, tool selection, tool-result continuation, MTP activity, model identity, and native-log anomaly scan.
7. Back up the Compose, Router profile, and OpenClaw model row. Apply the candidate atomically, verify the effective vLLM flags, and prewarm through the Router.
8. Repeat the complete 179.5K suite through the authenticated in-network Router path. Never place endpoint addresses or credentials in the public recipe.
9. Run a fresh OpenClaw turn and require the exact provider/model, no reroute or fallback, contextTokens=200000, promptBudgetBeforeReserve=180000, reserveTokens=20000, and fresh nonzero usage accounting.
10. On any post-mutation failure, restore all three configuration layers together and revalidate the catalog, configuration, backend idle state, assignments, and fallbacks.

Success is three identical 212,957-token cold starts, exact routed recall at 179,525 reported prompt tokens, clean recovery/vision/reasoning/tools, active MTP, and the 200K/180K/20K OpenClaw budget with no fallback. Failure includes any cold-start disagreement below the envelope, token-accounting mismatch caused by unequal templates, an unreachable Router path, partial configuration generation, missing rollback proof, or a native anomaly. The model is not admitted merely because it allocates memory and looks pleased with itself.

Starting point

At the original 138K configuration, vLLM measured 147,857 tokens of KV capacity. That left 9,857 tokens—about 7.1%—above the configured window.

The measured B70 memory shape was:

The immediate lesson was that “native 262K” describes model capability, not the memory budget of this specific quantization, vision/MTP stack, runtime, and GPU.

This was the moment when the simple capacity question became a three-layer configuration transaction. I refilled the coffee before the configuration files developed opinions.

First repair: make usage visible

Before changing the context window, we corrected OpenClaw’s accounting for the custom Model Router route.

Generic OpenAI-compatible provider defaults had disabled streaming usage requests. The vLLM endpoint could report correct final usage, but OpenClaw was not asking for it. We enabled compat.supportsUsageInStreaming for this model row only.

A real Bert validation then reported:

• 9,655 input tokens;
• 34 output tokens;
• 9,689 total tokens;
• persisted totalTokensFresh=true; and
• the exact intended provider/model with no fallback.

A giant context window with a context meter permanently reading zero is less an observability feature than a decorative circle.

Finding the whole-card ceiling

The isolated vLLM probe varied GPU allocation and context without changing the Router or OpenClaw.

Utilization 0.95 was the highest accepted value. Values 0.96 and above failed against the observed free-memory baseline. At 30.3 GiB total, the 0.95 policy budgets approximately 28.785 GiB.

At 0.95, every tested context length through 210,000 started. Across those exploratory boots, vLLM reported:

• 7.82 GiB available for KV;
• cache capacities from 210,461 to 213,410 tokens; and
• approximately 1.01× maximum concurrency at the 210K setting.

The practical single-request ceiling was therefore around 211–213K. We selected 210K for the server envelope and retained 202K as the lower-headroom fallback if repeated cold starts disagreed.

The production contract

The final intended split was:

The distinction matters. The server may accept 210K, but OpenClaw begins protecting the session around a 180K projected prompt so generation, system content, and recovery have room. Filling the absolute allocation ceiling because the number exists would be benchmark theatre with a pager.

Three cold starts

Before persistent mutation, the transaction required three fresh 210K/0.95 starts. All three reported the identical KV capacity:

212,957 tokens
1.010x maximum concurrency

That left only 2,957 tokens of startup capacity above the 210K envelope, but the result was repeatable across the three cold starts. Had any start failed, the transaction would have selected the already-passed 202K fallback instead.

The validation suite

The admission test covered more than allocation:

• near-180K exact recall and continuity;
• post-context short-request recovery;
• vision;
• structured reasoning;
• tool selection and continuation;
• MTP activity and acceptance;
• native-log anomaly scanning;
• exact model/provider identity;
• assignment and fallback preservation; and
• rollback material before mutation.

The final routed suite used 179,525 reported prompt tokens. Exact recall completed in 255.102 seconds, then the same route passed post-context recovery, vision, reasoning, and tools. MTP accepted 171 of 208 drafted tokens, or 82.2115%.

A real Bert turn returned the expected 200K/180K/20K budget contract through the exact model with no reroute or fallback.

The transaction took four versions

The model passed before the deployment harness did. This is why production admission tests the transaction as seriously as the model.

V1 — the 36-token disagreement

V1 completed all three cold starts and recalled the near-180K needle, then stopped before mutation because:

• /tokenize reported 179,537 tokens; and
• generation usage reported 179,501 tokens.

The 36-token difference came from mismatched templates: the tokenizer used the default thinking-enabled template while generation explicitly disabled thinking.

This was a harness defect, not a context failure. The fail-closed gate behaved correctly and installed nothing.

V2 — permission denied before anything interesting

The V2 outer handoff attempted to inspect a root-owned prior evidence archive from an unprivileged fixture and received PermissionError. No host runner, model start, backup, Router stop, or mutation occurred.

The evidence-staging contract was repaired without weakening permissions.

V3 — correct model test, wrong network path, ambitious regex

V3 passed the corrected direct 179,525-token suite, committed the candidate configuration, and proved the effective runtime flags.

Routed validation then tried to reach the Router through the host-published LAN address from a backend-only container with no gateway route. It failed, as networking tends to do when asked to travel through a door that is not there.

Automatic rollback also failed because the helper pattern expected ten digits before T in a run identifier that correctly contained eight date digits. This left a split generation: candidate Compose/profile files with baseline OpenClaw metadata.

A bounded one-off recovery restored exact 138K/0.88 backups, a healthy authenticated Router catalog, valid OpenClaw configuration, and an idle B70 before work continued.

V4 — use the network that exists

V4 used the authenticated in-network Router service from the backend container and replaced the timestamp glob with an anchored numeric regex matching the actual run ID.

It validated the immutable V1/V3/recovery evidence, installed the candidate atomically, reran the complete routed suite, passed the Bert metadata smoke, and retained an independent rollback set.

Production context promotion completed successfully.

What we learned

Operational verdict

The route was promoted to:

server window: 210,000
OpenClaw active input: 200,000
prompt budget before reserve: 180,000
maximum output: 8,192
concurrency: 1

The Gateway hot-applied model metadata without restart. The B70 backend was stopped after validation while the broker and Router remained healthy and idle. Exact pre-upgrade backups and an independent rollback helper were retained.

The headline is that a 32 GB B70 sustained a useful 210K server envelope. The better lesson is that the final model test passed in V1; it still took until V4 to prove that the surrounding machinery deserved to touch production.

Disclosure and corrections

This experiment used 8BitsForge-owned equipment. No vendor sponsored the test, supplied hardware, reviewed the conclusion, or paid for the result. No affiliate links or third-party advertising are enabled.

No corrections have been issued. Any future correction will be dated and appended here.