Nemotron 3.5 Bolt Is Out: What NVIDIA's 24GB 2-Bit Checkpoint Actually Contains

The repository for N​VIDIA N​emotron 3.5 Bolt 74B-A9B 24GB appeared on Hugging Face on 7 October 2026 and sat there, quietly, for eight days. On 15 October N​VIDIA announced it — a blog post, a newsroom release and a research page stamped with that date — which means the interesting part of this story is not the launch. It is the gap. For eight days the only thing anyone could know about N​emotron 3.5 Bolt was what a model card and a set of safetensors said, and everything else was inference. That gap has now closed, and it is worth being precise about what closing it actually settled, because a good deal of what people are repeating about this model came from the repository rather than from the announcement, and the two are not the same document.

What is certain now, in one paragraph, is this. N​emotron 3.5 Bolt 74B-A9B 24GB is a 74-billion-parameter mixture-of-experts vision-language model with 9 billion parameters active per token, released on 15 October 2026 under the OpenMDW-1.1 licence, storing its expert weights at 2.0625 bits per element so the whole thing fits in 24GB. It is designed to run local agentic work — tool calls, browsing, terminal use — on RTX and DGX Spark hardware. It is on Ollama as a 25GB tag with a 256K context window. It is not on any hosted API, and has no price. One part of that sentence is measured independently and the rest is the vendor's own word.

Two dates, and which one to quote

There is a real difference between the two dates and it decides how you should read the rest of this article.

The Hugging Face repository was created on 7 October 2026 at 17:50 UTC. The model card inside it carries a Release Date field reading Hugging Face 10/15/2026. N​VIDIA's own research page for the checkpoint is stamped Published: October 15, 2026. The vendor blog post announcing the Bolt tier is dated 15 October 2026 and says, in as many words, that Super VL and Bolt 74B-A9B "are available now".

So the honest reading is that 15 October is the release date and 7 October is when the folder showed up. Anyone quoting 7 October as the launch is quoting a file timestamp; anyone quoting 15 October is quoting the vendor. The eight-day gap matters for a different reason than dates, though. It means that for over a week the model existed in a state where the repository was the only surface — no announcement, no pricing page, no leaderboard entry, nobody from N​VIDIA saying what it was for. Every early description of Bolt written in that window, including the ones now circulating, was built from the card. The card turns out to have been accurate. That is worth saying plainly, because the usual outcome of these situations is that it is not.

What the repository alone proves

Everything in this section was readable from the model card and the config files before N​VIDIA said a word, and it is all still true.

• Parameters — 74 billion total, 9 billion active per token. Sparse mixture-of-experts, not a dense model.

• Architecture — a Mamba-2 and Transformer hybrid with a LatentMoE backbone and a C-RADIO v4-H vision encoder. The config registers the class as NemotronH_Omni_Reasoning_V3.

• Stored precision — IQ2_XXS at 2.0625 bits per element on the routed and shared expert FC1 and FC2 layers, which is where the bulk of the weights live. The Mamba linear layers sit at NVFP4, and the embeddings, output head, attention and LatentMoE weights at Q8_0.

• Context — up to 256K tokens, or 262,144 in the deployment recipes.

• Inputs and outputs — text, images and video in; text out.

• Licence — OpenMDW-1.1, and the card states it plainly: "ready for commercial or non-commercial use."

• Bases and provenance — the card declares its base model as the BF16 checkpoint of the same name and its relation to it as quantized, and the dataset tags point at N​VIDIA's N​emotron pre-training and post-training collections.

• Model dates — December 2025 to May 2026, with a post-training data cutoff of May 2026. Read that carefully if freshness matters to you: the model is a 2026 release built on data that stops four and a half months before it shipped.

• Runtimes — vLLM on N​VIDIA Ampere, Hopper and Blackwell, and llama.cpp for RTX Spark and DGX Spark. Ollama's listing is more blunt about the limits: CUDA deployment is documented, and Apple Silicon and Metal are not currently supported.

What the announcement added

The vendor material adds three things the repository could not have told you.

First, the family framing. Bolt is positioned as the edge tier of the N​emotron family, below N​emotron 3.5 Super VL at 120B for always-on server agents and above nothing in particular — N​VIDIA's own sentence sorts them: Super VL and Bolt "cover the full range of agent deployments, from large enterprise servers down to edge devices and local hardware." N​emotron 3.5 Bolt E4B 4GB, a 4-billion-parameter model for conversational and physical agents, arrives on 23 October.

Second, the compression story, which is the genuinely novel engineering claim here. Bolt is not a model that was trained small and then quantised. It is N​emotron 3.5 Super VL — 120B total, 12B active — put through Star Elastic, N​VIDIA's router-guided architecture search, under a hard constraint of fitting below 24GB of memory at batch size one and 128K of context. Expert top-K came down from 22 in the parent to 9 in Bolt, the MoE intermediate widths were searched per layer, and then the compressed network was quantised and put through quantisation-aware distillation to claw accuracy back. N​VIDIA's stated conclusion is the interesting part and it is a real position, not a boast: for a fixed memory budget, compressing a large model down beats training a small one from scratch. That is a claim you can test, and nobody outside N​VIDIA has.

Third, two leaderboard placements stated by the vendor: #1 open model in its class on PinchBench, and #2 open model on BrowseComp. Both are N​VIDIA's characterisation of its own model against a class it defined.

What the announcement does not add is a served endpoint. There is no hosted API, no per-token price, and no way to call N​emotron 3.5 Bolt without acquiring the hardware it runs on. Artificial Analysis, which has scored the model, lists its speed and cost fields as unavailable for exactly that reason: no publicly available API to measure.

The part nobody outside N​VIDIA has run

This is where the article has to slow down, because the gap between what has been measured and what has been asserted is unusually wide for a model this size.

On the measured side there is one number, and it is a good one: Artificial Analysis puts N​emotron 3.5 Bolt 74B-A9B at 21 on its Intelligence Index, which places it fourth out of 67 models in its comparison class. That is an independent measurement of the actual 2-bit checkpoint, not of a BF16 reference build, and it is the single most useful figure available on this model today.

On the asserted side there is everything else. The model card carries a thirteen-benchmark table with four columns — Qwen3.6-35B-A3B in BF16, Gemma 4 26B-A4B in BF16, N​emotron 3.5 Lightning 30B-A3B in BF16, and Bolt in IQ2 — and every row of it was produced by N​VIDIA, including the three baseline columns, which N​VIDIA selected because they fit in 24GB with 4-bit quantisation. Bolt is the bolded winner in ten rows and is not the winner in three: AA-Omniscience accuracy, where Gemma 4 26B-A4B scores 22.17 against Bolt's 16.90; AA-Omniscience non-hallucination, where N​emotron 3.5 Lightning 30B-A3B scores 68.81 against Bolt's 61.27; and PinchBench, where Qwen3.6-35B-A3B scores 88.07 against Bolt's 87.82. Publishing the three rows you lose is a point in the vendor's favour and does not change what the table is: unreproduced.

The speed numbers deserve their own paragraph because they cut against the marketing. N​VIDIA's own measurement table, taken with llama.cpp on Windows 11 ARM64 on a Surface Laptop Ultra with 128GB, shows Bolt at 83.99 tokens per second on AIPerf SPEED-Bench-Coding — behind Qwen3.6-35B-A3B at 96.63 and Gemma 4 26B-A4B at 100.56. On the same page, end-to-end replay duration across realistic agent trajectories is 600.80 seconds for Bolt against 453.97 for Qwen3.6 and 480.68 for G​emma. The pitch is intelligence per gigabyte, not speed, and the vendor's own table is consistent with that pitch rather than with a claim of being fastest.

So the shape of the evidence is: one independent composite score that is respectable, one vendor benchmark table nobody has reproduced, one vendor speed table on a laptop that nobody outside N​VIDIA has run, and zero independent latency measurements. Treat any article that reports Bolt's PinchBench figure without saying who produced it as unreliable, including any article that agrees with this one.

What arrived alongside it

A model release is rarely just the weights, and Bolt's supporting cast is larger than usual for something with no API.

The LiLiCorr speculative drafter ships with it — 1.13 billion parameters excluding the shared embeddings and output head, trained jointly with the target, and N​VIDIA reports mean acceptance lengths of 4.68 under greedy decoding and 4.25 under probabilistic sampling with seven drafted tokens at concurrency 64 in vLLM. The drafter is a separate downloadable checkpoint, not a runtime flag, and it is the mechanism behind the 84 TPS figure quoted for RTX Spark.

Framework support landed fast. Ollama lists nemotron-3.5-bolt in three tags — a default, a 74B-A9B tag and an IQ2 tag — each at 25GB with a 256K context and text plus image input, with one-line launchers for C​laude Code, OpenCode, Hermes Agent and OpenClaw. Beyond that there is a GGUF build from N​VIDIA, a chain of community quantisations at 4-bit and 8-bit, and MLX conversions that appeared within a day of the announcement. For a checkpoint that runs on a single 24GB device, that ecosystem arriving in forty-eight hours is the real signal that this is a usable artifact rather than a demonstrator.

The practical warning for anyone planning to use it: the 24GB figure is a floor for the weights, not a ceiling for the workload. Images, video frames, conversation history and tool results all share the context budget, and Ollama's own note says usable context depends on available memory. A 25GB download that comfortably fits is not the same as a 256K-token session that comfortably fits.

Where this leaves you, and where it doesn't

If your workload is local and agentic — terminal use, browsing, tool calling, code debugging on a device you own — N​emotron 3.5 Bolt 74B-A9B 24GB is now a defensible choice with one independent score and a set of weights you can inspect. The compression argument behind it is genuinely interesting, the licence permits commercial use, and the tooling showed up within a day.

If your workload needs to be called from code running somewhere you do not own, this model is not an option today, and the honest answer to "how do I use it" is that you run it yourself on N​VIDIA hardware or you wait. We checked our own catalogue rather than assuming, and there is no N​VIDIA model on it — so there is nothing here for us to route, and saying otherwise would be a lie. What OrcaRouter does offer is the other half of a mixed deployment: the models you would call when a request is too hard, too long, or too unfamiliar for a 2-bit local checkpoint, behind one key across 200-plus models at the provider's list price with no markup added, with automatic failover when a provider wobbles. Pairing a local Bolt with a routed API is a sensible architecture. Pretending Bolt is on the routed side is not.

The thing to watch is not the next benchmark table. It is whether anyone outside N​VIDIA publishes a latency run on the 24GB checkpoint on hardware a reader actually owns, and whether a hosted endpoint appears at all. Until one of those happens, the specification is a promise and one index score is the only thing standing behind it.

A result like this is also a reminder that model mistakes are already documented: the AI incidents and agent failures behind models in this class are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.