Nemotron 3.5 Bolt: what a 74B model in 24GB really means

A large model, cut to a specific budget

NVIDIA released Nemotron 3.5 Bolt on October 15, alongside Super VL. The product name is unusually informative: 74B is its total parameter count, A9B describes the active count, and 24GB is the deployment target. Each number answers a different engineering question. NVIDIA release announcement (source)

The full parameter count describes the model's stored capacity. The active count describes the subset used to process a token. Neither number, on its own, tells you how much free memory your computer needs.

Bolt starts with a larger parent model, compresses its architecture, and recovers capability through further training. The resulting checkpoint then uses low-precision storage. This is a more deliberate process than downloading a large model and choosing a smaller quantization file after the fact.

For a buyer, the attraction is straightforward. If a local model can complete the job well enough, private documents and tool outputs can stay within infrastructure the buyer controls. That option becomes more practical when the model leaves room for the rest of the application. It still requires an evaluation of the complete agent, including its tools and network access.

24GB is a configuration, not a promise about every workload

There is a tempting piece of arithmetic here. Store 74 billion parameters at two bits each and the weights occupy 18.5 billion bytes, or 18.5 GB in decimal units. That explains how the headline can be plausible. It is not a measurement of Bolt's actual footprint.

Real inference also needs runtime state. The precision varies across parts of the model, and longer inputs consume additional memory. Images, concurrent requests, and the serving implementation all change the budget. A computer with 24GB installed is also different from a process with 24GB available.

That distinction matters most for agents. A chat prompt may be short. A coding agent accumulates repository excerpts, terminal output, failed patches, and previous tool calls. The workload that passed a quick demonstration can become a different memory problem after forty steps.

Plan around the input your agent will reach during an ordinary difficult run. Record peak memory, time to first token, and the point at which the runtime starts offloading or rejecting requests. Test the browser or editor you intend to leave open beside it.

The speed table is more useful than the speed headline

A good deployment comparison needs at least two clocks. Output speed measures one part of generation. End-to-end duration also captures the time spent consuming prompts, waiting for tools, and moving through repeated model calls.

Bolt is a viable point in this comparison, but two alternatives finish the replay sooner. The table also compares serving configurations, so the result belongs to the model, quantization, runtime, and decoding setup together. A replay duration should not be mistaken for proof that every model solved an equally difficult task equally well.

Separately, Artificial Analysis reports a launch Intelligence Index score of about 21 for Bolt and about 26 for Super VL. That establishes a capability tradeoff with the larger parent. It does not tell us whether a particular repository task will succeed locally. Artificial Analysis launch evaluation (source)

The useful purchase test is completed work under a memory limit. Keep the task and success criteria fixed, then measure success rate, elapsed time, and resource use. A fast failed patch is still a failed patch.

Choose the software path before buying the hardware

The model name is not a compatibility list. In particular, the existence of GGUF weights does not prove that every application supporting GGUF already supports this architecture and its low-bit format. Check the exact runtime release, rather than the application logo.

Start with the documented recipe. Establish a baseline with a short input and one request, then expand to your real context, media, and tool loop. Change one setting at a time. Otherwise a failed deployment tells you little about whether the problem is memory, model support, or an optimization that the runtime has not implemented.

Use our local-model guide to build a wider shortlist. The Ollama guide is useful for readers prioritizing installation simplicity, but Bolt's published weights should not be treated as a claim of turnkey Ollama support.

The next local-model race is about the whole computer

Parameter count made a convenient shorthand when local model selection mostly meant finding a checkpoint that would load. Agents make that shortcut less useful. The model must read enough context, call tools reliably, and leave resources for the applications it operates.

Bolt makes the memory constraint part of model design. That is the development worth following. A system designed for a particular budget can be preferable to a higher-scoring model that regularly pushes the machine into offload or forces the agent to discard important context.

There is also a strategic consequence. NVIDIA can work on the model and the execution path together. For users, that may produce a better local experience. It also means the performance claim has to travel with its software and hardware assumptions. A downloadable checkpoint and a portable performance result are different things.

For local agents to earn a permanent place on a developer's machine, they need to be convenient on an ordinary busy day. That means finishing useful work while the editor, browser, and tests are still running. Bolt should be judged against that standard.