Skip to content
NLEN
Illustration: OpenAI pipeline found 948 Linux root bugs in one run: what autonomous bug hunting means

NVIDIA Nemotron 3.5 Bolt E4B: a small open model for local assistants, tested on a Mac

By Ivo Donker · · updated — text and test: Grok Bot (AI)

What is Nemotron 3.5 Bolt E4B? It is a small open language model from NVIDIA, post-trained on Google's Gemma 4 E4B. It has 8 billion parameters, of which 4 billion are effective. NVIDIA aims it at local voice and device assistants: short answers, multi-turn conversations and tool calling. On a Mac Studio with an M1 Max it ran at over 40 tokens per second in our test and got ten of ten test tasks right. It works without internet, so what you type stays on your own computer.

We read the model card on Hugging Face and ran the model ourselves in LM Studio. The model was released on October 23, 2026, but went viral on X on November 17.

What it is

NVIDIA took Gemma 4 E4B as its base and trained it further with reinforcement learning (GRPO) on instruction following, multi-step tool use, conversations and structured output. The version with "4GB" in its name is a 4-bit quantization (Q4_K_M) as a GGUF file for llama.cpp. The model reads and writes text only and handles up to 64,000 tokens of context. You can let it "think" before it answers, or turn that off. For faster generation there is a separate drafter model (DFlash) for speculative decoding. The model card lists English, Spanish, French, German, Italian and Japanese, "among others". Dutch is not listed, but it did well in our test. How open such models really are depends on the license; see the debate on open-model licenses.

What NVIDIA claims

NVIDIA compares the model with Gemma 4 E4B and Qwen3.5 4B, all three at Q4_K_M. On IFBench (instruction following) Bolt scores 64.0, against 42.7 for Gemma and 66.7 for Qwen. On Tau2-Bench (agent tasks with tools) it is 52.3 against 38.7 and 61.7. On BFCL-v3 (function calling) 67.3 against 65.0 and 70.6. On long texts Qwen clearly wins: RULER at 64K gives 85.8 for Bolt and 95.4 for Qwen. In short: much better than its own base model at instructions and tools, but not better than Qwen everywhere. These are NVIDIA's own measurements; we did not reproduce them.

Our test on an M1 Max

We ran the model in LM Studio (llama.cpp with Metal) on a Mac Studio with an M1 Max and 32 GB of memory, with 16,384 tokens of context, the recommended settings (temperature 1.0, top_p 0.95) and thinking on, which is the default. Loading took 8 seconds. The model used about 6.2 GB of memory, so we did not see the "4GB" from its name in this setup.

Quality was strong for its size. A Dutch summary in exactly two sentences was right, though it dropped the detail about local servers. A Python palindrome function was correct. In a broken average function it found both bugs: the first element was skipped and an empty list made it crash. A logic puzzle and a time calculation (departure 14:47, 2 hours 38 minutes, 12 minutes of delay: 17:37) went well. From a business sentence it pulled clean JSON with an ISO date and the amount as a number. Asked for a timer and the weather, it called both tools at once with the right arguments. As a router it assigned four tasks correctly to four agents. Translations Dutch to English and back sounded natural, but it offered three versions nobody asked for. On the long 7,230-token text it kept thinking so long that it did not reach an answer within 300 tokens.

What it is good for, and what not

Good: local assistants and chatbots that need to respond fast, call tools, route requests to the right agent, extract data from text and write short summaries, also offline and with privacy-sensitive text. Less good: long documents, complex coding jobs and questions that need a lot of factual knowledge. For those, larger models or Qwen3.5 4B (for long context) are a better choice. The model cannot process images or audio.

Running it yourself in LM Studio

Search LM Studio for "Nemotron 3.5 Bolt E4B" or download the GGUF file from Hugging Face. You can also load it from the terminal with lms load nvidia-nemotron-3.5-bolt-e4b-4gb --context-length 16384; the model name may differ on your machine. LM Studio then serves it through the OpenAI-compatible API at http://localhost:1234/v1. 16 GB of memory leaves plenty of room.

License

The model is covered by the OpenMDW License 1.1 and may be used commercially and non-commercially. The underlying Gemma 4 model is covered by Apache 2.0.

What we did not check

We did not re-measure NVIDIA's benchmarks. Our test is small (ten short tasks and one long one), and answers vary because of the temperature of 1.0. We did not test turning thinking off or the DFlash drafter. The post on X calls the model new; according to the model card it has been online since October 23, 2026.

Tested and checked on November 17, 2026.

This article was written with AI (Grok Bot). The test figures come from our own test on a Mac Studio.

Sources