Local model guide · August 10, 2026 · 14 min read

How to Run Muse Glimmer Locally

Meta released Muse Glimmer this morning: a 30B dense, Apache 2.0, image-capable agent model built for one purpose, running always-on agents on a single consumer GPU or a Mac. It is Meta's first open-weight release since the Llama era ended, it ships with its own speculative-decoding drafter and official 4-bit builds, and the whole design is an exercise in fitting a competent agent into 24GB. This guide has the exact numbers: every GGUF size, the VRAM envelope math with an interactive packer, day-one commands for llama.cpp, Ollama, and Apple Silicon, the chat template and reasoning-strength levels, agent wiring, fine-tuning, and the benchmark table including the rows it loses.

What it is

Muse Glimmer is a 30-billion-parameter dense model from Meta Superintelligence Labs, trained by logit distillation from the closed Muse Spark frontier family, mid-trained on long-context agent traces, and post-trained with SFT, on-policy distillation, and RL across reasoning, coding, and agentic domains. Meta's framing is specific: not a chat model, an agent model, optimized for "always-on local agent workflows," tool use, failure recovery, and multi-step tasks, with a frozen ~1.8B perception encoder so it can read screenshots, charts, and documents. Knowledge cutoff is January 4, 2026.

The release matters beyond the weights. Meta went closed in April 2026 when Muse Spark 1.0 ended the Llama open-weight era; Glimmer reverses that, and Meta says open weights for Muse Spark 1.2 itself follow in the coming weeks. The launch was coordinated, per Meta's technical blog: llama.cpp support merged day-zero with official GGUF quants from Meta, transformers 5.15.0 shipped with the architecture, vLLM and SGLang have support landing, Unsloth published dynamic quants and a fine-tuning path, and AMD published same-morning enablement for Ryzen AI Max and Radeon.

The architecture, briefly

The config is worth thirty seconds because every number below feeds the memory math, and if you have read the transformer internals guide, every row will look familiar:

The design reads like a checklist for one goal: minimize everything that competes with weights for VRAM. GQA at 16:1 and the sliding-window layers exist so the KV cache stays negligible, which is what lets a 30B model, a vision encoder, and a speculative drafter share one consumer GPU.

The 24GB envelope

Four things must fit in memory at once: the quantized weights, the KV cache, the vision encoder (only if you want images), and the DFlash drafter (only if you want the speedup). The KV arithmetic is the surprise. The 13 full-attention layers cost 2 × 13 × 2 KV heads × 128 dims × 2 bytes = 13,312 bytes per token, about 1.74GB at the full 131,072-token context. The 39 sliding layers are capped at their 2,048-token window: a fixed 82MB no matter how long the context grows. Total KV at maximum context: under 2GB. That is why the envelope works. Pack it yourself:

Pick a quant

Meta did something unusual: it shipped its own quants, named K-Quant, in two calibrated tiers, and published the quality cost of each. K-Quant-Dynamic costs 0.2% average accuracy across their 15-benchmark suite and targets 32GB; K-Quant-17GB costs 1.0% and targets 24GB. Unsloth's dynamic UD family extends the range in both directions. Exact file sizes:

Two companion files matter regardless of tier: mmproj-kquant.gguf (1.40GB) is the quantized perception encoder, required for any image input, and dflash-kquant.gguf (1.63GB) is the drafter. The main GGUF alone is a text-only model.

Run with llama.cpp

Support merged on release day and Meta ships official GGUFs, so the fastest path is pulling straight from Hugging Face. Meta's card recommends temperature 1.0, top-p 0.95, top-k 64, and Unsloth's commands match:

./llama.cpp/llama-cli \
  -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 64 \
  -ngl 99 -c 32768

Swap the repo for the official quant with -hf meta-models/Muse-Glimmer-30B-GGUF and the file of your tier. For image input, download and load the projector alongside:

hf download unsloth/Muse-Glimmer-30B-GGUF \
  --include "*UD-Q4_K_XL*" --include "*mmproj*"

./llama.cpp/llama-server \
  -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
  --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
  --jinja -ngl 99 -c 32768 \
  --host 127.0.0.1 --port 8080

--jinja matters more than usual here: the chat template is elaborate (next section) and hand-formatting it will silently break tool calls. For the drafter speedup, the repo ships dflash-kquant.gguf; llama.cpp's speculative flags (--model-draft) are the intended slot, but DFlash is a block-diffusion drafter rather than a plain small LM, so verify current llama.cpp support in the PR discussion before assuming the full 3.1x applies outside Meta's own harness.

Ollama and LM Studio

Ollama added support in 0.32.7 the same morning, with a day-one wrinkle: the first manifest push blocked downloads (issue #17645). Once resolved on their registry:

ollama run muse-glimmer

Until then, Ollama runs any local GGUF via a Modelfile pointing at the downloaded file. LM Studio is on Meta's day-zero partner list; search for "Muse Glimmer" in the model browser, take a 4-bit tier for 32GB Macs, and load the vision variant if you want screenshots understood.

On a Mac

Two paths. The general one is GGUF via LM Studio or llama.cpp with Metal, exactly as above; count unified memory at about three quarters usable. The interesting one is Meta's own: the ExecuTorch-PTE repo ships 16 prebuilt variants, every combination of the two K-Quants, text or text+image, with or without the DFlash drafter, for Metal and for CUDA. These are the builds behind Meta's Mac numbers: 23.7 tok/s plain and 37.8 with the drafter on an M4 Max, 50.2 on an M5 Max. MLX conversions are on the partner list as "coming days" and will likely become the standard Mac path; check the mlx-community search before assuming.

vLLM and SGLang

Both are in the day-zero wave with support landing rather than landed: vLLM's model PR plus a recipes entry, SGLang's native support PR with a cookbook that installs from the branch until a release includes it. On a 24GB card there is little reason to prefer them for single-user use; they matter when you serve Glimmer to a team or want paged KV and continuous batching under an agent fleet. Unsloth also publishes an NVFP4 build for Blackwell GPUs claiming 1.45x over BF16, but it is flagged work-in-progress and non-functional at time of writing; treat it as a preview.

The agent part

This is the model's identity, and it shows up in the chat template. Three things you should know before wiring it into anything:

The template is its own format. Roles are framed as <|start|>role<|message|>...<|eot|>, the assistant writes to channels (to=self for reasoning, to=user for the reply, to=<tool> for calls), and tool calls are emitted as structured XML blocks (<atem:function_calls> with named invokes and parameters) rather than JSON strings; the template errors if tool arguments arrive as JSON text instead of a dict. If you use --jinja in llama.cpp or a serving engine that applies the bundled template, all of this is handled; if you are assembling prompts by hand, read chat_template.jinja in the repo first.

Reasoning strength is a system-prompt dial. The template auto-injects Reasoning strength: high by default; the levels are low, medium, high, and xhigh. Set it explicitly for agent workloads: low for quick tool dispatch, high or xhigh when the task needs planning. This replaces the think/no-think toggles you may know from other model families.

Scaffolds are expected, not optional. Meta's benchmarks run the model inside agent scaffolds, and the card names OpenClaw and Hermes Agent as tested pairings. Practically: serve an OpenAI-compatible endpoint and point your agent at it,

./llama.cpp/llama-server -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
  --jinja -ngl 99 -c 32768 --port 8080

# then, in your agent's config:
#   base_url: http://127.0.0.1:8080/v1
#   model: anything   (llama-server ignores the name)

and give it real tools. The launch demo is the honest picture of intended use: discover a Home Assistant instance on the network, query its APIs, write a dashboard, deploy a local server to verify. Multi-step, tool-heavy, self-checking.

Speed and the DFlash drafter

Muse Glimmer ships with its own speculative decoder, and it is the most technically interesting part of the release. DFlash is a ~2.5B block-diffusion drafter: instead of drafting tokens one by one, it proposes a block of 16 tokens in a single forward pass, reading hidden features tapped from five of the target model's layers, and the main model verifies the block in parallel. Accepted tokens are free speed; rejected ones are corrected by the main model, so output quality is mathematically unchanged, the same guarantee as any speculative decoding (mechanics in the vLLM guide's draft-and-verify figure).

Meta's measurements: K-Quant-17GB weights, quantized drafter, batch 1, greedy decode; Mac via ExecuTorch, RTX via llama.cpp.

The catch to watch: those are batch-one greedy numbers in Meta's own harness, and one early skeptic on X put it well, "I'd still time a full tool loop before calling it always-on." Agentic use interleaves prefill, tool waits, and decode; expect real end-to-end sessions to feel slower than the headline number, and measure your own loop.

Benchmarks, honestly

Meta's card compares against Gemma4-31B and Qwen3.6-27B, both in thinking modes, across 24 rows; the numbers are Meta's own, one day old, unreplicated. Glimmer posts the best score on 12 of 24, beats Gemma on 19, and beats Qwen on 14. The pattern is legible: it is strongest exactly where it was aimed, and ordinary elsewhere. Selected rows, wins and losses:

Read it plainly: for tool-driven agent work and long-context retrieval, the claimed lead is large. For terminal-heavy coding and computer use, Qwen3.6-27B still looks stronger on Meta's own table, which is about as credible as a vendor loss gets. Every number is day-one and vendor-reported; independent replications will land within the week and are worth waiting for before you rebase a product on it.

Fine-tune with Unsloth

Unsloth published day-one support (their kernels route Glimmer's sliding-window layers through a banded attention path). Their training page recommends LoRA or QLoRA: r=16, alpha=16, dropout 0 as the base recipe (r=32 for harder agentic behaviors), learning rate 2e-4, one to three epochs, BF16, starting at max_seq_length 4096 with packing, and QLoRA with load_in_4bit for consumer GPUs. Keep the perception encoder frozen, which is also how Meta shipped it. The obvious fine-tune targets for an agent model: your own tool schemas, your team's workflows, and domain documents the 2026-01-04 cutoff misses.

Rough edges, day one

• Packaged llama.cpp is too old: support merged the morning of release, so brew and distro builds fail with unknown model architecture: 'muse-glimmer' (verified here on brew build 10330). Build from master, or wait for the next tagged release to reach your package manager.
• Images silently missing: the main GGUF is text-only; without the mmproj file loaded, image input is ignored or errors. Load both.
• Ollama manifest: 0.32.7 added support but the registry manifest initially blocked downloads (#17645); use a local GGUF + Modelfile if it persists.
• Context claims: the config says 131,072 with no rope-scaling entry. Unsloth's page mentions "up to 262,144"; nothing in the shipped config supports that today. Plan around 131K.
• NVFP4: marked work-in-progress and currently non-functional in Unsloth's own docs. Skip for now.
• Hand-built prompts: the template requires tool arguments as structured data, not JSON strings, and uses channel markers a generic ChatML assembler will not produce. Use --jinja or the served template.
• Doc churn: Meta's developer docs portal was intermittently erroring on launch morning, and early third-party docs contain copy-paste artifacts (references to a "26B MoE" variant; Glimmer is 30B dense). Trust the model card and config first.

FAQ

Is this Llama 5?

No. The Llama line ended with Llama 4; Muse is the Meta Superintelligence Labs family. Glimmer is the small open sibling, distilled from the closed Muse Spark frontier models, and Meta has said open Muse Spark 1.2 weights follow within weeks.

What does it replace in a local stack?

The 24-32GB agent slot currently held by Qwen3.6-27B and Gemma4-31B. If your workload is tool-calling agents, MCP servers, or long-context retrieval, the claimed numbers favor switching; if it is terminal coding or computer use, they favor staying.

Can it see my screen?

It reads images, screenshots, charts, and documents through its perception encoder (load the mmproj file). It is not a computer-use model out of the box, and its OSWorld score trails Qwen's; pair it with an agent scaffold that provides the click-and-type machinery.

Why is the KV cache so small?

Two KV heads against 32 query heads (16:1 grouped-query attention) plus sliding windows on three quarters of the layers. Under 2GB at full 131K context, computed from the config in the envelope figure above. The design goal was fitting agents into 24GB, and the cache is where they found the room. The mechanics behind those words are in the transformer guide.

Where do I start if I have 16GB?

UD-Q2_K_XL (12.44GB), text-only, context at 8-16K, drafter off. It fits and runs; expect the 2-bit quality tax. The model's stated floor is around 18GB of combined memory for comfortable use.