How much memory does the Ripple-65B cache need?

Only the 16 global layers keep a cache that grows with the context. Each stores 8 KV heads by 128 values by two tensors by two bytes, 4,096 bytes per token, so the 16 layers cost 65,536 bytes per token in BF16. That is 8.6 GB at 131,072 tokens and 68.7 GB at 1,048,576. The 64 sliding layers hold a fixed 1,024 tokens each, 256 MiB in total.

Does vLLM 0.33.0 support Ripple-65B?

No. Support merged into the main branch on November 5, 2026 in pull request 65745, three days after the 0.33.0 release. Install a nightly wheel built after commit 03bd3c6, or use the vllm/vllm-openai:nightly image, until the next release.

Local model guide · November 6, 2026 · 19 min read

How to Run Ripple-65B

Q: How fast does Ripple-65B decode on a DGX Spark?

The NVFP4 checkpoint reads 56.19 GB of weights for every token, and a DGX Spark moves 273 GB/s. That is a ceiling of 4.8 tokens a second at a short context and 4.2 at 131,072 tokens. The DFlash2 drafter changes the arithmetic: one target pass verifies a 16-token block, so the output rate is the accepted tokens per pass times the ceiling. At the 7.34 tokens a pass that vLLM measured on GSM8K, the Spark ceiling is near 31 tokens a second.

Q: Which llama.cpp build runs Ripple-65B?

The ricursive architecture merged into llama.cpp on November 5, 2026 in pull request 31692. Release b12167, from November 6, is the first that carries it. Older builds stop at load with an unknown architecture error.

Ricursive Intelligence published Ripple-65B Preview on November 4, 2026: a dense 65B transformer trained from scratch on 12 trillion tokens, under Apache 2.0. The card says it fits a DGX Spark, and the 57.2 GB NVFP4 checkpoint does. A mixture-of-experts model reads a few experts per token. Ripple is dense, so it reads all of its weights for every token, 56.2 GB in the NVFP4 build. At 273 GB/s a DGX Spark cannot decode faster than 4.8 tokens a second at a short context. Ricursive answers with a 16-token DFlash2 drafter that turns one weight read into several tokens. The sections below derive the ceiling and the cache from config.json and the file sizes. Then they give the commands for llama.cpp, the Spark script, vLLM, and SGLang on two B200s.

Machine · Ripple-65B, the dense press

Load a build on a machine, prefill a 4,096-token prompt, decode one token, let the DFlash2 drafter propose a 16-token block, and grow the context to 128K and then 1M. The 16 cells are the 80 layers in groups of five: four window layers in front and one global layer at the back. Every cell lights on every token, because the model is dense. Only the global layers feed the tank, and the cups stop at 1,024 tokens. Bytes, memory, and tokens per second come from config.json, the file sizes, and vendor bandwidth. Tokens per pass is a knob, because acceptance depends on your prompts. Real decode lands below the ceiling.

Architecture map · Ripple-65B Previewconfig.json

Eighty dense layers. Sixteen attend to the whole context with no positional encoding. The other 64 see a 1,024-token window with RoPE, four of them before every global layer.

80 layers, first to last, one cell each

16 global attention, no RoPE64 sliding window (1,024 tokens)

Dense model: all 64,782,689,360 parameters run for every token. There is no router and no expert to skip.

hidden 5,120 · 80 query heads · 8 KV heads · head dim 128 · MLP 40,960 · vocabulary 100,352 · positions 2,097,152

KV cache: 16 global layers × 8 KV heads × 128 × 2 × 2 bytes = 65,536 bytes per token in BF16, plus a fixed 256 MiB for the 64 windows

Source: ricursiveAI/Ripple-65B-Preview config.json

65B dense · all active positions 2,097,152 license Apache-2.0 llama.cpp b12167 or newer

What Ricursive released

Ricursive Intelligence builds an AI chip design platform, and Ripple is the general-purpose variant of the model behind it. On November 4, 2026 the lab published three repositories on Hugging Face beside a base checkpoint from October 14. The instruct model in BF16 is 129.6 GB across 16 safetensors shards. The NVFP4 build is 57.2 GB, and its card calls it experimental. The DFlash2 drafter is 8.1 GB. Every repository carries the Apache 2.0 license file.

The card gives the training recipe in one table. Pretraining ran 877,500 steps over 12 trillion tokens at an 8K sequence length. Midtraining added 200 billion tokens at 64K. A long-context stage ran 6.5 billion tokens at 2M positions, then SFT at 128K, long-context SFT at 2M, and 700 steps of RL at 128K. Ricursive reports 80.52 on Terminal-Bench 2.1 and calls it the best score among US open-weight models. Those are its own runs, and the card itself says agentic scores shift with the harness.

The launch post says a team that grew from one to five people chose a dense transformer over a mixture of experts. The reason was a simple training stack. It reports 40% MFU with plain sharding, no custom kernels, and no quantization. The post puts the whole project at 32 million dollars, a number I cannot check. The cost of that choice appears at decode time, on your hardware.

The architecture, from config.json

The values below come from the repository's config.json and the tensor names in the safetensors index. The parameter total is the Hugging Face safetensors metadata, read on November 6, 2026.

Field	Ripple-65B Preview
Architecture	`RicursiveForCausalLM`, model type `ricursive`
Parameters (BF16)	64,782,689,360, all active for every token
Layers	80, dense MLP in each
Attention layers	16 global, 64 sliding window of 1,024 tokens, in a four-to-one pattern
Positional encoding	RoPE with theta 10,000 on the window layers, none on the 16 global layers
Hidden size	5,120
Query heads / KV heads	80 / 8, head dimension 128, per-head QK RMSNorm
MLP	SwiGLU, intermediate size 40,960, inputs clamped at 20
Attention gate	`use_gated_attention`: one sigmoid gate tensor per layer on the attention output
Value residual	`use_valres`: one learned scalar per layer mixes the token embedding back into V
Embedding scale	71.554, the square root of 5,120
max_position_embeddings	2,097,152
Vocabulary	100,352, input and output embeddings not tied
BF16 checkpoint	129.6 GB in 16 shards

The layer list has one pattern. Four sliding layers, then one global layer, sixteen times. Each sliding layer sees the current token and the 1,023 before it. The 16 global layers see the whole context and carry no positional encoding at all, which the post calls global NoPE. A layer with no rotation angle has nothing that runs out of range when the context grows. That is how the position limit reaches 2,097,152 without RoPE scaling.

The shape is thin and deep. The hidden size is 5,120 and there are 80 layers. The 80 query heads at 128 dimensions make a 10,240-wide attention, twice the hidden size. The card calls that overprovisioning the heads and says the short window and the four-to-one ratio compensate for it. A layer with 40 heads would hold 89.1 million attention parameters, and Ripple's holds 167.8 million. Over 80 layers the extra heads add 6.3 billion parameters, 12.6 GB in BF16.

Four more flags come from the launch post's list of additions. The gate tensor g_proj is a sigmoid on the attention output, kept in full precision. The scalar value_res_lambda blends the scaled token embedding into each layer's V input. The clamp at 20 bounds the SwiGLU inputs. And each layer carries four norms, before and after attention and before and after the MLP, the layout the post credits to Gemma 3N.

The tensor shapes give the count. Attention in one layer is 167,772,160 parameters. The query, output, and gate projections are 5,120 by 10,240. The key and value projections are 5,120 by 1,024. The MLP is 629,145,600, three matrices of 5,120 by 40,960. The norms and the scalar add 20,737. Eighty layers, two embeddings of 513,802,240, and a final norm sum to 64,782,689,360, the number in the metadata. Attention is 20.7% of the model and the MLPs are 77.7%.

One decode step in bytes

At batch size one, a decode step reads every weight it uses once. A mixture-of-experts model skips most of its experts, so the question there is which tensors a token touches. Ripple has no router and no expert to skip. The one tensor it reads in part is the input embedding, one 5,120-value row per token. So the BF16 model reads two bytes for each of the other 64,268,887,120 parameters: 128.54 GB per token.

The NVFP4 build changes only the MLPs. Its quantization_config lists one group for Linear layers: 4-bit floats in groups of 16 with an FP8 scale per group. Its ignore list names lm_head, the input embedding, and all 80 self_attn modules. So the 240 MLP matrices hold 50.3 billion weights at 4.5 bits, 28.31 GB, while attention stays in BF16 at 26.84 GB. With the 1.03 GB output head, one token reads 56.19 GB. Attention is now 47.8% of the bytes, which is why the file is 57.2 GB and not 35. The GGUF files quantize attention too. AtomicChat's AD-Q4_K_M reads 40.15 GB per token, and its AD-IQ3_XXS reads 27.69 GB, from the tensor types in their headers.

Divide bandwidth by bytes and you have the ceiling. NVIDIA's DGX Spark page gives 273 GB/s and 128 GB of unified memory. At a short context the NVFP4 build tops out at 4.8 tokens a second, the 4-bit GGUF at 6.7, and the 3-bit GGUF at 9.7. A B200 moves 8 TB/s, from the DGX B200 specification of 64 TB/s across eight GPUs. That sets 141 tokens a second for NVFP4 and 62 for BF16 on one GPU. Real runs land below. The author of the SGLang port measured 66 tokens a second with NVFP4 on one B200 and 39 with BF16. That is 47% and 63% of those ceilings. Ricursive's 250 tokens a second on two B200s is at batch size eight, so eight requests share each weight read.

The cache adds to every step. At 131,072 tokens the 16 global layers hold 8.59 GB, and a decode step reads all of it. On the Spark that raises the bytes per token to 65.0 GB and lowers the ceiling to 4.2. The machine above the contents list has knobs for the machine, the build, the context, and the tokens per pass. Every number it prints comes from the same functions as this section.

The cache: 16 tanks and 64 cups

A global layer stores a key and a value for every token. That is 8 KV heads by 128 values by two tensors by two bytes, 4,096 bytes per token. Sixteen such layers cost 65,536 bytes per token, 64 KiB. That is 0.27 GB at a 4,096-token prompt and 2.15 GB at 32,768. At 131,072 tokens it is 8.59 GB, and at 1,048,576 it is 68.7 GB. The full 2,097,152 positions would hold 137.4 GB.

A sliding layer never stores more than its window. Sixty-four layers by 1,024 tokens by 4,096 bytes is 268 MB, and it stops there. Had Ricursive kept all 80 layers global, the cache would be five times larger. That is 42.9 GB at 131,072 tokens and 343.6 GB at 1,048,576. That pattern is why the cache of a 65B dense model still fits one box at a long context.

The two vendor scripts show the limit in memory. The Spark script sets a context of 131,072 tokens and a pool of 139,264. With the 57.2 GB NVFP4 weights and the 8.1 GB drafter, that cache brings the total to 74.7 GB, inside the 128 GB. A 1,048,576-token cache would bring it to 134.3 GB, more than the box holds. The two-B200 launcher sets a context of 1,048,576 at tensor parallel two. Each GPU then holds 64.8 GB of BF16 weights, 4.0 GB of drafter, and 34.5 GB of cache, 103.3 GB of its 180 GB. The drafter adds a second limit on the Spark. Its config allows 131,072 positions, and the B200 launcher sets SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 to extend that cache, which the Spark script does not.

Fig. 1 · cups and tanks

All 80 layers in config.json order. The 64 window layers are cups that fill to 1,024 tokens and then close. The 16 global layers are tanks that keep growing, drawn on a log scale. Turn the context dial and the ledger prices the cache in bytes, then checks it against the memory the two vendor scripts leave for it.

window cup, 1,024 tokensglobal tank, grows with contextcup closed

layer 0four cups, one tank, sixteen timeslayer 79

context = 4,096 tokens

Bytes per token: 8 KV heads × 128 × 2 × 2 bytes per global layer, from config.json. The Spark row uses the NVFP4 weights, the 8.08 GB drafter, and the 0.78 memory fraction in spark.py. The B200 row uses the BF16 weights, the drafter, and the 0.80 fraction in serve.py, over two 180 GB GPUs. Activation buffers are not counted.

The DFlash2 drafter

The drafter is its own repository: DFlash2DraftModel, 4,037,752,064 parameters in BF16, 8.08 GB. Its config.json describes five layers at the target's width: four sliding layers with a 4,096-token window and one global layer with no RoPE. The block size is 16. The candidate selector keeps 32 per position with a rank of 256, and the dynamic convolution has kernel 4 and group 16. The field target_layer_ids lists 1, 20, 39, 58, and 77. Those are the five target layers whose hidden states the drafter reads. It has no embedding and no output head of its own, and it borrows both from the target.

The drafter reads its 8.08 GB once and proposes a block of 16 tokens. The target then verifies the whole block in one pass, one read of its 56.19 GB. The tokens produced per pass is the acceptance length: the accepted draft tokens plus the target's own next token. So the output rate is the acceptance length times the bandwidth, divided by the bytes of target, drafter, and cache. The drafter is a net gain once the acceptance length passes 1.14. At 7 tokens a pass on a Spark, the NVFP4 build reaches about 29 tokens a second at a short context. At 131,072 tokens it reaches 26.

One measured number exists for this drafter. In the vLLM pull request, the author measured a mean acceptance length of 7.34 on GSM8K with 15 draft tokens. That run honored the draft's NoPE layer. With that layer rotated, it was 4.98. GSM8K is arithmetic, and other prompts can accept fewer tokens, so measure acceptance on your own prompts. The launch flags also disagree on the block. The card and the two-B200 launcher ask for 16 draft tokens, the Spark script asks for 32, and the vLLM recipe asks for 7. The launcher keeps a 16,384-token draft window, because the drafter was trained on 16K rows. The drafter card says its validation used the ripple-sglang runtime on eight H100s with the RL-step-500 checkpoint. It adds that this does not establish compatibility with other revisions.

Every build and its size

File sizes come from the Hugging Face tree API on November 6, 2026, in decimal gigabytes. AtomicChat published its GGUF files on November 4 and wrote the llama.cpp port. Its card measures every file against the BF16 original with llama-perplexity over 87 chunks of 4,096 tokens. It reports the mean KL divergence and how often the file picks the same top token. Those are the quantizer's own numbers.

Build	Size	Same top token	Repository
`BF16` safetensors	129.6 GB	reference	ricursiveAI/Ripple-65B-Preview
`NVFP4` safetensors, MLPs only	57.2 GB	not measured	ricursiveAI/Ripple-65B-Preview-NVFP4
`DFlash2` drafter	8.1 GB		ricursiveAI/Ripple-65B-Preview-DFlash2
`Q8_0` GGUF, 2 parts	68.8 GB	99.0%	AtomicChat/Ripple-65B-Preview-GGUF
`AD-Q6_K`, 2 parts	57.7 GB	98.7%	AtomicChat
`AD-Q5_K_M`	47.9 GB	97.3%	AtomicChat
`AD-Q4_K_M`	40.4 GB	95.9%	AtomicChat
`AD-IQ4_XS`	38.7 GB	95.6%	AtomicChat
`Q4_K_M`, no importance matrix	38.8 GB	93.0%	AtomicChat
`AD-IQ3_S`	32.4 GB	93.4%	AtomicChat
`AD-IQ3_XXS`	28.0 GB	91.2%	AtomicChat
`AD-IQ2_S` / `AD-IQ2_XS` / `AD-IQ2_XXS`	25.3 / 22.1 / 20.0 GB	88.5% / 84.2% / 80.8%	AtomicChat
`AD-IQ1_M`	18.5 GB	76.8%	AtomicChat, a research file

The AD files use per-tensor layouts chosen with an importance matrix. In the AD-Q4_K_M header the attention output projection is Q6_K, the MLP matrices mix Q4_K and Q5_K, and the input embedding is IQ4_XS. The plain Q4_K_M stays in the repository as the baseline it beats. Below 4 bits the agreement falls fast: 91.2% at 28 GB and 80.8% at 20 GB.

Fig. 2 · the build frontier

Every published file as a dot: bytes on disk across, and how often it picks the same next token as BF16 up. Pick a machine, set the share of its memory an engine can use, and set the context you want. The line is the memory left for weights. The ringed dot is the best file left of it, and the verdict prices it.

machine

usable memory 80% (assumed)

context = 32,768 tokens

Sizes are the Hugging Face tree API. Agreement and mean KLD are AtomicChat's measurements on its 30-language corpus. The usable share is an assumption, because engines and operating systems reserve different amounts. Cache is 65,536 bytes per token plus 268 MB of windows, in f16. The decode ceiling divides the vendor bandwidth by the bytes a token reads: the file minus its embedding table, from the GGUF header, plus the cache. Two cards under llama.cpp's default layer split read in turn, so they do not add bandwidth.

Community conversions for MLX exist, and so do EXL3 and FP8 files. But mlx-lm has no Ripple model file on November 6, so the MLX weights need custom code. On a Mac, run the GGUF through llama.cpp with Metal. The 4-bit file fits a 64 GB machine, and the 2-part AD-Q6_K fits a 96 GB or 128 GB one.

llama.cpp

The ricursive architecture merged into llama.cpp on November 5, 2026 in pull request 31692. Release b12167, from November 6, is the first that carries it. The port needs no new operator. It maps the clamp, the value-residual scalar, the gate, and the embedding scale onto existing pieces. Its author checked the first ten real layers against Ricursive's own modeling code and reports a KL divergence of zero and 100% top-token agreement. An older build stops at load with an unknown architecture error. The chat template with tools and reasoning is inside the file, so pass --jinja. Leave --swa-full unset, so the 64 window layers keep a cache near their 1,024-token window instead of a full-size one.

# 64 GB Mac, or two 24 GB cards: 40.4 GB of weights, 32K of context, f16 cache
llama-server -hf AtomicChat/Ripple-65B-Preview-GGUF -hff Ripple-65B-Preview-AD-Q4_K_M.gguf \
  --alias ripple --jinja -ngl 99 -fa on -c 32768 \
  --host 127.0.0.1 --port 8080

# 32 GB card: 28.0 GB of weights, 16K of context
llama-server -hf AtomicChat/Ripple-65B-Preview-GGUF -hff Ripple-65B-Preview-AD-IQ3_XXS.gguf \
  --alias ripple --jinja -ngl 99 -fa on -c 16384

# measure your own ceiling: prompt speed at 512 tokens, generation speed over 128
llama-bench -m ~/.cache/llama.cpp/AtomicChat_Ripple-65B-Preview-GGUF_Ripple-65B-Preview-AD-Q4_K_M.gguf \
  -p 512 -n 128 -ngl 99 -fa on

The tg128 number from llama-bench is your decode rate. Compare it with the ceiling: your memory bandwidth divided by the file size. The two SGLang runs in the bytes section land at 47% and 63% of their ceilings. That port still runs the Q, K, V, and gate projections as separate GEMMs. The quantizer reports one such run: the plain Q4_K_M on a GH200 generated 40.6 tokens a second and processed the prompt at 309. No drafter is available on this path. The GGUF repository holds no drafter, and the port covers the target model only. The cache costs 64 KiB per token of context in f16. If memory is short, -ctk q8_0 -ctv q8_0 cuts that to 34 KiB. Ricursive publishes no sampling defaults, and generation_config.json has none.

DGX Spark

The NVFP4 repository holds the Spark path in inference/local: a notebook, spark.py, a model file, and the parser. It runs SGLang's offline engine inside the Python process, with no HTTP server, one request at a time. The script refuses to start on anything but a GB10 GPU. It checks that the target is the official NVFP4 export with BF16 attention. It also checks that the drafter is unquantized BF16. Its flags are the ones to copy if you build your own server. The context is 131,072 tokens with a pool of 139,264, and the memory fraction is 0.78. Attention runs on FlashInfer, and the FP4 GEMM on the FlashInfer CUTLASS kernel. The drafter gets 32 draft tokens and a 16,384-token window. The radix cache is off, and CUDA graphs cover one request. The notebook installs SGLang 0.5.19, PyTorch 2.13 for CUDA 13, Transformers 5.12.1, and FlashInfer 0.6.18 into a Python 3.12 environment.

# on the Spark, Python 3.12, CUDA Toolkit 13 installed
python3.12 -m venv ~/.venvs/ripple-spark && source ~/.venvs/ripple-spark/bin/activate
pip install --pre --extra-index-url https://download.pytorch.org/whl/cu130 \
  "sglang==0.5.19" "torch==2.13.0+cu130" "transformers==5.12.1" \
  "flashinfer-python==0.6.18" "ninja==1.13.2" "nest_asyncio==1.6.0" huggingface_hub
hf download ricursiveAI/Ripple-65B-Preview-NVFP4 --local-dir ripple-nvfp4 \
  --include "inference/local/*" chat_template.jinja

# run.py: the three calls the notebook makes
import sys; sys.path.insert(0, "ripple-nvfp4/inference/local")
from spark import Spark
model = Spark("ricursiveAI/Ripple-65B-Preview-NVFP4", "ricursiveAI/Ripple-65B-Preview-DFlash2")
print(model.chat("Write a Python function that merges overlapping intervals.", max_tokens=1024))
model.shutdown()

The first call downloads 65 GB of weights and compiles kernels, so allow time and disk. The complete() method returns the OpenAI-style message with reasoning_content, content, and tool_calls. The context stays at 131,072 tokens for the two reasons from the cache section. A 1M cache does not fit beside the weights. And the drafter's position table stops at 131,072 unless an engine extends it.

vLLM

Ripple merged into vLLM on November 5, 2026 in pull request 65745. The port is a native RicursiveForCausalLM with tensor and pipeline parallel, torch.compile, and CUDA graphs. It adds the ripple reasoning and tool parsers and the DFlash2 draft with its NoPE layer. Release 0.33.0 came out on November 2 and does not contain it. Until the next release you need a nightly wheel built after commit 03bd3c6, or the vllm/vllm-openai:nightly image. The pull request's recipe verifies the NVFP4 build on one B300. It sets the memory floor at 69 GB for NVFP4 and 156 GB for BF16, each file size times 1.2. No --trust-remote-code is needed, because the config class is built in.

# nightly wheel; pip cannot resolve the nightly index, uv can
uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

# NVFP4 on one Blackwell GPU
vllm serve ricursiveAI/Ripple-65B-Preview-NVFP4 --max-model-len 131072 \
  --enable-auto-tool-choice --tool-call-parser ripple --reasoning-parser ripple

# the same with the DFlash2 drafter, 7 draft tokens as in the recipe
vllm serve ricursiveAI/Ripple-65B-Preview-NVFP4 --max-model-len 131072 \
  --enable-auto-tool-choice --tool-call-parser ripple --reasoning-parser ripple \
  --speculative-config '{"method":"dflash","model":"ricursiveAI/Ripple-65B-Preview-DFlash2","num_speculative_tokens":7}'

The pull request reports its own checks on a B300. Teacher-forced logprobs on three 2,200-token prompts agreed with Ricursive's code at 99.5% top-one, and GSM8K five-shot scored 0.916 with and without the drafter. The 7.34 acceptance length in the drafter section comes from the same runs with 15 draft tokens. The author lists the BF16 checkpoint at tensor parallel above one, ROCm, and LoRA as untested.

SGLang on two B200s

Ricursive's own serving path is the launcher in the BF16 repository's inference/server folder. It is a uv script that requires SGLang 0.5.19 and PyTorch 2.13 for CUDA 13. It registers the model class and starts the stock HTTP server with Ripple's defaults. Those defaults are tensor parallel two, BF16 weights and cache, and TensorRT-LLM target attention with a page size of 64. The drafter runs on Triton attention with 16 draft tokens and a 16,384-token window. The context is 1,048,576 tokens, the memory fraction is 0.80, and 16 requests can run at once. The server binds 127.0.0.1:30000. Stock SGLang does not include the model yet. Pull request 48636, opened on November 5, adds it to main with its own parsers and is still open.

hf download ricursiveAI/Ripple-65B-Preview --local-dir ripple-bf16          # 129.6 GB
hf download ricursiveAI/Ripple-65B-Preview-DFlash2 --local-dir ripple-dflash2   # 8.1 GB
TARGET="$PWD/ripple-bf16" DRAFT="$PWD/ripple-dflash2"
export CUDA_HOME=/usr/local/cuda-13.0 PATH="/usr/local/cuda-13.0/bin:$PATH"
cd "$TARGET/inference/server"
CUDA_VISIBLE_DEVICES=0,1 uv run --script serve.py \
  --model-path "$TARGET" --speculative-draft-model-path "$DRAFT"

curl http://127.0.0.1:30000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"ripple","messages":[{"role":"user","content":"Hello"}],"max_tokens":128}'

Keep the repository layout, because the launcher reads chat_template.jinja two folders up. It also sets SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1. That lets the drafter's position cache grow from its 131,072 limit to the 1M context without widening its attention window. For four GPUs the README says to start a second copy on devices 2 and 3 with --port 30001, two replicas of two GPUs each. The launcher refuses any dtype other than BF16 for weights and cache.

Reasoning, tools, sampling

Reasoning. The model decides per turn whether to open a <think> block, and the template has no switch and no effort level. The ripple parsers on vLLM and SGLang split that block into reasoning_content and keep the framing newlines out of both fields. Send reasoning_content back on assistant messages when you replay a conversation. The template writes it into the prompt again unless you pass clear_thinking as true in chat_template_kwargs, and the vendor keeps it by default.

Tools. The template renders the tool list as JSON inside the system message and asks for one JSON object per call inside <tool_call> tags, Hermes style. Tool results go back as tool messages inside a user turn, wrapped in <tool_response>. For tool_choice set to required, set strict to true on the function and additionalProperties to false in its schema. The README reports extra argument fields without them.

Sampling. Ricursive publishes no defaults. The Spark notebook uses temperature 0, and the two engine pull requests ran their GSM8K checks at temperature 0 as well. Reasoning counts toward max_tokens, so give hard tasks a large budget and check finish_reason.

A worked example

Take a 64 GB MacBook Pro with an M4 Max. The 4-bit file is 40.4 GB. At 32,768 tokens the global cache is 2.15 GB and the windows add 0.27 GB. With about 1 GB of compute buffers the total is near 44 GB. By default macOS lets the GPU use about three quarters of unified memory, 48 GB, so it fits with little room. Apple lists 546 GB/s for this chip. Each token reads 40.15 GB of weights and 2.42 GB of cache, so the ceiling is 546 divided by 42.6, or 12.8 tokens a second. Measure first, then serve.

llama-bench -m Ripple-65B-Preview-AD-Q4_K_M.gguf -p 512 -n 128 -ngl 99 -fa on

llama-server -m Ripple-65B-Preview-AD-Q4_K_M.gguf --alias ripple --jinja \
  -ngl 99 -fa on -c 32768 --host 127.0.0.1 --port 8080

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "ripple",
       "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals and include tests."}],
       "temperature": 0, "max_tokens": 16384}'

If tg128 comes back near 7 tokens a second, the run is at 55% of the ceiling. That is normal for a 40 GB read per token. A coding agent that writes 2,000 tokens a turn will wait five minutes per turn at that rate. On a Spark with the drafter at 7 tokens a pass, the ceiling is about 29 tokens a second. On one B200 with vLLM and the same drafter, the ceiling is about 860. Do the arithmetic before the download, because for a dense 65B the machine sets the rate.

Benchmarks, vendor-reported

The card prints 80.5 on Terminal-Bench 2.1 under two harnesses and 79.1 on SWE-bench Pro v2.0 Hard. It prints 76.5 on SWE-bench Verified, 85.2 on GPQA Diamond, and 39.4 on SciCode, its lowest row. The comparison columns mix other labs' card scores with Ricursive's in-house runs, which the card marks in italics. All of it is Ricursive's measurement. The two engine ports add one independent number each. GSM8K lands between 96.3% and 97.1% depending on the build and the harness. NVFP4 sits within 0.3 points of BF16 in the SGLang run.

Failure modes and fixes

llama.cpp reports an unknown model architecture. The build is older than b12167. Update and rebuild.
vLLM 0.33.0 rejects RicursiveForCausalLM. The release predates the merge. Install a nightly wheel with uv, or run the nightly image.
Stock SGLang does not load the model. Use Ricursive's launcher with SGLang 0.5.19, or build the branch from pull request 48636.
The Spark script raises "Run this notebook on a DGX Spark / GB10 GPU". It checks for compute capability 12.1. Use the vLLM or llama.cpp path on other GPUs.
The Spark script raises a ValueError about NVFP4 or the draft. It accepts only the official NVFP4 export with BF16 attention and an unquantized BF16 drafter. Community files fail the check.
Out of memory at a 1M context on the Spark. The global cache alone is 68.7 GB at 1,048,576 tokens, and the weights and drafter are 65.3 GB. Stay at 131,072, or move to two B200s.
Decode stays near 4 tokens a second. That is the ceiling for a 56 GB read at 273 GB/s. Use a smaller GGUF, or an engine path that runs the drafter.
The drafter accepts few tokens. An engine that rotates the draft's NoPE layer gets 4.98 instead of 7.34 on GSM8K. Update the engine, then measure acceptance on your prompts.
The cache grows five times faster than expected. --swa-full is set. Unset it so the 64 window layers keep a 1,024-token cache.
Reasoning text repeats or newlines double across turns. Use the ripple parsers, which keep the template's framing newlines out of reasoning_content, and send that field back unchanged.
A required tool call carries extra fields. Set strict to true and additionalProperties to false, as the README asks.
The 4-bit file fails to load on a 48 GB Mac. The default GPU memory limit is near 36 GB. Raise it with sudo sysctl iogpu.wired_limit_mb=44032 until the next restart, or use the 3-bit file.
mlx-lm reports an unknown model type. It has no Ripple model file yet. Run the GGUF through llama.cpp with Metal.

FAQ

How fast does Ripple-65B decode on a DGX Spark?

The NVFP4 build reads 56.19 GB per token, and the Spark moves 273 GB/s. That is 4.8 tokens a second at a short context and 4.2 at 131,072 tokens. With the drafter, multiply by the tokens accepted per pass. At the 7.34 that vLLM measured on GSM8K, the Spark reaches about 31.

How much memory does the cache need?

65,536 bytes per token for the 16 global layers, in BF16. That is 8.6 GB at 131,072 tokens and 68.7 GB at 1,048,576. The 64 window layers hold a fixed 268 MB.

Which llama.cpp build runs Ripple-65B?

Release b12167 from November 6, 2026, or newer. Pass --jinja and leave --swa-full unset.

Does vLLM 0.33.0 support it?

No. Support merged on November 5, after the 0.33.0 release. Use a nightly wheel or the nightly image until the next release.

Is Ripple-65B free for commercial use?

The four Ricursive repositories and the AtomicChat GGUF repository each carry the Apache 2.0 license file.

Keep reading

FlintA 40B mixture of experts that reads 2.5B parameters per token, the opposite shape. Tokens per second is memory bandwidthWhy bytes read per token set the decode ceiling.

AI Engineering from Scratch free course The curriculum behind these pages: every core AI algorithm built by hand, 503 lessons across 20 phases, from the math to the tokenizer to the agent loop. Open source, learn at your own pace. Start at aiengineeringfromscratch.com →

Rohit Ghumare

CNCF Ambassador and Google Developer Expert. I build agent infrastructure and write about the fundamentals underneath the AI stack. Config values, tensor shapes, GGUF header types, and file sizes come from the Hugging Face repositories of ricursiveAI and AtomicChat, read on November 6, 2026. Merge dates and measurements come from the llama.cpp, vLLM, and SGLang pull requests. Bandwidths are NVIDIA's and Apple's specifications. Benchmark scores are Ricursive's own.

Flint · Hardware guide · More guides · X