Rohit Ghumare · September 15, 2026 · 10 min read

BITCOS: Intel's Ternary LLM Weight Format Goes Below 1.58 Bits

A ternary language model stores every weight as one of three values, minus one, zero, or plus one, and the field has treated log2(3), about 1.585 bits, as the floor on what one of those weights can cost. On September 14, three Intel researchers posted Breaking the 1.58-bit Barrier for Ternary LLMs, which stores a real ternary checkpoint at 1.485 bits per weight without retraining it and without losing a single value. The trick is an observation about the checkpoints people actually ship: across 29 ternary models, between 29.66% and 51.48% of all weights are zero. The 1.585 figure assumes the three values show up equally often. They do not, and a format that admits it gets to go under the ceiling. Two days later the paper sat at 245 points on Hacker News. This is how the encoding works, why its cost is a one-line formula, and where the speedup stops.

What BITCOS is. BITCOS, short for bitmap and compacted signs, is a weight-packing format for ternary large language models, introduced by Intel Labs researchers Evangelos Georganas, Alexander Heinecke, and Pradeep Dubey. It is a file and memory layout for model weights, with CPU and GPU unpacking kernels to match. It has nothing to do with the cryptocurrency or DeFi token that shares the name.

Where 1.58 comes from

Start with the ladder most people already know. A weight in FP16 costs 16 bits. INT8 costs 8. The 4-bit formats that run most local models cost 4 bits plus a small share for the per-group scales that map integers back to real numbers. Each step down halves the bytes that have to cross the memory bus for every generated token, which is why, as I wrote in Tokens per Second Is a Memory Bandwidth Number, decode speed on a single stream tracks the size of the weights more closely than it tracks FLOPs.

Ternary models are the far end of that ladder, and they are a different kind of object. You cannot take a trained FP16 model and round it to three values without destroying it. The model has to be trained, or heavily fine-tuned, knowing its weights will be ternary. Microsoft's BitNet b1.58 paper from February 2024 made the recipe popular: scale a weight matrix by its mean absolute value, round, and clip to the set {-1, 0, +1}. The "1.58" in the name is log2(3). If each weight is one of three symbols and all three are equally likely, information theory says you need log2(3) bits per weight on average, and no encoding can do better.

That conditional clause is the one the new paper attacks. Shannon's bound is not log2(3). It is the entropy of the actual distribution of symbols, and log2(3) is only what the entropy works out to when the distribution is flat. Change the mix and the floor moves.

What the shipping packers actually store

Before getting to the floor, look at what deployed kernels pay today, because none of them sit at 1.585 either. The paper describes the two common layouts.

The simplest is 2 bits per weight. Two bits hold four states, three of which are used, and decoding is a shift and a mask. It wastes a quarter of the space and it is the fastest thing to unpack, which is why the paper uses 2-bit kernels as its speed baseline.

The tighter one packs five ternary digits into a byte. Five trits have 3⁵ = 243 combinations, which fits under 256, so the rate is 8/5 = 1.6 bits per weight. That would be close to the bound, except that real kernels quantize in blocks of a power of two, typically 128 weights, and 128 is not divisible by 5. A block needs ⌈128/5⌉ = 26 bytes, and the paper puts the stored rate at 26 × 8 / 128 = 1.625 bits per weight. That is the number to beat.

The zeros nobody was counting

The authors measured the zero density of 29 published ternary checkpoints across seven families. The lowest is Bonsai 27B at 29.66%. The highest is CAT-Q Qwen3-1.7B at 51.48%. BitNet b1.58 2B4T, the model most people have actually run, sits at 42.19%. In between: the other Bonsai models at 37.71% to 39.89%, the TriLM family from 38.70% to 40.97%, the BitCPM-CANN models from 37.67% to 39.30%, Maple 20B-A1B at 40.67%, the ParetoQ models from 41.07% to 47.10%, and the rest of the CAT-Q Qwen3 conversions from 32.88% to 47.11%.

A flat distribution would put each symbol at 33.3%. Every model in the set except Bonsai 27B has more zeros than that, several of them by a wide margin.

You can see why from the quantizer. The BitNet recipe divides each weight by the matrix's mean absolute value and rounds, so anything within half a mean-absolute-value of zero becomes zero. If you assume the pre-quantization weights are roughly Gaussian, that window catches about 31% of them. That is my arithmetic from the recipe, not a number from the paper, and it lands close to the least sparse model in the survey. Training with the quantizer in the loop then pushes the mass further toward zero in most families, which is where the 40% and 50% figures come from.

A bitmap and a sign

BITCOS, the paper's layout, splits every weight into two questions and answers them in two separate arrays. The first is a presence bitmap: one bit per weight, set when the weight is non-zero. The second is a sign vector: one bit per non-zero weight only, in the same order, saying whether it is +1 or -1. Zeros cost their one bitmap bit and nothing else.

The cost falls out directly. With a zero density of z, every weight pays 1 bit for the bitmap and a fraction (1 − z) of them pay 1 more for the sign, so the rate is 1 + (1 − z) = 2 − z bits per weight. The paper writes it the same way. At CAT-Q Qwen3-1.7B's 51.48%, that is 1.4852, the 1.485 in the headline.

The formula also tells you exactly when it loses. BITCOS beats 1.625-bit five-trit packing whenever 2 − z < 1.625, which is any zero density above 37.5%. Three models in the survey fall below that line: Bonsai 27B at 29.66%, CAT-Q Qwen3-30B-A3B at 32.88%, and CAT-Q Qwen3-235B-A22B at 34.07%. The paper reports BITCOS winning in 26 of 29 models. The three it does not win are exactly the three the formula predicts, and it is worth noticing that they are the two largest mixture-of-experts conversions plus the largest Bonsai. Bonsai 4B, at 37.71%, clears the line by two hundredths of a bit.

The most interesting question is how close 2 − z gets to the real floor. If signs are balanced between +1 and −1, the entropy of the ternary distribution is h(z) + (1 − z), where h(z) is the binary entropy of "is this weight zero." BITCOS spends exactly 1 bit on that question, and h(z) is at most 1, reached when z = 0.5. So the gap between BITCOS and the Shannon floor is 1 − h(z). At z = 0.5148 that gap is 0.0006 bits per weight. The layout is, for the sparsest model in the survey, within a rounding error of optimal.

One Hacker News commenter said they were tempted to write a follow-up paper using arithmetic coding "to squeeze out a few more centi-bits." The formula answers that too. At a zero density of 40%, an ideal entropy coder would save 0.029 bits per weight over BITCOS. At 30% it would save 0.12. A few centi-bits is exactly right, and every one of them would cost decode work, which is the part that matters next.

Decoding is the other half of the bill

A smaller file only helps if the kernel can turn it back into numbers faster than the memory bus delivers it. Every token of decode streams the whole weight matrix through the compute units once, so the win from BITCOS is at most the byte ratio against the 2-bit baseline, 2 / (2 − z). At 51.48% zeros that ceiling is 1.35x. At 29.66% it is 1.17x. Everything the decoder spends on unpacking comes out of that margin.

The paper's decoders are careful about this. On CPUs with AVX-512, the kernel loads 32 bits of the presence bitmap into a mask register, then uses pdep, the parallel bit deposit instruction, to scatter the next run of compacted sign bits into the positions the mask marks as present. Every unmarked position comes out zero. Masked moves then pick +scale or −scale per lane. On AVX2, which has no mask registers, the kernel expands presence and sign into byte masks and computes each weight as p − 2n, presence minus twice the negative flag. On Intel's Xe2 GPUs it uses a 256-entry lookup table in shared local memory, indexed by a 4-bit presence nibble plus a 4-bit window of sign bits, returning four fp16 ternary codes at a time.

The measured results are honest about the ceiling. Against the 2-bit reference kernel, matrix-vector multiplication got 1.14 to 1.28x faster on a 64-core Emerald Rapids Xeon and 1.13 to 1.27x on a 24-core Arrow Lake desktop part, across the survey's range of zero densities. On the Arc 140V integrated GPU it was 1.04 to 1.14x, and on the Arc Pro B70 discrete card 1.01 to 1.12x. End to end, decode throughput improved by 1.10 to 1.18x on Emerald Rapids and 1.02 to 1.27x on the B70. The 1.28x top line on the Xeon sits close to the 1.35x byte ceiling, which says the AVX-512 decoder is nearly free there.

The Lunar Lake row is the most useful line in the paper. That laptop chip has 108 GB/s of bandwidth and only eight cores to spend it. The authors measure each core as having 3.2 to 3.7 bytes per cycle of bandwidth available but only able to consume 0.87 to 1.38 bytes per cycle through the BITCOS unpack sequence, so the kernel sustains 28.9 GB/s at a zero density of 40% and loses to the simple 2-bit kernel everywhere. When the pipe is wide and the cores are few, the cheapest decode wins, even if it moves more bytes. The paper names the same effect at the other extreme: push the zero density toward 95% and the unpack instructions become the bottleneck on every platform.

So the rule for local inference is the one from the memory-bandwidth essay with a second term added. Tokens per second is bytes moved divided by bandwidth, as long as the decoder keeps up. BITCOS shrinks the bytes. Whether that shows up as speed depends on how many cores are available to do the unpacking.

What this changes, and what it does not

The first thing to be clear about is that BITCOS is lossless. It does not change a single weight. It re-encodes the ternary values a model already has, which is why it needs neither retraining nor any special sparsity hardware, and why the paper contrasts it with approaches like Sparse-BitNet that train in structured sparsity. Any ternary checkpoint can be converted, and a converter could compute 2 − z for each checkpoint and fall back to five-trit packing when z is under 37.5%. That per-model choice is my suggestion, not something the paper ships.

The memory savings are real but modest. In bits per weight the gain over five-trit packing is 1.625 − 1.485 = 0.14 at the best case, about 8.6% smaller. For an 8-billion-parameter model the arithmetic is tidy, because bits per weight times 8 billion divided by 8 bits per byte equals gigabytes: 1.625 GB of ternary weights becomes 1.485 GB. The embeddings, norms, and scales that stay in higher precision are not part of that number.

The bigger question is whether ternary models matter at all, and the Hacker News thread split on it the way the field does. One commenter argued that vector quantization and trellis-based methods beat ternary for post-training quantization at these bit rates. Another pointed at the hardware angle: a format this simple, with a bitmap and a sign, is the kind of thing a small accelerator could consume directly. Both can be true. Ternary checkpoints still have to be trained that way, and the families in the survey top out well below frontier scale in dense form.

What the paper does settle is narrower and more durable. The information floor for a ternary model is a property of that model's weights, not of the number three. Measure the distribution before choosing the container. For the sparsest checkpoints anyone has published, the container that falls out of that measurement is two bit arrays, a formula you can do in your head, and a result within six ten-thousandths of a bit of what Shannon allows.