Local model guide · October 19, 2026 · updated November 3, 2026 · 18 min read

How to Run FLUX 3 Image Klein Locally

Black Forest Labs released FLUX 3 Image Klein on October 15, 2026: a 5.6B open-weights image model for text-to-image, editing, and layout with bounding boxes. The step-distilled checkpoint makes a 1024 × 1024 image in 16 passes of its transformer, with no classifier-free guidance. The int8 build of that transformer is a 5.8 GB file, and BFL says it runs on 8 GB cards such as the RTX 3070. The sections below follow one image through the text encoder, the latent space, and the denoise loop. Then they show what step distillation removed and give the commands for ComfyUI, Diffusers, and the reference CLI.

What Black Forest Labs released

Four repositories appeared on Hugging Face on October 13 and 14, and went public with the BFL blog and the Diffusers blog on October 15. The distilled checkpoint is flux-3-image-klein: 16 steps, no guidance, and the same weights that serve the BFL API. The undistilled checkpoint is flux-3-image-klein-base: 50 steps with guidance 3.5, built for fine-tuning, LoRA training, and research. The prompt upsampler is flux-3-image-klein-pup, a 9B vision-language model fine-tuned from Qwen3.5-9B. The ComfyUI repack is flux-3-image-klein-comfy, with int8 and NVFP4 builds beside BF16. The three weight repositories are gated, and the upsampler repository is not. You accept the license on Hugging Face before a gated file downloads.

The 5.6B label counts the image transformer only. Its safetensors index holds 5,548,681,984 BF16 parameters, an 11.1 GB file. The text encoder is a separate 4B model, Qwen3-VL-4B-Instruct, at 8.9 GB in BF16. The autoencoder is 0.3 GB. The upsampler is 18.8 GB in BF16. One image touches all four at some point, so the download is larger than the headline number.

One model does three jobs. It makes an image from a caption, and it edits with up to 10 reference images. It also places elements with bounding boxes written into the caption on a 0 to 1000 grid. It renders text, and it generates up to 4K. The card repeats one rule in bold: do not use the model without a prompt upsampler. The model was trained on long scene captions, and a short prompt sent as typed gives generic output.

BFL reports first place in its internal Elo ratings among open-weights models, with 38% fewer parameters than FLUX.2 [klein] 9B. Those are vendor numbers from internal evaluations. I found no independent evaluation when I checked. The file sizes below come from the Hugging Face tree API on October 23, 2026, in decimal gigabytes.

The int8 and w4a8 files exist only in the ComfyUI repository. The FP8 and NVFP4 files sit under variants/ in the two original repositories as well. NVFP4 needs a Blackwell GPU. FP8 needs Ada, Hopper, or Blackwell. The int8 files are the ones for RTX 30 series cards and anything older without FP8 support.

The text encoder

The transformer never reads words. It reads vectors from Qwen3-VL-4B-Instruct, a vision-language model under Apache 2.0. The pipeline runs the caption through the chat template and pads it to a multiple of 80 tokens. Then it does something unusual. Instead of the last layer alone, it reads the hidden states of eight of the 36 layers. Those are layers 4, 8, 12, 16, 20, 24, 28, and 32. The eight 2,560-value vectors for each token stack into one vector of 20,480 values. A linear layer projects that to the transformer width of 3,072. The Diffusers blog explains the choice: the transformer gets features from several depths of the language model, not only its final layer. The transformer attends to the padding too, which is why the pipeline takes one prompt per call.

The encoder sees only the caption. Reference images do not go through it. They go through the autoencoder, as the next section shows. The encoder runs once per image, before the loop. After that, its output is a fixed input to every denoise step.

In front of the encoder sits the upsampler, and the card treats it as part of the model. It is a full fine-tune of Qwen3.5-9B with the vision tower frozen, trained in two stages. First, supervised training on pairs of short requests and their long captions. Then preference tuning on pairs of its own captions, rendered by the image model at the same seed and ranked on the resulting picture. The upsampler card gives the example this guide uses later. The request is "A man walking with his dog, afternoon sun". The caption it writes is 220 words: medium, shot, light direction, the man's clothes, the dog, and where each sits in the frame. The card lists the decoding settings as greedy, top_p 0.95, top_k 20, and a presence penalty of 1.5 for text-to-image. Any other LLM can take its place with the system prompts in system_prompts.json, one each for text-to-image, edits, and the two box modes.

The latent space and the VAE

The transformer also never sees pixels. It works in the latent space of the FLUX.2 autoencoder, a 0.3 GB model that the card ships beside the weights. The encoder half compresses an image 8 times in each direction into 32 channels. Then every 2 × 2 patch of that latent is packed into one token of 128 values. So one token stands for a 16 × 16 pixel patch. A 1024 × 1024 image is a 64 × 64 grid of 4,096 tokens. A 4096 × 4096 image is 65,536 tokens. The Diffusers code draws its noise in exactly that shape. It is a tensor of (batch, 128, height / 16, width / 16), flattened into a token sequence.

A reference image takes the same path. The VAE encoder compresses it, the packing turns it into tokens, and those tokens join the sequence at their own size. The Diffusers blog measured the cost. One 1024 × 1024 reference adds 4,096 tokens to every transformer call, which about doubles the time of an edit. A single reference keeps its size up to 4 megapixels. Several references are each fitted under 1 megapixel.

Every token needs a position, and attention alone has none. FLUX 3 gives each 128-value attention head four rotary position axes of 32 values each: a timestep axis, height, width, and a sequence axis. Text tokens line up along the sequence axis like words. Image tokens sit on the height and width grid. Reference tokens use the same grid, shifted along the sequence axis by their index. The same spot in a reference has a precise meaning to the model, and that is what moving an element by box depends on.

The decoder half of the VAE runs once, after the last step. It turns the 128-channel grid back into pixels. The Diffusers docs load the VAE in fp32 for the base model and keep everything else in bf16.

Inside the transformer

The transformer is 3,072 wide, with 24 attention heads of 128 values. It takes three kinds of tokens: the caption, the noisy image, and any references. The Diffusers source builds it in two stages. In the first stage each stream has five blocks of its own, with its own weights, and each stream attends only to itself. In the second stage the streams join into one sequence: caption first, then image, then references. That sequence goes through 28 blocks that share one set of weights. The 5 + 28 plaque on the machine above is this split. At the end, the caption and reference tokens are dropped. Only the image tokens pass through the output layers, which project 3,072 values back to 128.

Every block has the same shape. Attention and a SwiGLU feed-forward run in parallel on the same normalized input, and their sum is added back to the residual stream. The layer norm has no learned scale or shift of its own. The scale, the shift, and a gate come from a timestep embedding instead. Queries and keys get an RMS norm and the rotary positions from the section above. In a joint block the weights are shared, but caption, image, and reference tokens each get their own scale, shift, and gate.

That last point is new in FLUX 3. In FLUX.1 and FLUX.2, one timestep embedding modulates every token. In FLUX 3, the image tokens are modulated by the current noise level, and the caption and reference tokens are always modulated by timestep 0. Zero means clean, nothing to denoise here. A reference that looks almost the same as the output is still marked as given context, never as something to generate.

The denoise loop

FLUX 3 is a flow-matching model. During training, a clean latent x and a noise sample ε are mixed as (1 − σ)·x + σ·ε. The noise level σ runs from 0 to 1. The transformer learns to predict the velocity from noise toward the picture, which is ε − x. At sampling time the loop starts at σ = 1, which is pure noise. Each step the transformer predicts a velocity v, and the scheduler moves the latent by (σ_next − σ)·v. The scheduler source is one line: prev_sample = sample + dt * model_output. At any step, the model's current guess of the final picture is the latent minus σ times v.

The schedule is where the model spends its steps. The pipeline starts with 16 noise levels spaced evenly from 1 down to 1/16. Then the scheduler shifts them with the constant 6.93 that the pipeline sets: σ' = 6.93σ / (1 + 5.93σ). The shifted values are 1.000, 0.990, 0.980, and so on down to 0.615, 0.497, and 0.316, and the last step jumps from 0.316 to 0. Fourteen of the 16 steps sit above σ = 0.5. The first step moves the noise level by 0.010. The last one moves it by 0.316, a third of the whole path. With the same shift at 50 steps, 44 steps sit above 0.5, which matches the count the Diffusers blog gives. The blog's step-by-step decodes show why the shift is there. The fox, its pose, and the light are in place by step 10 of 50, while σ is still 0.97. Composition is decided near pure noise, so the schedule keeps the model there for most of the run.

What step distillation removed

The base checkpoint is a plain flow model. Given a noisy latent and its noise level, it predicts the velocity at that point. A small step along that velocity is accurate, and a large one is not, because the true path curves. So the base model takes 50 small steps.

The distilled checkpoint is a flow map. At every step it is told two noise levels: the current one and the one the step jumps to. It predicts the velocity for that whole jump, curve included. The transformer carries a second timestep embedder for the target level. The source adds its output to every mode's modulation. So it reaches the five per-mode blocks, the 28 joint blocks, and the output layer. The Diffusers docs say the model is trained for 4 to 16 steps, with 16 as the default. At 4 steps the schedule ends at σ = 0.698 before the final jump to 0, and the model still has to land the picture.

The two checkpoints are otherwise the same shape, and the parameter counts prove it. The base transformer has 5,538,458,368 parameters. The distilled one has 5,548,681,984. The difference is 10,223,616, which is 256 × 3,072 + 3,072 × 3,072: one two-layer timestep embedder with no biases, and nothing else. Distillation removed the need for small steps and added one embedder. It also removed the guidance, which is the next section. The cost is on the training side. The card says to train on the base checkpoint, which was trained without step or guidance distillation. It treats the distilled one as a generation model.

Why dropping CFG halves the compute

Classifier-free guidance is a sampling trick for models that were trained with the caption dropped some of the time. At each step the model runs twice, once with the caption and once with an empty or negative prompt. The sampler then pushes the prediction along the difference: v = v_uncond + 3.5 × (v_cond − v_uncond). The Diffusers guider source reports two conditions when guidance is on. The base pipeline's denoise step loops over both and calls the transformer for each. Two transformer passes per step, every step.

The distilled pipeline has no guider and takes no negative prompt. The distilled model was trained to produce the guided result in one pass. One transformer pass per step. So the arithmetic is 50 steps × 2 passes = 100 transformer calls for the base model against 16 × 1 = 16 for the distilled one. The encoder and the VAE run once in both. BFL measured about 3 seconds against 12 seconds per 1 megapixel image on one H200 for the two checkpoints. One table in the reference README prints guidance 4 for the base model. The card, the ComfyUI template, and the Diffusers default all use 3.5.

Fitting an 8 GB card

Three parts must touch the card for one image. With the int8 files, the transformer is 5.8 GB, the encoder is 4.9 GB, and the VAE is 0.3 GB. That is 11.0 GB in total. That is more than 8 GB, and the model still runs there. The parts never need the card at the same time. The encoder runs once, before the loop. The transformer runs sixteen times. The VAE runs once, after. ComfyUI loads a model when its node runs and moves models out when memory runs short. So the largest thing resident at any moment is the 5.8 GB transformer plus the working memory of one pass. The upsampler is a fourth turn in front of the encoder, 6.7 GB in the w4a8 build the templates use. BFL's statement is plain: the int8 files run on 8 GB GPUs such as the RTX 3070, and on GPUs without FP8 support.

The 8 GB story is a ComfyUI story. The reference code has no int8 path. BFL measured the distilled model on a B200 at 1024 × 1024 with the upsampler off. The peaks are 31 GB in BF16, 26 GB in FP8, and 25 GB in NVFP4. The Diffusers blog measured a 20.7 GiB peak in bf16 on an H100, and 15.3 GiB with NVFP4 on an RTX 6000. It points at offloading for smaller cards, without a benchmark. For speed, BFL reports about 4 seconds per 1 MP image on an RTX 5090 with the distilled int8 build in ComfyUI. The same build takes under 9 seconds on an RTX 4070. A GB300 in NVFP4 takes under 0.6 seconds. Diffusers on an H100 in bf16 took 1.93 seconds. All of those are vendor numbers.

Community GGUFs for stable-diffusion.cpp appeared the next day. The Unsloth repository from October 16 holds the distilled transformer from 2.62 GB at UD-Q2_K_XL to 6.09 GB at Q8_0, with Q4_K_M at 3.67 GB. I have not run those builds, and this guide makes no claim about their quality.

The license, per variant

The three weight repositories carry the same license, and the cards state it in the same words. Klein, Klein Base, and the ComfyUI repack are released under the FLUX Kommunity License v.1.0. Each card states the terms in one sentence, quoted here in full. "Non-commercial use, plus commercial output generation by Qualifying Users with less than US$5M in gross annualized revenue, including affiliates, subject to the license terms." The next card sentence reads: "For other uses, see commercial licensing." The key features list of each card says the license "permits non-commercial use and provides a commercial-output exception for Qualifying Users, subject to its terms." The out-of-scope section adds that the exception "is not blanket permission for commercial fine-tuning or distribution." Section 4 of the license lists the restricted uses. They include military purposes, surveillance, biometric processing, unlawful or abusive activity, and violating third-party rights. They also include using the model or its outputs to improve another AI model that performs a similar function. The BFL blog states the same terms as free commercial use for individuals and companies with annual revenue under $5M, alongside research and non-commercial use.

The parts around the weights are more open. The Qwen3-VL-4B-Instruct text encoder is under Apache 2.0, and each card says so. The prompt upsampler is released under the Apache License 2.0, derived from Qwen3.5-9B, which is also Apache 2.0. The reference code on GitHub is Apache 2.0. The trademark section of the three weight cards asks for written permission before you use the Black Forest Labs or FLUX names. Required attribution and descriptive references are the exceptions. The upsampler card asks only that modified versions do not imply sponsorship or endorsement.

ComfyUI

ComfyUI is the path for an 8 GB card, and the one the templates are built for. The ComfyUI tutorial asks for ComfyUI 0.41.0 or later. Accept the license on the comfy repository, then download one transformer, the text encoder, the VAE, and the upsampler into the model folders. The command on the comfy card fetches the int8 upsampler, 9.8 GB, which the workflows in that repository load. The command below swaps it for the 6.7 GB w4a8 file that the ComfyUI template library loads.

hf download black-forest-labs/flux-3-image-klein-comfy \
  diffusion_models/flux3-image-klein-int8-convrot.safetensors \
  text_encoders/qwen3vl_4b_int8_convrot.safetensors \
  vae/flux2-vae.safetensors \
  text_encoders/qwen3.5_9b_flux3_prompt_upsampler_w4a8.safetensors \
  --local-dir ComfyUI/models

Open the template named "FLUX 3 Image Klein Distilled: Text to Image" from the template library. The model runs on the standard loaders. Load Diffusion Model takes the transformer file. Load CLIP, with its type set to flux3, takes the text encoder. Load VAE takes the autoencoder. Type a short prompt, because enable_prompt_upsampler is on by default and expands it before sampling. Keep steps between 4 and 16 and leave cfg at 1, since the distilled checkpoint runs without guidance and a higher value does nothing useful. Pick a frame in ResolutionSelector and press Queue. The image lands in ComfyUI/output/. For an edit, open the "Distilled: Image Edit" template and load your picture into LoadImage. Connect it to Set Reference Multi Latent, and write the result as a full caption. The first reference is <ref_image_0> in the prompt. With more than 24 GB of memory, the tutorial suggests the BF16 transformer and the BF16 upsampler, because they follow long prompts more closely.

Diffusers

FLUX 3 arrives in Diffusers as a modular pipeline, and the pipeline docs describe three differences from the usual DiffusionPipeline. Loading is lazy: from_pretrained builds the pipeline and load_components loads the weights. A call returns a state, and you name the output you want. And the pipeline runs the prompt upsampler on every request, text-to-image and editing alike. The distilled repository resolves to a pipeline with no guider and no negative prompt, and num_inference_steps defaults to 16. Diffusers 0.42.0, released on October 23, is the first release with the pipeline, so a normal upgrade installs it. The model cards still show an install from git main, which was the only path at launch.

pip install -U "diffusers>=0.42.0" transformers

import torch
from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein", components_manager=manager)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
manager.enable_auto_cpu_offload(device="cuda")

out = pipe(
    prompt="A man walking with his dog, afternoon sun",
    generator=torch.Generator().manual_seed(7),
    output=["images", "prompt"],
)
out["images"][0].save("dog.png")
print(out["prompt"])

The second output is the caption the upsampler wrote, which is the only text the image model saw. Without the manager, add pipe.to("cuda") after loading and expect about 39 GB of weights in bf16, 19 GB of which is the upsampler. The docs show how to pop the prompt_upsampler block out and run it as its own stage. That is the right shape for a small GPU. Write the caption first, on the CPU or elsewhere. Then build the image pipeline without the upsampler and pass the caption as prompt. Pass image with one picture or a list to edit, and boxes for layout. For a smaller transformer, load the variants/fp8r or variants/nvfp4 single file with Flux3Transformer2DModel.from_single_file and a BFLQuantizationConfig, then update_components on the pipeline. On GPUs without the matching kernels, the Diffusers blog says the weights are dequantized each step: lower memory, no speedup. The 0.42.0 release notes list FP8r and NVFP4 loading and no int8 loader. The BFL blog also names Diffusers as a route for 8 GB cards. The int8 files are ComfyUI files, and the published RTX numbers come from ComfyUI. So on an RTX 30 card, ComfyUI is the route with measurements behind it.

The reference CLI

The reference code is the shortest route when the GPU is large. It loads the upsampler by default and reserves 35% of the GPU for it. The install was tested on a B200. The fast model with its upsampler and the safety classifier is a 52 GB download.

python3.12 -m venv .venv && . .venv/bin/activate
pip install -e . --extra-index-url https://download.pytorch.org/whl/cu130
hf download black-forest-labs/flux-3-image-klein --local-dir weights/flux-3-image-klein --exclude 'variants/*'
hf download black-forest-labs/flux-3-image-klein-pup --local-dir weights/pup
hf download Falconsai/nsfw_image_detection --local-dir weights/nsfw-classifier

python scripts/cli.py "A man walking with his dog, afternoon sun" --model distilled --out out.png --seed 7
python scripts/cli.py "put a red bandana around the dog's neck" --image out.png --out out2.png --seed 22

python -m flux3.pup --prompt "A man walking with his dog, afternoon sun" --out prompt.txt
python scripts/cli.py "$(cat prompt.txt)" --model distilled --fp8 --no-pup --out out.png

The last two lines are the README's recipe for a GPU that cannot hold the upsampler and the model together. Write the caption first, then generate with --no-pup. The --fp8 flag loads the FP8 transformer on Ada, Hopper, or Blackwell, and --nvfp4 needs Blackwell and the [nvfp4] install. Every generated image comes with a JSON file that records the prompt, size, step count, guidance, and seed. Boxes go in a JSON file with values between 0 and 1, and the upsampler writes the 0 to 1000 table into the caption. The README also lists SGLang Diffusion, vLLM, and TensorRT-LLM as serving paths with their own guides.

A worked example

Take the request from the upsampler card, "A man walking with his dog, afternoon sun", through the ComfyUI distilled template on an RTX 3070. The upsampler loads first and turns 11 words into about 220. The caption fixes a medium-wide shot at eye level and late afternoon light from the upper right. It puts a man in an olive overshirt walking toward frame-left, with the dog beside him. The upsampler then leaves the card. The text encoder loads, runs the caption once, and produces 160 token vectors. It leaves. The transformer loads, 5.8 GB, and runs 16 times over 4,096 image tokens plus the 160 caption tokens. On the counter in the machine above, the noise level drops from 1.000 to 0.316 over the first 15 steps and then to 0. The VAE decodes the 64 × 64 grid into a 1024 × 1024 PNG. BFL's figure for the RTX 4070 is under 9 seconds for the sampling part of this. The 3070 is slower and has no published number.

For the edit, open the distilled edit template and load that PNG as the reference. Write the result, not the change. The caption describes the same scene with a red bandana around the dog's neck. The upsampler writes that from the short instruction. The reference adds 4,096 tokens to every call, so the sampling takes about twice as long as the first picture. The output comes back at the size of the input. If the shot reframes instead of editing, the prompt was a one-line instruction without the upsampler. That is the first item in the list below.

Failure modes and fixes

• Generic or flat pictures, or an edit that reframes the shot. The upsampler was off and the prompt was short. Turn enable_prompt_upsampler on, or write the full caption yourself with the system prompts from the upsampler repository.
• Raising cfg on the distilled checkpoint changes nothing. It runs with guidance 1 and has no unconditional pass. Use the base checkpoint with guidance 3.5 when you want guidance.
• 401 on download. The weight repositories are gated. Accept the license on Hugging Face and log in with hf auth login before hf download.
• The template is missing or nodes fail to load. ComfyUI is older than 0.41.0. Update it.
• Out of memory with the reference code on a 24 GB card. The reference code has no int8 path and reserves 35% of the GPU for the upsampler. Pass --fp8 on Ada or newer, or write the caption with python -m flux3.pup and generate with --no-pup, or move to ComfyUI.
• NVFP4 or FP8 loads but runs no faster. NVFP4 kernels need Blackwell and FP8 kernels need compute capability 8.9 or newer. On other GPUs Diffusers dequantizes the weights each step.
• An element spills past its box, or a small one never appears. Boxes set placement and scale, not a hard mask. Enlarge the box or describe the element in the caption as well.

FAQ

Can I use it commercially?

The card states the FLUX Kommunity License v.1.0 in one sentence. "Non-commercial use, plus commercial output generation by Qualifying Users with less than US$5M in gross annualized revenue, including affiliates, subject to the license terms." The card adds that the exception is not blanket permission for commercial fine-tuning or distribution. The text encoder, the upsampler, and the reference code are Apache 2.0.

Does it run on an 8 GB GPU?

BFL says the int8 files in the ComfyUI repository run on 8 GB GPUs such as the RTX 3070. The int8 transformer is 5.8 GB, the int8 encoder is 4.9 GB, and the VAE is 0.3 GB. They take turns on the card: the encoder runs once before the loop, the transformer runs 16 times, and the VAE runs once after.

What is the difference between Klein and Klein Base?

Klein is step-distilled: 16 steps, no guidance, one transformer pass per step. Klein Base is the undistilled checkpoint: 50 steps with guidance 3.5, two passes per step, for fine-tuning, LoRA training, and research. The distilled transformer has 10,223,616 more parameters, which is exactly one extra timestep embedder for the level each step jumps to.

Why does it need the prompt upsampler?

The model was trained on long scene captions, and the card says a short prompt sent as typed gives generic output. The 9B upsampler expands a request such as "A man walking with his dog, afternoon sun" into about 220 words. The text encoder sees only the long one. Any other LLM can do the job with the system prompts in system_prompts.json.