Training internals guide · October 16, 2026 · 28 min read

Inside Miles

Reinforcement learning on a language model runs two programs at once. One program writes answers, and the other program learns from those answers. Miles is the open-source framework from RadixArk that runs both programs. SGLang writes the answers and Megatron-LM learns from them. This guide follows one training step through the Miles main branch as of October 15, 2026, one week after release v0.1.2. It stops at each part that decides speed, memory, or correctness.

Every number in this guide has one of three labels. A Mac measurement is a run I did on an Apple M1 Max on October 16, 2026. A source number comes from the Miles or SGLang code, or from their documentation. A claim is a number that a company published about its own work, and I could not check it. Three figures replay the Mac data, and one figure replays a rule set from the source with simulated input.

Step 1: See why RL needs an inference engine

In supervised fine-tuning, the training data is on disk before the run starts. In reinforcement learning, the model makes its own training data. The trainer asks the current model for answers, scores each answer, and updates the weights. Then it needs new answers from the updated model, and the cycle starts again.

Writing answers is inference. So every RL system for language models contains a complete inference engine, and that engine does most of the work. In a research post from Modal, sampling uses 80 to 85 percent of the GPUs in large RL runs, from figures that other labs published. A slow engine gives a slow training run. An engine with different numbers from the trainer gives a wrong training run.

Three jobs follow from this design, and each job is a subsystem in Miles. First, the engine must load new weights after each update. Second, the engine and the trainer need the same GPU memory at different times. Third, the probabilities that the engine reports go into the loss. So the engine must agree with the trainer, or the trainer must correct for the difference. Steps 4 to 9 of this guide cover these three jobs.

Miles began as a fork of slime. The slime README names it as the RL framework behind Z.ai's GLM models, from GLM-4.5 to GLM-5.3. RadixArk released Miles on November 19, 2025, and the two projects still share fixes. RadixArk also maintains SGLang, so one company works on both halves of the loop. The vLLM guide on this site explains the serving side of an engine in the same depth.

On October 16, 2026, the Miles repository had 3,144 GitHub stars and 564 forks, from the GitHub API. Its git log showed 616 commits in the previous 30 days. On the same day, slime had 8,673 stars. RadixArk says that Miles runs production RL at humans&, Periodic Labs, Modal, Decagon, Eigent AI, Nebius, and IBM. That list is a claim. Modal confirmed its own use in public, and Step 10 follows that run.

Step 2: Follow one RL step through train.py

The synchronous loop is in train.py, and the file is 176 lines long. Read it once, because every other part of Miles changes one of its steps. The setup creates the rollout side first: the SGLang engines, the router, and a rollout executor. It needs the rollout side first to calculate how many rollouts one epoch contains.

Next, the setup creates the trainer. Then it pushes the trainer's weights to every engine once. After that push, the engines serve exactly the weights that the trainer holds. Without it, the first rollout would come from a different checkpoint.

Each loop iteration then does six things in a fixed order. It asks the executor for one rollout, which generates answers and scores them with a reward function. It releases engine memory if the run shares GPUs. It trains, and under PPO the critic trains before the actor. It saves a checkpoint on schedule and runs an evaluation when one is due. Last, it loads the engine memory back and pushes the new weights.

One equation holds the batch sizes together. The Miles documentation writes it as rollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout. You set any three values, and Miles calculates the fourth. The quick start uses 32 prompts with 8 answers each. That gives 256 samples, and the trainer uses all 256 for one optimizer step.

Four log keys tell you where the time goes. The key perf/rollout_time is generation time, and perf/actor_train_time is training time. The key perf/train_wait_time is the time the trainer waited for data. The key rollout/raw_reward is the mean reward of the new answers, and the quick start calls it the number to watch.

The reward function decides what the model learns. Miles has built-in reward types, chosen with --rm-type, and a hook for your own function, set with --custom-rm-path. The built-in set includes math answer checkers and benchmark graders such as GPQA and IFBench. A reward can also run tests in a sandbox, as in Step 10. The reward runs on the rollout side after generation, so a slow reward function adds to perf/rollout_time.

Step 3: Schedule groups, not single answers

The Miles technical report defines four units, and the rest of the system uses them. A prompt is one task from the dataset. A trajectory is one attempt at that task. A group is all the trajectories for one prompt. A session is the ordered set of requests that one episode sends to the serving layer.

The group is the unit that matters for training. GRPO, the default advantage estimator, scores each answer against the other answers to the same prompt. The advantage is the reward minus the group mean. So Miles generates, stores, and drops data one group at a time. A single trajectory without its group has no advantage.

A group where every answer gets the same reward has zero advantage on every token. It costs generation time and teaches the model nothing. Miles can drop such groups with a filter function, for example check_reward_nonzero_std, passed through --dynamic-sampling-filter-path. It then samples extra prompts with --over-sampling-batch-size to fill the batch again.

Long answers create a second problem. One slow trajectory holds its whole group, and the group holds the batch. Partial rollout, turned on with --partial-rollout, stops unfinished samples and puts them back in a buffer for the next round. AMD reported a 40 percent throughput gain in synchronous mode from this method on its GPUs, in a post about slime. That number is a claim.

Miles offers five advantage estimators in the source: grpo, gspo, ppo, reinforce_plus_plus, and reinforce_plus_plus_baseline. slime has one more, cispo, which Miles does not have. So Miles is not a strict superset of slime.

Step 4: Share GPUs between the engine and the trainer

You place the two programs in one of two ways. In a colocated run, set with --colocate, the engines and the trainer use the same GPUs and take turns. In a disaggregated run, each side has its own pool of GPUs. A colocated run needs fewer GPUs. A disaggregated run can generate and train at the same time.

Taking turns needs a way to free GPU memory without losing state. SGLang does this with torch_memory_saver, behind the server flag --enable-memory-saver. Each allocation has a tag: kv_cache, weights, or cuda_graph. A release gives the physical memory back to the GPU but keeps the virtual addresses. When the memory comes back at the same addresses, the captured CUDA graphs still work, so the engine does not capture them again.

The release has three conditions in the SGLang v0.5.22 source. The server must be idle. A release of the KV cache also clears the prefix cache, because the cached tokens are gone. A release of the weights needs a copy somewhere. The flag --enable-weights-cpu-backup keeps one in host memory for runs where the trainer does not send the full weights again.

Miles controls what the engine releases with --offload-rollout-level. The default releases the KV cache and the weights, and Miles always releases the CUDA graphs too. A second flag, --colocate-memory-peak-device, decides where the two sides overlap during the handoff. The default is the CPU. The value gpu makes them overlap in GPU memory, and it needs both offload flags and a run without a critic.

Colocation also decides the weight transport. A colocated run always uses CUDA inter-process handles, which Step 6 covers. The flag that chooses a transport has no effect in that mode.

Step 5: Open a refit session

A refit is the moment the engines load new weights. In Miles, a refit is an HTTP session that the trainer's first rank runs against every engine. The order below comes from updater.py and session.py in the Miles source.

1. Pause generation with the chosen pause mode.
2. Clear the prefix cache, unless the pause mode is in_place.
3. Open a weight-update session with /begin_weight_update.
4. Send the weights in buckets through the chosen transport.
5. Close the session with /end_weight_update.
6. Set the new weight version on every engine.
7. Continue generation.

The pause mode decides what happens to requests that are still running. In abort mode, which is the default of the SGLang API, the server ends them. In retract mode, the Miles default, the server moves them back to the waiting queue. It computes their KV cache again after the update. In in_place mode, the server freezes them and keeps their KV cache, so tokens computed under the old weights continue under the new weights.

The session exists for quantized models. The SGLang documentation says the update calls write values in checkpoint format. The close call runs each quantized layer's post-load processing once, so the engine pays the repacking cost once per refit, not once per bucket. SGLang rejects update calls outside a session with HTTP 400.

I sent the same calls to SGLang 0.5.22 on my Mac, with the MLX backend and Qwen3-0.6B in 4-bit. The calls below ran in this order four times. The two /generate calls before and after the version change returned the same token ids, because the weights did not change.

curl -s localhost:30411/pause_generation -H 'Content-Type: application/json' -d '{"mode":"retract"}'
curl -s -X POST localhost:30411/flush_cache
curl -s localhost:30411/update_weight_version -H 'Content-Type: application/json' -d '{"new_version":"step-1","abort_all_requests":false}'
curl -s localhost:30411/continue_generation -H 'Content-Type: application/json' -d '{}'

The control calls worked and were fast. Pause took 1.8 to 3.9 ms, and the cache flush took 11.7 to 21.2 ms. The version change took 1.5 to 1.6 ms, and continue took 0.8 to 0.9 ms (Mac measurement). The calls that touch the weights did not work. The MLX backend keeps the real model in MLX and gives the PyTorch side a placeholder object. So /begin_weight_update stopped the scheduler with '_DummyModel' object has no attribute 'named_modules', and the client waited about 5 seconds before the connection dropped.

Figure 1 replays the session. Choose a pause mode and step through the seven calls. Each step shows the Mac result where I have one, and the error text where the call failed. The four running requests are an illustration of what each mode does. Their lengths are not measured.

Two details from the session matter later. First, SGLang records a weight version on every token. A request that runs through a refit returns meta_info.weight_versions with the span of tokens that each version produced, according to the SGLang documentation. Step 7 uses these versions to measure staleness. Second, the Mac result tells you something about Miles itself. Miles cannot load weights into an MLX engine, so there is no way to run its refit on Apple silicon today.

Step 6: Move the bytes

The transfer in the middle of the session moves the whole model, bucket by bucket. The flag --update-weight-transfer-mode chooses one of four transports, and broadcast is the default. Each bucket holds 512 MiB by default, set with --update-weight-buffer-size. The trainer first converts Megatron's layout to the Hugging Face tensor names that SGLang expects.

CUDA inter-process handles. A colocated run always uses this path. Each training rank packs its bucket into one flat buffer, a FlattenedTensorBucket from SGLang. It sends a handle to that GPU memory, not the bytes, to the engine through /update_weights_from_tensor. The engine reads the tensors where they already are. The trainer must keep the tensors alive until the engine confirms, so the code waits at a barrier.

NCCL broadcast. Miles builds one NCCL group for each pipeline stage of the trainer. The group holds one trainer rank and every GPU of every engine. For each bucket, the trainer sends the names, types, and shapes over HTTP, and then broadcasts the data. Every tensor-parallel rank of every engine receives the full tensor and keeps its own shard. A variant, broadcast_packed, sends each bucket as one flat byte buffer, for Megatron runs that do not share GPUs.

Peer-to-peer RDMA. With p2p, the trainer writes straight into the GPU memory of each engine rank through Mooncake. Each rank receives only its own shard. To compute those shards, the trainer builds a replica of each engine rank's layout on a meta device, which holds shapes but no data. Then it runs SGLang's own weight loaders into two pinned transfer buffers.

RadixArk measured the peer-to-peer path on H100 GPUs with 1 GB buckets. Update time fell 19 percent for Qwen3-30B-A3B and 86.4 percent for Kimi-K2 at one trillion parameters. The same document reports P2P as 68.6 percent slower for a 9B model on one node. These numbers are claims from the Miles documentation.

Disk delta. With disk-delta, the trainer compares the new weights with a CPU copy of the previous update. It writes only the changed bytes to shared storage, as an XOR of old and new by default.

Each engine then calls /pull_weights, applies the change to a checkpoint on its own disk, and reloads. RadixArk's v0.1 announcement says about 2 percent of parameters change per step for BF16 rollout. It also reports a GLM-4.7-Flash update falling from 62.4 GB to between 0.69 and 0.83 GB, with a pause of 3 to 5 seconds. Both are claims.

The disk-delta claim has a simple cause, so I tested it. The trainer keeps weights in FP32, but the engine receives BF16. BF16 keeps 8 bits of precision, so the gap between two BF16 values grows with the size of the value. AdamW moves each weight by about the learning rate, whatever its size. So with a small learning rate, most weights round back to the same BF16 value after a step.

I trained Qwen3-0.6B for three steps on the Mac CPU with AdamW, then cast each step to BF16 and compared it with the step before. The model has 596,049,920 parameters, which is 1.19 GB in BF16. The loss was a supervised loss on two math examples, not an RL reward. Treat the result as a test of the mechanism, not a copy of RadixArk's run.

At a learning rate of 1e-6, 1.34 to 1.39 percent of the BF16 values changed per step. The XOR file, compressed with zlib, was 2.9 percent of the full checkpoint. At 1e-5, 9.3 to 13.5 percent of the values changed. So the saving from disk delta depends on the learning rate (Mac measurement). Figure 2 shows where the changes happen.

Disk delta also changes the shape of the refit. Its protocol class turns off the weight-update session, so the trainer prepares and publishes the change outside the pause. The engines pause only to reload. That is how a large update can pause generation for seconds, not minutes.

Choose a transport by placement first. Shared GPUs leave one choice, CUDA handles. On separate GPUs, start with broadcast, because it needs only NCCL. Move to p2p for large models spread over many engine ranks, where broadcast sends every tensor to every rank. The P2P path rejects engines that use pipeline parallelism. Try disk-delta when the engines sit on other nodes without a fast link to the trainer, and the learning rate is small.

Two patterns show in the map. The layer norms almost never change. At 1e-6 and step 1, the input norm of every layer and the final norm changed no values at all. The attention and MLP projections change the most, at 1.4 to 1.9 percent on average. The k_norm vectors are the exception among the norms, at 1.1 percent on average, but each one holds only 128 values. The embedding table is the largest tensor, and 0.46 to 0.90 percent of it changed over the three steps.

Step 7: Keep the engines busy

Miles has three schedules. The synchronous loop in train.py generates, then trains, then updates, and each side waits for the other. The pipelined loop in train_async.py starts the next rollout while the trainer works on the current one. Its samples are one weight version old, and the code waits for that generation to finish before it updates the weights.

The third schedule is fully asynchronous, set with --fully-async. A background worker keeps a fixed number of trajectories in generation at all times. The trainer takes whatever groups have finished when it needs a batch. The fully async documentation and the technical report say that both sides then progress at once. The only engine pause is the refit itself.

The worker refills its slots by one of two rules. With group granularity, it starts a new group only when every trajectory of an old group has finished. With sample granularity, each finished trajectory frees its own slot at once. Sample granularity is the default in fully async mode, because one slow trajectory would otherwise block a whole group's share of the slots.

Finished groups go into a bounded buffer. Its capacity is a multiple of the training batch, and a full buffer makes the worker wait. The buffer drops a group for one of three reasons, and it checks each reason at a fixed point. A group that timed out in generation is dropped on arrival. A group that the user filter rejects is dropped on arrival. A group that is too stale is dropped when the trainer collects it.

Staleness has a strict definition in the report. It is the current weight version of the trainer minus the oldest weight version of any token in the group. The report calls this definition pessimistic on purpose, because a group is never fresher than its oldest token. The limit is --max-weight-staleness, and it has no default, so a run without the flag accepts any staleness.

Miles logs seven numbers for the buffer on every training step, under rollout/fully_async/. They are queue_size, avg_staleness, max_staleness, buffer_avg_staleness, buffer_max_staleness, aborted_groups_filtered, and stale_groups_filtered. The report gives one rule to read them. A queue size that stays at zero means the engines are too slow. A queue size that stays at capacity means the trainer is too slow, and staleness climbs with it.

The limit is a trade. A tight limit drops more finished groups, so the engines did that work for nothing. A loose limit trains on older data, and the correction methods in Step 9 must absorb a larger gap. Modal's run in Step 10 set no limit at all, and its average staleness was about 4 versions.

Figure 3 runs that rule set on simulated arrivals. The drop rules, the staleness definition, and the metric names are from Miles. The arrival times are generated from a fixed seed, so the same settings always give the same result.

Try three settings in the figure. Set the engine rate below the need, and the trainer starts to wait while the queue stays empty. Set it to three times the need, and the queue fills to capacity while staleness climbs. Then set a staleness limit, and watch stale_groups_filtered count the groups it throws away.

Two more facts complete the async picture. The report describes three ways to evaluate a model during an async run. The default uses the same engines and pauses production while it runs. The other two use a separate fleet, or an outside backend, with a checkpoint copy. Also, the report says a fully async run refuses to start when the trainer and the engines share GPUs. The main branch accepts that combination since October 14, 2026, so check which version you read about.

Pause mode interacts with async runs too. With in_place pausing and fully async generation, Miles turns on a prefix cache that keeps separate entries per rollout call by default. Without it, prefix KV computed under old weights could serve a later request.

Step 8: Keep multi-turn sessions on one engine

Agent tasks make the engine work harder. Each turn of a conversation repeats almost the whole history of the earlier turns. The engine that served the last turn already holds that history in its prefix cache. If the router sends the next turn to another engine, that engine must compute the whole history again.

Miles solves this with a session server that sits between the agent and the engines. The server holds the conversation and does the tokenization itself, and it keeps the exact token ids that the engines produced. So the trainer scores the same tokens that the model sampled. The Miles documentation calls this token-in-token-out. A source-level review from Runguo Li, comparing seven open post-training frameworks, found server-side tokenization in Miles alone.

The session server also gives each session a routing key. The router maps that key to one engine, and to one data-parallel rank when attention runs data-parallel. A new session goes to the engine with the fewest active requests, and then it stays there. A request without a key fails with an error. The report says a silent fallback to load-based routing would lower the cache hit rate with nothing in the logs to explain it.

The report credits affinity and least-loaded placement with a 96 percent prefix-cache hit rate in its GLM-5.2 run. That number is a claim. The router itself is the SGLang Model Gateway, built from a fork named sgl-router-for-miles. Its recent commits carry RL data through the router: routed experts, sampling masks, and up to 128 top log-probabilities per token.

Mixture-of-experts models add one more source of difference. The engine and the trainer can pick different experts for the same token. Rollout routing replay, set with --use-rollout-routing-replay, records the experts that the engine chose and replays them in the trainer. The record costs (tokens minus 1) × layers × top-k × 4 bytes per sample. For 32,768 tokens, 60 layers, and top-k 8, that is about 60 MiB per sample.

The tasks themselves run in sandboxes. The Miles documentation lists four sandbox providers, AgentENV, Daytona, E2B, and Modal, behind connectors such as Harbor and OpenEnv. RadixArk said on X that it joined the OpenEnv committee in June 2026.

Step 9: Measure the gap between sampler and trainer

The engine reports a probability for each token it samples. The trainer computes its own probability for the same token. In policy-gradient methods, the ratio of the two goes into the loss. If the engine and the trainer disagree, the update is biased, even with identical weights.

Many things make them disagree. Modal's post lists kernels, number precision, accelerator type, stale weights, expert routing, top-p sampling, and multi-turn context. A quantized engine on different hardware combines several of these at once.

Miles gives you two answers. The first answer is to remove the gap. True on-policy mode makes every operation bitwise equal on both sides. It uses batch-invariant kernels adapted from Thinking Machines Lab, FlashAttention 3 on both sides, and matching data types. The log key train/train_rollout_logprob_abs_diff then reads exactly zero. The source has one profile for this mode, for Qwen3 dense models only.

The second answer is to correct for the gap. Truncated importance sampling caps the ratio at a fixed value. Masked importance sampling keeps the ratio inside a range and sets it to zero outside. Score centering subtracts the expected gradient under the sampler at each position. The Miles flags for score centering with masked weights are below, from the Miles score-centering example with mis chosen.

--loss-type score_centering \
--advantage-estimator grpo \
--rollout-top-logprobs-num 128 \
--score-centering-is mis \
--rollout-temperature 1.0 \
--rollout-top-p 1.0 \
--rollout-top-k -1 \
--use-rollout-logprobs \
--disable-grpo-std-normalization \
--calculate-per-token-loss

The default masked range is 0.5 to 5, set with --score-centering-mis-low and --score-centering-mis-high. The example notes that this loss turns on an FP32 output layer in the engine, so sampling probabilities are not rounded before Miles uses them. That costs extra serving compute and memory.

To see the gap, I made one on the Mac. The sampler was Qwen3-0.6B in 4-bit on MLX, at temperature 1 with a fixed seed. It answered two math questions, 96 tokens each. The trainer was the BF16 version of the same model on the CPU, which scored the same 192 tokens in the same context.

The median ratio was 1.000, so most tokens agreed. The mean absolute difference in log-probability was 0.369, and the k3 estimate of KL from sampler to trainer was 0.218 per token. The range 0.5 to 5 masked 22 of 192 tokens, and the range 0.2 to 5 masked 12 (Mac measurement). Four-bit quantization of a small model gives a large gap, so read these numbers as a demonstration.

One token shows why masking helps. The question says Natalia sold half as many clips in May. The 4-bit sampler wrote "2 × 48" and gave the token "2" a probability of 0.52. The BF16 trainer gave "2" a probability of 0.0023 and preferred "½" at 0.81. The ratio is 0.0044, so a masked loss drops that token from the gradient. Figure 4 lets you inspect every token.

The two answers are not exclusive. A run can match some sources of difference and correct the rest. Modal's post argues that the growth of the gap matters more than its size, and that correction stops the growth. RadixArk's true on-policy work removes the gap where it can. Both approaches are flags in the same codebase.

Step 10: Run it on rented GPUs

On October 13, 2026, Modal published "Toward RL on mixed-precision, mixed-accelerator samplers" by Nan Jiang. The announcement thread on X had 1,657 likes on its first post three days later. The work uses Miles as the RL framework, SGLang engines as samplers, and a Megatron-LM trainer in BF16 on B200 GPUs. Every number in this step is Modal's claim.

Modal trained Qwen3.6-35B-A3B on agentic coding with up to 256K tokens of context. The tasks were 2.7K repository tasks from Xiaomi's MiMo-V2.6 release, with the repository history removed. In each episode, the model wrote a patch over many turns and submitted it in a Modal Sandbox. The reward was 1 if the patch passed the tests in a fresh sandbox, and 0 otherwise.

The run was fully asynchronous with no staleness limit, and each update used 128 prompts with 8 trajectories each. Modal compared two sampler fleets. The mixed fleet used six kinds of accelerators. B200 and B300 served NVFP4 weights, and H100 and H200 served FP8 and BF16 weights. A100 and RTX PRO 6000 served BF16 weights. The second fleet used only B200 GPUs in BF16, which matches the trainer.

Plain GRPO failed on both fleets. The training reward fell, the gap between sampler and trainer grew three times or more, and most episodes ended on repeated format errors. At the step-40 checkpoint on the mixed fleet, 62 percent of evaluation episodes ended this way. The fleet that matched the trainer started with a gap four times smaller, but it failed in the same way from about step 100.

Score centering with masked importance sampling trained to the end on both fleets. With top-p mask replay added, the mixed fleet matched the matched fleet on SWE-bench Pro V2 pass@4, with an average gap 4.7 times larger. On the hard subset, pass@4 rose over about 100 to 140 steps and then held at about 65 to 75 percent. The sampling cost per update was about 15 percent lower on the mixed fleet for most of training.

The post does not publish the GPU count, the node count, the training time, or the total cost. It also says the setup was not tuned for cost. The algorithm is in upstream Miles, as the flags in Step 9 show. The mixed fleet is Modal's own infrastructure.

Modal released the fleet layer as Stitch, a control plane for rollout engines that run apart from the trainer. The trainer publishes numbered policy versions to a store. Each rollout engine catches up to the newest version on its own. A request can ask for a minimum version or an exact version, and an engine that cannot serve it answers with a retryable 409. The repository has no license file, so check with Modal before you reuse its code.

Stitch does not run stock Miles. Its integration notes pin a Modal branch of Miles that adds several changes that are not merged upstream, plus a Modal sandbox adapter. Stitch also publishes its own update timings for GLM-5.2 on B300 GPUs. A full checkpoint loaded from disk paused the engine for 171.69 seconds. A delta prepared as per-rank images in host memory paused it for 2.91 seconds. These are Stitch's claims.

The smallest Stitch recipe trains Qwen3 4B on math with two H200 trainer GPUs and one H200 rollout replica. It runs 120 rollouts of 16 prompts with 8 samples each. At Modal's list price of $0.001261 per second for an H200, three GPUs cost $13.62 per hour. The recipe does not state a run time, so I cannot give a total cost.

Step 11: Know which SGLang you run

Miles does not run stock SGLang. Its version notes say SGLang comes from the sglang-miles branch of the SGLang repository. The Dockerfile builds from the v0.5.21 image with that branch on top. The notes say that installing stock SGLang or upstream Megatron-LM next to Miles does not work.

On October 16, 2026, the GitHub compare API put sglang-miles 46 commits ahead of v0.5.21 and 0 commits behind. The first 32 commits are a numbered patch series. It covers true on-policy math, routing replay, disaggregation for RL, LoRA for mixture-of-experts models, peer-to-peer weight mapping, and the /pull_weights endpoint. The repository has 35 branches whose names start with sglang-miles, several of them for one new model each.

I listed every endpoint that the Miles HTTP client calls and searched for each route in the SGLang v0.5.22 server file. 21 of the 25 names exist in v0.5.22. The table shows the ones that matter for a refit, with my Mac result for each one.

The fourth name, load_lora_adapter_from_distributed, is not a route string in the server file of either branch. So I could not find where SGLang registers it. Miles also asks for /get_weight_version when /model_info fails, as a fallback for older SGLang releases.

Some of the patches reach upstream over time. The refit session and the per-token weight versions are in the stock v0.5.22 documentation, for example. The disk-delta pull endpoint and the peer-to-peer layout endpoint are not. So the gap between the two branches moves with each release.

The practical rule is short. Run the Miles container image, and read the SGLang version from it, not from the SGLang release notes. A feature in a new SGLang release reaches Miles only when the sglang-miles branch moves to that release.

Step 12: Start with the smallest run

The Miles installation guide lists the supported hardware. GB300, GB200, B300, and B200 are marked production. H200 and H100 are marked production with CI tests. A100 is supported with the FP8 features turned off. AMD MI300X, MI325, MI350X, and MI355X run through ROCm, which AMD announced with the Miles team in March 2026. Runs across nodes need InfiniBand, RoCEv2, or Slingshot at 200 GB/s or more per node.

There is no path for Apple silicon. I tried to run the CPU test suite on the Mac. The tests import SGLang, SGLang 0.5.22 imports Triton when it loads, and no macOS package of SGLang 0.5.22 exists. The Mac runs in this guide used SGLang alone, or plain PyTorch.

The quick start needs one node with 8 GPUs, 500 GB of free disk, and Docker with GPU access. It trains Qwen3-4B with GRPO on 17K math problems and evaluates on AIME 2024. Pull the image on the host. Inside the container, update Miles, download the model and data, and convert the weights to Megatron's format. Then one command starts the run.

docker pull radixark/miles:latest
cd /root/miles && git pull && pip install -e . --no-deps
python scripts/run_qwen3_dense.py --model-name Qwen3-4B

The launcher starts a local Ray cluster, saves a checkpoint every 20 rollouts, and evaluates every 20 rollouts. If the run stops, the same command resumes from the last checkpoint. Two flags help when something is wrong. --debug-rollout-only runs the engines without training, and --debug-train-only trains without the engines.

Read the first log lines before you change anything. perf/rollout_time against perf/actor_train_time tells you which side is slower. If generation dominates, more engines help more than a faster trainer does.

Step 13: Scale to larger models

Miles has three training backends, chosen with --train-backend: megatron, the default, fsdp, and torchtitan. slime supports Megatron only. Megatron covers the parallelism that large mixture-of-experts models need, such as tensor, pipeline, context, and expert parallelism. FSDP reads Hugging Face checkpoints directly, which helps with smaller models.

Beyond full-parameter RL, the technical report covers LoRA RL, on-policy distillation, supervised fine-tuning, and vision-language models. LoRA runs can hold several adapters, and an experimental server exposes a Tinker-compatible API on top of them. A separate project, Miles-diffusion, applies the same loop to image and video diffusion models.

LoRA changes the refit. A run that trains only adapters sends only the adapter bytes, and the base weights on the engine stay where they are. The engine registers each adapter with /register_lora_adapter, an endpoint from the sglang-miles branch, before any bytes move. If the engine released its weights during training, the base weights come back from the CPU backup in Step 4. The trainer does not send them again.

The report ends with one large run. It trained GLM-5.2, a 744B model with 40B active parameters, on terminal-use coding tasks with fully async RL. The run used 64 NVIDIA GB300 GPUs, and the median step took 263 seconds over the first 30 measured steps. These are RadixArk's numbers.

RadixArk also lists models that it says Miles supports on release day, including Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, and MiniMax H3. That list is a claim. The per-model sglang-miles branches in Step 11 are the code side of that work.

When to choose Miles

Choose Miles when your rollout engine is SGLang and your model is a large mixture of experts. Choose it when your tasks are long, multi-turn, and agentic, so async generation and session affinity pay off. Choose it when you need a fixed refit protocol with several transports and version stamps on every token. The slime project is the smaller codebase with the same core loop, and it fits teams that run only Megatron.

Know the limits before you start. Miles needs NVIDIA or AMD GPUs, and it pins its own SGLang branch. True on-policy mode covers one model family today. The user list and most speed numbers come from RadixArk, and I found no independent benchmark of Miles against other frameworks.

Whatever you choose, keep the habits from this guide. Log the weight version with every sample. Watch queue_size to find the slow side. Measure the gap between sampler and trainer before you trust a reward curve. Read the SGLang version from the container you run.