You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f0fbbbe
Browse filesBrowse the repository at this point in the historyBrowse files
Say at the top of both READMEs that the model can also run on vLLM
The opening list still described only transformers on your own machine.
- Models: the model also runs on vLLM, in Docker started from the sidebar's
What runs the model, or on a server of your own at VLLM_URL
- MTP speculative decoding: on vLLM with a GPU both models decode
speculatively (Qwen3.5 with its own MTP layer), by default in Docker
- The paragraph below: the token details come from your machine or from
what the vLLM server sends back, which depends on its flags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,14 +8,14 @@ English | [繁體中文](README.zh-TW.md)
8
8
9
9
This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. A chat app shows only the question and the answer; this page draws every step as it happens, down to each token the model writes, so the whole mechanism is laid out in front of you. Each part of the page matches a section of the Unit 01 reading of [Agent Atlas](https://cyyeh.github.io/agent-atlas-learning/) (in Traditional Chinese), the course this project belongs to, and [Experiments to try](#experiments-to-try) turns those ideas into things you can try yourself.
10
10
11
-
- Models: pick [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) or [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) in the sidebar; they run on your machine with Hugging Face `transformers`
12
-
- MTP speculative decoding: for Gemma 4 E2B you can switch it on in the sidebar (it is off by default); it uses [google/gemma-4-E2B-it-assistant](https://huggingface.co/google/gemma-4-E2B-it-assistant) as the drafter, so one forward pass produces several tokens
11
+
- Models: pick [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) or [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) in the sidebar; they run on your machine with Hugging Face `transformers`, or on [vLLM](https://docs.vllm.ai/): pick vLLM (Docker) under What runs the model in the sidebar and the page starts a vLLM server in Docker for you, on the GPU if there is an NVIDIA one and on the CPU otherwise, or set `VLLM_URL` to use a server of your own, on this machine or on a Linux machine with a GPU ([Running the model on a vLLM server](#running-the-model-on-a-vllm-server))
12
+
- MTP speculative decoding: one forward pass produces several tokens. With `transformers`you can switch it on in the sidebar for Gemma 4 E2B (it is off by default), with [google/gemma-4-E2B-it-assistant](https://huggingface.co/google/gemma-4-E2B-it-assistant) as the drafter; on vLLM with a GPU both models decode speculatively, Gemma 4 E2B with that drafter and Qwen3.5 with the MTP layer in its own checkpoint, and vLLM in Docker does so by default
13
13
- Files: attach images and videos (both models) and sound (Gemma 4 E2B, which also takes a recording from the microphone) with the + in the input box, or pick one of the sample files above it (a receipt, a recording, a video, sales data) and ask your own question about it; each becomes a run of tokens in the context, and the page shows them
14
14
- Writing code (optional): Let the model write code in the sidebar gives the model a `run_python` tool, which runs its Python in a Docker sandbox, reads the data files you attach (CSV, TSV, JSON, text, Markdown, Excel) and draws charts with matplotlib, which appear in the chat. Docker is needed for this alone
15
15
- Languages: the page and the How it works guide are in English and Traditional Chinese. They start in your browser's language, the switch at the top right changes it, and the language you pick is remembered for your next visit
16
16
- License: [Apache License 2.0](LICENSE)
17
17
18
-
On the left is a chat like ChatGPT's; on the right the agent loop is drawn step by step: the model reads the context and proposes an action, the harness carries it out, and the result goes back into the context, until it stops. The model runs on your own computer, so every token's probability, the candidate tokens, the special tokens, the prefill and the generation speed are all in view. The models run the same harness, so switching models changes only their text formats, which makes them easy to compare.
18
+
On the left is a chat like ChatGPT's; on the right the agent loop is drawn step by step: the model reads the context and proposes an action, the harness carries it out, and the result goes back into the context, until it stops. The model runs on your own computer, or on a vLLM server that sends back each token's log probabilities, so every token's probability, the candidate tokens, the special tokens, the prefill and the generation speed are all in view (on vLLM, as much of it as the server's flags let it send). The models run the same harness, so switching models changes only their text formats, which makes them easy to compare.
19
19
20
20
[How it works](https://cyyeh.github.io/agent-loop-visualizer/en/) maps each step of one turn, from sending a question to the reply, to the code that does it, with file names, line numbers and excerpts. It lives on GitHub Pages; How it works at the top of the app opens it in a new tab, in the language the app is in.
0 commit comments