ruff and ty, vendored (a folder in this repo, not a git dependency). It is a copy with one small patch.
- Upstream URL: https://github.com/astral-sh/ruff
- Pinned commit / version:
ac201b86aa37da5140ed716ebd4eca36d1fe32da(2026-09-02, ruff 0.16.5). Vendored on 2026-11-11, byte for byte from pylspt's commit670ae79a15(git archive 670ae79a15 vendor), not from upstream again. The pylspt hash is the one to read when asking "what did we start from". - Engine crates used: the 26 crates in
vendor/ruff/crates/(the closure ofty_projectandty_python_semantic).python-syntax(fast):ruff_python_parseronly.python-analysis(slow, PY5): the ty crates throughsem.rs. - Patches: one, see below.
- How to bump: see below. An engine bump is a change of its own.
- Licence notice: ruff and ty are MIT, Copyright (c) 2022 Charles Marsh and the ruff contributors (
vendor/ruff/LICENSE).ruff_annotate_snippetscarries its ownLICENSE-APACHEandLICENSE-MIT, and the bundled typeshed hasty_vendored/vendor/typeshed/LICENSE. All three stay in the tree unchanged. A binary that linkspython-analysisembeds typeshed.
The copy is the commit's vendor/ tree minus four directories of test data (42 MB to 23 MB, 3,733 files to 1,747). Nothing else was removed or
edited by the copy commit (git diff of the copy commit against a plain git archive shows exactly these):
| Removed | What it was |
|---|---|
crates/ruff_python_parser/resources/ |
the parser's valid / invalid / inline test programs |
crates/ruff_python_parser/tests/snapshots/ |
the parser's insta snapshots |
crates/ty_python_semantic/resources/mdtest/ |
ty's Markdown tests |
crates/ty_python_semantic/resources/corpus/ |
ty's corpus test programs |
Kept on purpose:
- every
tests/*.rs. The parser's[[test]] fixturesand ty's[[test]] mdtesthave nopath, so Cargo looks fortests/fixtures.rsandtests/mdtest.rswhen it reads the manifest; crates/ty_python_semantic/resources/lint_docs/: 132#[doc = include_str!("../resources/lint_docs/…")]attributes (one per lint) read it at build time. It looks like test data and is not.
The vendored crates' own tests are not run here (their fixtures are gone). The proof that the engine works is our tests, not ruff's.
Rust privacy is why there is one: ty's most useful functions are pub(crate), so no code outside the crate can call them.
| Edit | Where | Why |
|---|---|---|
| 1. NEW, 203 lines | crates/ty_python_semantic/src/types/analyzer_ext.rs |
pub wrappers over private ty accessors (MRO, member lookup, the callable's parts, type arguments). Its header states the rules: one wrapper per need, one internal call each, only public types in signatures |
| 2. EDIT, one line | crates/ty_python_semantic/src/types.rs |
pub mod analyzer_ext;, next to the other pub mod lines |
It was types/vnoc.rs in pylspt. The rename is the second PY1 commit, so git diff <copy commit> <rename commit> -- languages/python/vendor is
the patch and nothing else. The compiler catches a wrapper whose signature broke after a bump; only a test catches one whose behaviour
changed, so PY5 adds one test per wrapper. They are in crates/python-analysis/src/sem.rs (attrs_tests), not in a file of tests/: the repo's rule 4 says only sem.rs names the engine, and these tests must. Where each is: class_mro (the mro tests), member and instance_of (member_*), widen, type_of_definition, bound_method_self, type_args and callable_parts (one test each, on tests/fixtures/classes/wrappers.py). callable_definition is tested only for its None cases: its positive case, a callable ty made from a function such as property(f), is exercised only by the fixture programs, so a changed answer there would show in the 41 tests and not in a direct one. A bump re-runs all of them first.
When python-analysis starts depending on the engine (PY5), its entries in [workspace.dependencies] come from pylspt's Cargo.toml at 670ae79a15
(git show 670ae79a15:Cargo.toml). Two comments there must come with them, because the failure is silent:
- salsa: copy the
salsaline (andrustc-hash,thin-vec,uuid,serde,serde_json,anyhow,smallvec) fromvendor/ruff/Cargo.toml, never retype it, so Cargo's feature unification keeps exactly one salsa in the graph. Two copies are two incompatibleDbtraits. - zstd:
ty_projectisdefault-features = false, features = ["zstd"].default-features = falseis what ruff's own root table says, and it is safe there only because ruff'stybinary re-enables zstd. We are that binary, so we re-enable it. Without it, decompressing the vendored typeshed breaks silently.
ruff_db is default-features = false, features = ["os"].
Cargo.lock started as pylspt's, at 670ae79a15, so that the only thing that changed is our code. Cargo drops the entries nothing depends on yet:
pylspt's has 287 packages, this one has 90 (the parser's closure plus our five crates). Every one of the 90 third-party packages has the version pylspt's lock
has.
At PY5, in this order, so that no version is chosen again:
- Put pylspt's lock back first:
git -C ../pylspt show 670ae79a15:Cargo.lock > Cargo.lock(read by hash, never the working tree). - Add the ty dependencies to
[workspace.dependencies]and build. Cargo keeps every locked version that still satisfies the requirements, and only adds our own crates and drops what is unused. - Compare the packages of the new lock with pylspt's: any third-party package whose version differs is a drift to explain, not to accept. The comparison should come out empty.
If the order is reversed (add the dependencies to the 90-package lock), Cargo has no pins for the ~197 dropped packages and takes the newest compatible version of each, so drift becomes the expected result.
ProjectMetadata::discover(dir, system) (ty_project/src/metadata.rs, discover_with_uv_workspace) walks up from dir through every
parent folder and takes as the root, in this order: the closest folder with a ty.toml or a pyproject.toml that has [tool.ty]; the uv
workspace root, if uv is on; the closest folder with any pyproject.toml; else dir. A config in a parent therefore moves the root, and
Project::file("app.py") then looks under the parent.
What open_project (sem.rs) does: it calls discover, and uses the answer only when metadata.root() == dir. When the root is a parent,
or discover fails, it uses ProjectMetadata::new(<name of dir>, dir): the project discover itself returns when no folder has a config.
- Spelling. Checked in R1:
root()is one of theancestors()of thedirgiven, so it is cut from the same text and is equal to it when it isdir(SystemPathcompares by components, so a trailing separator does not matter either). A parent is never equal to the longer path, so nocanonicalizeis needed inopen_project.Project::openstill canonicalizes once, before it. - An empty
ty.tomlis a valid config and makes its folder the root (tests/fixtures/09-root/above-ty: red before the fix). - Known limit.
dirhas apyproject.tomlwithout[tool.ty]and a parent has aty.toml(or a[tool.ty]pyproject): the engine prefers the parent, we fall back to the default project, anddir'srequires-pythonis not read. No public function reads one folder only (discover_inis private). Rare. - Known limit. A virtual environment in a parent folder (
<repo>/.venv, withservices/apiopened) is no longer found: library types do not resolve. R1b is the step that gives it back. - uv.
UseUv::from_system(ty_project/src/uv.rs) reads the variableTY_UV:1ortrueis on,scriptsis scripts only, anything else (and unset) is off. Off,discoverruns nouvprocess. On, it runsuvonce perdiscover, and the V-NOC driver opens the project on every request. A uv workspace root abovediris a parent root, so it is ignored like any other. Read, not run.
Measured 2026-11-12 at 33d65ca, debug build (the dev profile: opt-level = 1 for this workspace's crates, opt-level = 3 for every
dependency, so the vendored engine is optimized and our own frames are not). A temporary #[ignore] test in the sem module
(docs/05-python-migration/11-driver-needs.md §7, PY9.0), run from the built test binary, each run twice and the second run kept (the OS
file cache warm); deleted after. Times in ms. "Jarvis" is CallGraphGenerator.do_pass in jarvis.py; "post" and "get_files" are
FilesResource.post and get_files in app/api/file.py of the Flask service. One request is call_wire(&project, file, &entry, &Options::default(), false).
(a) Cold: a new Project for each request, as the V-NOC driver does today.
| Entry | open 1 / 2 / 3 | solve 1 / 2 / 3 |
|---|---|---|
| Jarvis | 1.3 / 0.8 / 0.8 | 88.9 / 85.9 / 84.5 |
| post | 1.2 / 0.9 / 0.9 | 185.7 / 174.5 / 173.4 |
| get_files | 1.4 / 0.9 / 0.9 | 107.4 / 94.0 / 119.5 |
(b) Warm: one Project, three requests. The three answers are equal (the serialized frames compared byte for byte).
| Entry | solve 1 / 2 / 3 | wire bytes | cold (median) ÷ warm (2 and 3) |
|---|---|---|---|
| Jarvis | 85.6 / 5.8 / 5.6 | 99 837 | ≈ 15× |
| post | 229.0 / 9.1 / 9.9 | 131 379 | ≈ 18× |
| get_files | 101.2 / 2.6 / 2.5 | 5 300 | ≈ 37× |
The first request on a kept project costs what a cold one does: what is saved is the engine's memoized work (parsing, imports, types), and that exists only after one request.
(c) The refresh, nothing changed: Files::sync_all on the warm project of (b), three times, then one more request.
| Entry | sync_all 1 / 2 / 3 |
the request after it | its answer |
|---|---|---|---|
| Jarvis | 0.10 / 0.05 / 0.05 | 5.7 | equal to (b) |
| post | 2.21 / 1.33 / 1.28 | 9.3 | equal to (b) |
| get_files | 2.30 / 1.63 / 1.50 | 2.8 | equal to (b) |
The project stays warm after sync_all. It costs more on the Flask service, which has more files the engine has looked up (its cost is one
stat per path the engine knows, ruff_db/src/files.rs); the number of paths was not printed.
(c′) The refresh after one change, on a copy of tests/fixtures/06-wire/python/ in the temp folder: ask main of
ids_and_classes.py, rewrite the file so that unmarked calls a new added, ask again without sync_all, then sync_all and ask.
The same in all three runs (the largest numbers shown).
| Step | time | added in the answer |
|---|---|---|
| first request (a fresh project: builtins and the file are read) | 20.5 | no |
the file rewritten, no sync_all |
0.1 | no: the old answer, from memory |
sync_all |
0.05 | |
| the request after it | 0.4 | yes |
So the engine never reads the disk again on its own: without a refresh a kept project answers from the old text. With sync_all, only what
read the changed file is recomputed (0.4 ms, against 20 ms for the first request). A file created after the first request was not tried here
(PY9.2 (b), 11-driver-needs.md §10 claim 2).
Through the CLI (PY9.4). python-cli calls <dir> <file> --entry <entry> --wire --timing --repeat 3, the debug binary, run twice on 2026-11-13
at the commit of PY9.4 (the second run is in the second line of each cell; refresh is Project::refresh, which includes sync_all). Times in
ms; the three answers were equal (the CLI checks it and exits 1 if not).
| Entry | refresh 1 / 2 / 3 | solve 1 / 2 / 3 |
|---|---|---|
| Jarvis | 0.04 / 0.16 / 0.13; 0.03 / 0.16 / 0.12 | 128.4 / 5.8 / 5.8; 88.5 / 5.7 / 5.5 |
| post | 0.05 / 2.19 / 2.19; 0.05 / 2.20 / 2.31 | 279.3 / 9.3 / 9.5; 167.7 / 9.7 / 9.0 |
They agree with (b) and (c) (solve 5.6–5.8 and 9.1–9.9). Refresh 1 is before the first request, when the engine knows no file yet, so it is the cheapest; later ones are about as (c) says (Jarvis a little over it, 0.12–0.16 against 0.05–0.10).
Memory. Resident size from ps inside the test after (b), and the peak of the whole run from /usr/bin/time -l. The test starts at
5.8 MiB. The size after (b) is an upper bound of one warm project: the three cold projects of (a) were dropped before it, and the allocator
may keep their pages.
| Entry | resident after (b) | peak of the run |
|---|---|---|
| Jarvis | 60.0 MiB | 60.6 MiB |
| post | 78.4 MiB | 87.5 MiB |
| get_files | 61.5 MiB | 62.5 MiB |
The stack. The built python-cli calls <dir> <file> --entry <entry> on the main thread, its stack set with ulimit -s in a subshell; an
overflow ends it with exit 134 ("has overflowed its stack"). The chain programs are def f0(): f1() … def fN(): pass, def main(): f0(), in
a temp folder; the 256-level one reaches max_stack (the CLI prints cut max_depth: 1, core's name for Cut::Stack).
| Program | fails at | passes at |
|---|---|---|
chain of 256 levels (to max_stack) |
2048 KB | 2052 KB |
| chain of 128 levels | 1072 KB | 1088 KB |
| Jarvis | 192 KB | 224 KB |
| post | 272 KB | 288 KB |
| get_files | 128 KB | 256 KB (not narrowed) |
One level of the solver's recursion takes about 7.5 KB in this build ((2052 − 1088) ÷ 128); 2052 ÷ 256 ≈ 8.0 KB is the upper bound with
the fixed part included. A request that reaches max_stack needs just over 2 MiB, so it overflows a thread with Rust's default 2 MiB
(std::thread::spawn), and an overflow aborts the whole process, not the thread. The driver asks for 8 MiB
(std::thread::Builder::stack_size(8 << 20)): 2 MiB rounded up to the next power of two is 4 MiB, and doubled for a margin, because a level of
real code (nested expressions in a body) can be larger than a one-line body's. A release build uses less.
The longest stretch without a check (11-driver-needs.md §10 claim 5). A temporary timer in Solver::cost_limit (reverted) kept the
longest time between two calls of it in one request, and the time from the last call to the end of call_wire (the render and the wire,
which no check covers). These runs were about twice as slow overall as the ones above (the machine was busier), so the numbers are high.
| Entry | longest gap, cold (4 requests) | longest gap, warm | after the last check, cold / warm |
|---|---|---|---|
| Jarvis | 34.4 / 35.0 / 34.8 / 95.6 | 1.05 / 1.77 | ≤ 36.1 / ≤ 11.7 |
| post | 14.0 / 14.7 / 63.1 / 13.0 | 0.32 / 0.31 | ≤ 27.4 / ≤ 6.7 |
| get_files | 4.6 / 4.8 / 4.6 / 4.9 | 0.29 / 0.28 | ≤ 2.6 / ≤ 1.7 |
A cancel checked in cost_limit would be seen within about 0.1 s at worst on a cold project, and within 2 ms on a warm one, plus the render.
Project::environment_inputs lists the files whose change means "answers may differ" (docs/05-python-migration/11-driver-needs.md §5).
The environment among them is the one the engine uses, found with the engine's own functions (paths in vendor/ruff/crates/):
- The config's
environment.python, if any (ty_project/src/metadata/options.rsto_program_settings, line 214 is theOk(None)arm that falls through to step 2).MergedOptions::python_environment(metadata.rs:654, public) makes the path absolute withRelativePathBuf::absolute(metadata/value.rs:65: against the config's folder,~and$VARexpanded) and callsPythonEnvironment::new(options.rs:315-340). We call that function, not a copy. Ifnewfails (nopyvenv.cfg, a bad one), the engine continues without an environment; we still list the configured path, made absolute the same way. - Else
PythonEnvironment::discover(Some(root), system)(ty_site_packages/src/lib.rs:286):VIRTUAL_ENV(299), a childCONDA_PREFIX(308),<root>/.venv(315), a base conda (335), thenpython3/pythononPATH(340). We list the result only if<sys_prefix>/pyvenv.cfgis a file: a system Python has none. sys_prefix()(lib.rs:374) is canonical, so an entry outside the root is the canonical path (on macOS/private/var/..., not/var/...).
.python-version is not read. grep for .python-version as a file name in ty_project, ty_site_packages, ty_python_semantic,
ty_module_resolver, ruff_db finds nothing: the only hits are the TOML key environment.python-version in messages and tests. The Python
version comes from that key, from requires-python, or from the environment's pyvenv.cfg.
TY_UV is off unless set (ty_project/src/uv.rs:49: only 1, true or scripts turn it on). discover takes the setting from the
process environment, and open_project accepts no root but the one asked for, so D26 holds either way. VIRTUAL_ENV, CONDA_PREFIX,
PATH and TY_UV are variables of the driver's process, fixed for its life: not inputs.
The list is computed at open and at each reopen, not per request: discover looks python3 up on PATH.
The engine writes "found past a base it cannot see" as an intersection (Type::Intersection) whose positive parts are the def it found and
Unknown: Model.find, with class Model(Hidden, Mixin) and Hidden an unresolved import, is
(bound method <class 'Model'>.find() -> Unknown) & Unknown (R2b test, sem.rs). Sem::likely_def reads it with
IntersectionType::iter_positive, which is public (ty_python_semantic::types); nothing is added to the copy. iter_negative is
pub(crate), so negative parts cannot be read from here: they are ignored. A method the class defines itself, before the unknown base,
is typed exactly (bound method Model.own() -> Unknown).
Measured 2026-11-13 with the debug CLI, before = 99c60b3 (CS2a) and after = CS2b, the second of two runs, --graph --timing; nodes are the
unmerged wire's. Jarvis is CallGraphGenerator.do_pass, Flask is FilesResource.post in app/api/file.py, FastAPI is APIRouter.include_router
in fastapi/routing.py.
| Project | nodes | contexts | walks | unresolved | ref_unresolved |
solve ms |
|---|---|---|---|---|---|---|
| Jarvis | 207 → 207 | 55 → 55 | 55 → 55 | 30 → 30 | 0 → 0 | 89 → 88 |
| Flask service | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 3 → 0 | 170 → 170 |
| FastAPI | 436 → 458 | 68 → 70 | 68 → 70 | 27 → 29 | 14 → 0 | 138 → 132 |
The golden of Jarvis is equal. All 22 nodes FastAPI gains are one getter, fastapi._compat.v2.ModelField.type_, under callers that were already in
the tree: 20 under v1.is_pv1_scalar_field and v1.is_pv1_scalar_sequence_field, 2 directly under analyze_param.
Inline mode draws them too. PYTHON_DUMP_GAP=1 on this entry (the dump_graph test, which has inline mode) gives summary 458 nodes, inline 1019,
0 extra: every node summary mode draws, these 22 included, is in inline mode's tree. Both modes draw them for the same reason, a branch the walk does
not narrow:
fastapi/utils.pycreate_model_fieldreturns a v1ModelField(line 113) or, underelif PYDANTIC_V2, a v2 one (line 123), so the field the walk holds is either.fastapi/_compat/main.py:187-192is_scalar_fieldsends it tov1.is_scalar_fieldunderisinstance(field, may_v1.ModelField). The walk does not narrow byisinstance, so the v2 object goes down that branch too, to_compat/v1.py:282andis_pv1_scalar_field, whose readsfield.type_(v1.py:201-204) are calls of the property at_compat/v2.py:88-89.- The 2 under
analyze_paramarefield.type_atdependencies/utils.py:555, a real path when pydantic v2 is installed.
So the 20 are edges Python does not take (a v2 field never reaches v1.is_pv1_scalar_field), and both modes agree on them. They are not new
imprecision: the v1.is_scalar_field caller was already in the tree before CS2b. They are what "no narrowing" costs.
Measured 2026-11-13 at 9c22dec plus the gate, with the dump_graph test (PYTHON_DUMP_GAP=1, with and without PYTHON_DUMP_EFFECTS=1; the
debug test binary, two runs each, the second one given; the request path, python-cli, still has effects off). Same entries as CS2: Jarvis
CallGraphGenerator.do_pass, Flask FilesResource.post in app/api/file.py, FastAPI APIRouter.include_router in fastapi/routing.py. Nodes are
the unmerged tree's; missing and extra are against inline mode's tree (compare_with_inline).
| Project | nodes (inline) | contexts | walks | unresolved | solve ms | missing | extra | MaxRounds cuts |
|---|---|---|---|---|---|---|---|---|
| Jarvis, effects off | 207 (207) | 55 | 55 | 30 | 79 | 0 | 0 | 0 |
| Jarvis, on | 207 (207) | 71 | 71 | 32 | 122 | 0 | 0 | 0 |
| Flask service, off | 266 (307) | 70 | 70 | 81 | 297 | 41 | 0 | 0 |
| Flask service, on | 266 (307) | 70 | 70 | 81 | 174 | 41 | 0 | 0 |
| FastAPI, off | 458 (1019) | 70 | 70 | 29 | 182 | 561 | 0 | 0 |
| FastAPI, on | 1019 (1019) | 104 | 104 | 52 | 175 | 0 | 0 | 1 |
Time is noise at this size: the two runs of one case differ by up to 40% in either direction (Jarvis off 109 and 79, on 83 and 122), and nothing about effects makes a solve slower that these runs can see.
- FastAPI is complete against inline mode with effects on: 1019 nodes, 0 missing, 0 extra. The spike's 436 → 970 (CS0) and CS2's 458 are
now 1019, and the 22 nodes of CS2's
v2.ModelField.type_(a branch the walk does not narrow, README §8) are in inline mode's tree too, so they are notextra. - Jarvis gains no node and loses none; its golden is
equal(the request path has effects off). Two more unresolved sites, both stdlib calls the engine cannot type,importlib.machinery.all_suffixes()andimportlib.machinery.FileFinder.path_hook(...)atmachinery/imports.py:262-263: a function the walk now reaches, because__init__'s objects are known. 16 more contexts, and the same 207 nodes: keys split on what objects hold, and the renderer joins the siblings. - Flask does not move at all (87 effects applied, same tree): its 41 missing nodes are the gap it had, 1 topmost, silent (not at an
Unresolvedsite), unchanged by effects. - One group is cut:
fastapi.utils.create_cloned_fieldisIncomplete(MaxRounds)(32 rounds): it clones a field and itssub_fieldsrecursively, so every round builds objects the round before did not name (README §8 "an object does not keep its identity around a cycle": CS8). It is marked and counted; effects off had no such cut. Not caused by CS3d (the same cut with the effects comparison left out).
Measured 2026-11-13, the same entries and the same method as CS3e (dump_graph, PYTHON_DUMP_GAP=1, debug test binary), before = a28baf1, after =
the row 11 change. The key of the walked closure reads its free values at depth 0, as its parameters (key.rs, key); a closure passed as a value is
unchanged (free values one below its parameter), and so are a closure's defaults (depth 1). needs_defer reads the same depths (call.rs,
closure_reaches_pending).
| Project | effects | nodes | contexts | walks | unresolved | extra |
effects_applied |
|---|---|---|---|---|---|---|---|
| Jarvis | off | 207 → 207 | 55 → 55 | 55 → 55 | 30 → 30 | 0 → 0 | - |
| Jarvis | on | 207 → 207 | 71 → 71 | 71 → 71 | 32 → 32 | 0 → 0 | 43 → 43 |
| Flask service | off | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | - |
| Flask service | on | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | 87 → 87 |
| FastAPI | off | 458 → 458 | 70 → 70 | 70 → 70 | 29 → 29 | 0 → 0 | - |
| FastAPI | on | 1019 → 1019 | 104 → 104 | 104 → 104 | 52 → 52 | 0 → 0 | 180 → 180 |
Nothing moves on the three projects (missing is also unchanged: Flask 41, FastAPI off 561, the rest 0). Why was not looked into: the numbers
are equal, and that is all this run says. More contexts, the cost to watch, is 0. The golden of Jarvis is equal (wire and merged wire, the
request path with effects off). In the fixtures, three context digests move (Svc.build.step, Host.meth.Local.m, closure_effect.py main.swap: closures
that capture an object) and one edge appears, Svc.build.step -> JsonHandler.process, the row's own proof.
Measured 2026-11-13, the same entries and the same method as row 11 (dump_graph, PYTHON_DUMP_GAP=1, debug test binary), before = 8b91ba5, after = the
CS4.1 change. A site with two or more callees applies each project callee's effects to its own copy of the state before the call and joins the arms, as
inline mode does (03-effects.md §3 row 8); strong_ok is gone.
| Project | effects | nodes | contexts | walks | unresolved | extra |
effects_applied |
|---|---|---|---|---|---|---|---|
| Jarvis | off | 207 → 207 | 55 → 55 | 55 → 55 | 30 → 30 | 0 → 0 | - |
| Jarvis | on | 207 → 207 | 71 → 71 | 71 → 71 | 32 → 32 | 0 → 0 | 43 → 43 |
| Flask service | off | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | - |
| Flask service | on | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | 87 → 87 |
| FastAPI | off | 458 → 458 | 70 → 70 | 70 → 70 | 29 → 29 | 0 → 0 | - |
| FastAPI | on | 1019 → 1019 | 104 → 103 | 104 → 103 | 52 → 52 | 0 → 0 | 180 → 180 |
Only FastAPI with effects on moves, by one context (and so one walk): two callees of one site used to be keyed one after the other, the second against the
state the first had left; now both are keyed against the state before the call, so two that agree share a context. missing is unchanged (Flask 41,
FastAPI off 561, the rest 0) and create_cloned_field is still the one Incomplete(MaxRounds) cut. The golden of Jarvis is equal (wire and merged
wire, the request path with effects off, compared as JSON against out/golden/wire*.json: the files differ only in layout).
Measured 2026-11-13, the same entries and method as CS4.1 (dump_graph, PYTHON_DUMP_GAP=1, the debug test binary), before = 0eb5fa6 (the CS4.1 table
above), after = the CS5b commit. An object an input holds that the key did not name is named by its path (Entry(path.attr), up to Limits::max_path,
6, never less than fp_depth); past it, by Beyond { at, attr }, which holds no state. A write onto a Beyond object is a weak Unknown at at.attr,
the summary gets Gaps::DEEP_WRITE, and core_graph counts it (Stats.language["deep_write"]) and does not mark the context (option B: status maps
only Gaps::CYCLE to Incomplete). A Beyond object that leaves a summary as a return, an effect value or an object's attribute has no identity there
(Opaque): that is Gaps::DEEP_RETURN, counted under Stats.language["deep_return"], also not marked. Both counters are written only above 0.
How a named object keys. An input object that only attr_origin named (Walk.named: the origin did not exist before) and that nothing wrote an
attribute onto keys as Opaque(C), as ty's object did before it had a name. A write onto it makes it Inst(C, attrs); a child named by a later read does
not count as state, because a read writes nothing. Other Entry objects (made from a key) key as before. The reason: from_fp gives the callee the same
starting env for Opaque(C) and for Inst(C, []) (an Entry origin, no attributes), so keying the second differently only splits equal contexts and
changes every digest. The first version of CS5b keyed them as Inst(C, []): the golden of Jarvis was differs (49 of 207 frames, contexts[].digest
only) and recursion_objects.py visitor had a seventh Node.accept context.
| Project | effects | nodes | contexts | walks | unresolved | extra |
rounds_capped |
effects_applied |
deep_write |
deep_return |
contexts if every empty Inst keyed Opaque |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Jarvis | off | 207 → 207 | 55 → 55 | 55 → 55 | 30 → 30 | 0 → 0 | 0 → 0 | - | 0 | 0 | 53 |
| Jarvis | on | 207 → 207 | 71 → 71 | 71 → 71 | 32 → 32 | 0 → 0 | 0 → 0 | 43 → 57 | 0 | 0 | 69 |
| Flask service | off | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | 0 → 0 | - | 0 | 0 | 70 |
| Flask service | on | 266 → 266 | 70 → 70 | 70 → 70 | 81 → 81 | 0 → 0 | 0 → 0 | 87 → 87 | 0 | 0 | 70 |
| FastAPI | off | 458 → 458 | 70 → 70 | 70 → 70 | 29 → 29 | 0 → 0 | 0 → 0 | - | 0 | 0 | 70 |
| FastAPI | on | 1019 → 1019 | 103 → 103 | 103 → 103 | 52 → 52 | 0 → 0 | 1 → 1 | 180 → 180 | 0 | 0 | 103 |
deep_write and deep_return are 0 on all three projects: the limit is theory on this code. Only Jarvis with effects on moves: 14 more effects
applied (decision 1 and follow carry writes to objects the key did not name), the same 207 nodes, missing 0, extra 0. Flask and FastAPI do not move,
and create_cloned_field is still the one Incomplete(MaxRounds) cut. The golden of Jarvis is equal (scripts/run-on-project.sh python <Jarvis> jarvis.py CallGraphGenerator.do_pass --function-prefix FunctionSchema/ --class-prefix ClassSchema/ with GOLDEN=out/golden: wire.json: equal,
wire-merged.json: equal).
The last column is a measure, not a change (a temporary env check in fp_one, removed): if every Inst with no attributes keyed as Opaque, Jarvis
would have 2 contexts fewer (55 → 53 off, 71 → 69 on) and Flask and FastAPI none. Those two are objects a key holds with an empty state (Config() and the
like), whose digest changes with the spelling.
Fixtures. One graph snapshot moved: deep_write.py (the intended fix: main calls JsonHandler.process, not CsvHandler.process). Four programs
joined (deep_write_capped.py, deep_write_two_levels.py, deep_read.py, deep_recursion.py), and deep_write.py's wire moved from unverified/ to
the verified folder. deep_recursion.py under the default max_path shows deep_return 5: the object past the cap leaves five summaries as Opaque.
Measured 2026-11-13 on the three projects, effects off and on, the same entries and method as CS5b. Context.alias[i] is the first slot holding the same
object as slot i (07 §2): a slot can alias only if the walk can be affected by it (relevant_params), it holds exactly one value, and that value is an
object with an identity (Walk::holds_state: ty's OPAQUE object and one past max_path never alias; one that was only named, with no state, is still
one object). materialize binds slot i to slot alias[i]'s value, so a write through one is a read through the other. Only top-level slots.
CanonKey carries alias only when it is not the identity (skip_serializing_if), so a key with no aliasing has the digest it had: pinned in
p04_07_a_key_with_no_aliasing_has_the_digest_it_had (580af8cf013e505a6d505eeb31764101, written down before Context.alias existed).
| Project | effects | nodes | contexts | unresolved | extra |
contexts with a non-identity alias |
|---|---|---|---|---|---|---|
| Jarvis | off | 207 | 55 | 30 | 0 | 0 |
| Jarvis | on | 207 | 71 | 32 | 0 | 0 |
| Flask service | off | 266 | 70 | 81 | 0 | 0 |
| Flask service | on | 266 | 70 | 81 | 0 | 0 |
| FastAPI | off | 458 | 70 | 29 | 0 | 0 |
| FastAPI | on | 1019 | 103 | 52 | 0 | 0 |
Nothing moved: the numbers are CS5b's, and no context on any project has an alias (counted with a temporary check in key, removed: the number is a
measure, not a counter). The golden of Jarvis is equal (wire.json: equal, wire-merged.json: equal), which is the proof that is_identity leaves
every digest alone. In the fixtures one digest moved on purpose: set_and_read under aliases.py same (580af8cf… to c5621520…, the vector [0, 0]);
different keeps 580af8cf….
Known limits, pinned in both modes. aliases.py nested (a.peer is b: pairs of paths are not in the key, 07 §2 decision 4) and alias_free.py (a
parameter that is also a closure's free value: f(x) where f reads x; summary draws CsvHandler.process, inline JsonHandler.process). The second was
found by a scratch program while answering 07 §5 claim 2, and is pinned so it stays said.
Measured 2026-11-13 at 8de962a plus the CS4b change, the same entries and method as CS7 (dump_graph, PYTHON_DUMP_GAP=1, the debug test binary; the
Jarvis golden through scripts/run-on-project.sh). Options::effects is deleted: summary mode always computes a summary's effects and applies them, and
solve always runs the one prologue. Inline mode never read the flag (solve is summary mode's start; inline starts through its own prologue call).
PYTHON_DUMP_EFFECTS is gone from dump_graph: it always runs with effects, so a "before" is a checkout before this commit. The request path
(python-cli, lib.rs) therefore has effects on; the wire format is unchanged (version 2).
| Project | nodes | contexts | unresolved | extra |
rounds_capped |
effects_in_cycle |
deep_write |
deep_return |
solve ms |
|---|---|---|---|---|---|---|---|---|---|
| Jarvis | 207 | 71 | 32 | 0 | 0 | 0 | 0 | 0 | 172 |
| Flask service | 266 | 70 | 81 | 0 | 0 | 0 | 0 | 0 | 290 |
| FastAPI | 1019 | 103 | 52 | 0 | 1 | 0 | 0 | 0 | 271 |
These are CS7's effects-on rows (create_cloned_field is still the one Incomplete(MaxRounds) cut; Flask is still 41 nodes short of inline, 1 topmost,
silent). Time is noise at this size. The golden of Jarvis moved, in contexts only: with the default options now effects on, wire.json and
wire-merged.json have the same 207 and 66 frames, and with contexts left out they are equal; the contexts of 11 of the 207 frames (8 of the 66)
differ (digests, and how many contexts a frame holds), 55 distinct (digest, function) pairs before and 58 after. No name, id, kind or call site moved.
No fixture snapshot and no tests/api.rs pin moved (the suite passes unchanged: lib 130 and 2 ignored, api 52, cli 29); the fixtures ran with effects on
since CS3b. a_weak_write_marks_what_it_may_have_dropped_and_a_strong_one_marks_nothing lost its effects-off half (two_candidates has an Unresolved
site without effects); what it says with effects on is kept. Sem::classmethod_class was only the effects-off start of an entry and is deleted.
What the tag holds: class summaries CS2–CS7 with effects on for every request (CS4b: the flag is gone); the kept project, refresh, cancel and environment
inputs written with / for the driver (PY9); Limits::max_path; and the Jarvis golden regenerated at CS4b (only contexts moved). python-v0.2.0
(baae723, what the drivers pin) is not moved. The tag goes on the docs commit that holds this section, after it is pushed and CI is green on the
three systems.
Numbers (debug build, the "CS4b" table above): Jarvis 207 nodes, 71 contexts, 32 unresolved; Flask service 266 nodes, 70 contexts, 81 unresolved;
FastAPI 1019 nodes, 103 contexts, 52 unresolved; extra 0 on all three; deep_write, deep_return and effects_in_cycle 0; FastAPI has the one
MaxRounds cut.
Known limits (each is drawn or counted, never silent, except where it says so):
- No narrowing on a branch or on
isinstance: both arms are walked. FastAPI's 20 extra-looking nodes are this (class summaries README §8; the CS2b section above). - A parameter that is also a closure's free value is not an alias:
f(x)wherefreadsx. A summary-only wrong edge, pinned (alias_free.py, CS7 above). - A deep write past
max_pathis counted, not marked (Stats.language["deep_write"]; 0 on the three projects; CS5b above). - Recursion that builds objects can reach
max_roundsand is markedIncomplete(MaxRounds)(create_cloned_field; class summaries CS8, not built). - Two classes at one call site mix their writes (
split.py; class summaries CS6, not built). Inline mode draws the same. - A file's own top-level code is in no tree (
docs/05-python-migration/12-module-code.md, planned, not built). - A call on an untyped value is
Unresolved(unknown_callee; most of the Flask service's 81).
For a driver: build Options and Limits with ..Default::default() (Limits gained max_path); the wire is version 2 and did not change.
vendor/ruff/rust-toolchain.toml says 1.98.0 and ruff's rust-version is 1.96. It is ignored here (the repo root's rust-toolchain.toml decides,
1.99.1). Measured on 2026-11-11, macOS, cargo build -p python-syntax, cold target: 52 s, no warnings, on 1.99.1.
The full ty closure has not been built in this repo yet (PY5).
An engine bump is a change of its own: only vendor/, Cargo.lock, sem.rs, and this file.
- Clone upstream at the new commit into a temporary folder.
- Compute the closure: every crate
ty_projectandty_python_semanticdepend on (cargo metadata). Replacevendor/ruff/crates/with those crates and copy upstream'sCargo.toml,LICENSEandrust-toolchain.toml. - Remove the four test-data directories above (and check that no new
include_str!reads a directory you are about to remove:grep -rn 'include_str!\|include_bytes!' vendor/ruff/crates --include='*.rs'). - Re-apply the patch: copy
analyzer_ext.rsback and re-add the onepub modline. Keep the copy and the patch as two commits. - Copy the
salsaline (and its siblings) from the newvendor/ruff/Cargo.tomlinto[workspace.dependencies]. cargo test, thencargo test -p python-syntaxalone:cargo tree -p python-syntaxmust still list nosalsa,ruff_dborty_*.- Re-check
sem.rsagainst the new source and fixsem.rsonly. Write the new commit, its date and ruff version here.
If upstream's rust-toolchain.toml moved, that is a separate step: bump the repo's toolchain first, and tell the driver repo (one toolchain).
At most once per release of the analyzer: ty changes weekly.