refactor: migrate to modern Python project structure with uv

[xzAscC]

Summary

• Migrate from requirements.txt to pyproject.toml with uv package manager
• Restructure source code into src/probing_reflection/ package
• Add core type definitions: ProbingConfig, ReflectionResult, ContrastivePair
• Add comprehensive documentation (AGENTS.md, ARCHITECTURE.md, SPEC.md)
• Add test infrastructure with pytest
• Add development scripts (quick_test.sh, full_test.sh)
• Add experiment draft notes

Changes

Added

• src/probing_reflection/ - Main package with type definitions
• pyproject.toml - Modern Python packaging with uv
• tests/ - Test infrastructure with pytest
• scripts/ - Development test scripts
  • quick_test.sh - Fast validation with small model
  • full_test.sh - Complete validation across all models
• .sisyphus/drafts/ - Experiment draft notes
• .github/ - Issue templates, PR template, CI workflow
• docs/ - Quality score tracking and design docs

Removed

• Legacy scripts and placeholder files
• Old requirements.txt (replaced by pyproject.toml)
• Legacy source files (acc_length_rel.py, inference.py, etc.)

Verification

uv sync                  # ✅ Dependencies installed
uv run ruff check src/   # ✅ No lint errors
uv run ruff format src/  # ✅ Formatted
uv run mypy src/         # ✅ Type check passed
uv run pytest            # ✅ Tests pass

Commits

Commit	Description
73c09bd	refactor: migrate to modern Python project structure with uv
80c246a	chore: add development test scripts
391b00d	docs: add experiment draft for token probing

[cubic-dev-ai]

6 issues found across 34 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/probing_reflection/types.py">

<violation number="1" location="src/probing_reflection/types.py:42">
P2: The `metadata` dict is mutable despite the class being documented as immutable. `frozen=True` only prevents field reassignment, not in-place mutation of the dict contents. Use `MappingProxyType` or accept an immutable mapping type to enforce true immutability, matching the class's contract.</violation>
</file>

<file name="TEMPLATE.md">

<violation number="1" location="TEMPLATE.md:38">
P2: Package name in rename instructions (`ai_research_template`) doesn't match the actual package name (`probing_reflection`). Users following these steps will get "file not found" errors from both the `sed` and `mv` commands.</violation>

<violation number="2" location="TEMPLATE.md:71">
P2: Wrong file referenced: `packages = ["YOUR_PROJECT_NAME"]` is in `pyproject.toml` line 35, not `AGENTS.md`. This will send users to the wrong file.</violation>
</file>

<file name="SPEC.md">

<violation number="1" location="SPEC.md:23">
P2: `.sisyphus/plans/` and `.sisyphus/roadmap.md` are nested under the `docs/` bullet as if they live inside that directory, but their paths start with `.sisyphus/`, placing them at the repo root. This contradicts AGENTS.md §10 and will mislead agents about where to find these files. Move them to their own top-level bullets.</violation>
</file>

<file name="AGENTS.md">

<violation number="1" location="AGENTS.md:56">
P2: `unknown` is a TypeScript type that does not exist in Python. An agent following this guideline would write invalid code. The Python equivalents to avoid `Any` are `object`, generic type variables (`TypeVar`/`T`), or `Protocol` types.</violation>

<violation number="2" location="AGENTS.md:58">
P2: Python uses `raise`, not `throw`. This is JavaScript/Java terminology that doesn't apply to this Python project.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/full_test.sh">

<violation number="1" location="scripts/full_test.sh:17">
P3: Step counter is wrong: the first four steps say `/4` but there are 5 steps total. The denominator should be `5` throughout.</violation>
</file>

<file name="scripts/quick_test.sh">

<violation number="1" location="scripts/quick_test.sh:17">
P2: Step counter is wrong: there are 5 steps but the first four say `/4`. Update all labels to `/5`.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[xzAscC]

@codex I Please re-run review. All 8 issues identified have been fixed in commit 44579c3.

[cubic-dev-ai]

@cubic-dev-ai Please re-run review. All 8 issues identified have been fixed in commit 44579c3.

@xzAscC I have started the AI code review. It will take a few minutes to complete.