Skip to content

About

LLM-assisted review of Educational Affiliation Agreements (EIAA) against a codified playbook — ACCEPT or tracked-changes redline with footnoted rationales. Apache-2.0.

Resources

Contributing

Stars

3 stars

Watchers

0 watching

Forks

Latest commit

 

History

250 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Contract Toaster

OPF-native contract review: upload a counterparty-modified draft, and get back either an ACCEPT decision or a redlined .docx with in-place tracked changes and footnoted rationales, reviewed against your codified negotiating position (a "playbook").

Reviewing this repo? Start with docs/REVIEW-GUIDE.md — an honest orientation to what actually works today (the Docker Compose target runs the real review pipeline; the AWS target still runs a mock review stage), the two deployment targets, how to see it running, and the roadmap.

Status

In active development. Not yet deployed. See milestones for phase progress.

RUNBOOK.md and ARCHITECTURE.md describe the target design, some of it ahead of the code — that's a deliberate "docs as spec" choice for a solo team, but it means the docs alone don't tell you what's reachable today. See docs/implementation-status.md for the SHIPPED / STUBBED / PLANNED ledger of every admin-UI capability and observability surface RUNBOOK.md describes.

What this does

  1. Reviewer signs in (Google SSO on AWS, restricted to your organization's Workspace domain; username/password in the Docker Compose quickstart below).
  2. Reviewer uploads a counterparty-modified .docx.
  3. The service reviews it against the active playbook — the Synthetic NDA Sample playbook ships as the default, fully-working worked example (see "How review works" below).
  4. The service returns either:
    • ACCEPT — the counterparty's changes are within your acceptable range; no redline required.
    • REQUEST_CHANGE — a redlined .docx with in-place tracked changes and a footnoted rationale for each material edit.
  5. Every review is logged with the playbook version, model, token counts, and cost.

A human attorney reviews and approves before anything goes back to the counterparty. This tool produces drafts and analysis; it does not give legal advice and does not constitute approval — approval happens outside this tool. An ACCEPT means "no requested changes identified by the tool", not "no action needed".

What this is not

  • Not a replacement for attorney judgment.
  • Not connected to any counterparty system.
  • Not training a model on your documents. The AWS target calls Anthropic Claude via Amazon Bedrock, whose terms exclude prompts and outputs from model training. The Docker Compose target calls models through OpenRouter instead, under OpenRouter's and the model provider's terms; check them before you send real documents.

How review works: knowledge and precision

Contract Toaster is type-blind: adding a new agreement type is data, not code — author a playbook (an OPF-native JSON document), add one entry to playbooks/registry.json, and it's reviewable. No call site in backend/src/ or scripts/ hard-codes a playbook name.

Every playbook resolves to one of two review modes:

  • Knowledge (the universal default). The uploaded draft is compared against the playbook's codified negotiating positions directly — no canonical standard form is required, so this mode works for any counterparty-authored paper from day one.
  • Precision (a registry label, not a different review path). A playbook may additionally register a canonical standard form and a matching anchor map (standard-forms/), which scripts/playbook_registry.py::profile() reports as the precision profile. The deterministic section-anchored diff that profile once selected was retired from issue generation by the 2026-07-22 LLM-native decision and deleted in 2026-09 (issue #631), so every review today runs the same LLM-native path. The artifacts remain governed history and the profile still namespaces a playbook's evaluation fixtures.

The default synthetic-nda-sample playbook needs no standard form, so a fresh install is reviewable end to end with nothing to author first. On the Docker Compose target the bootstrap installs and activates it on first boot (see First run: the shipped playbook); on AWS an admin uploads and activates it (RUNBOOK.md, bootstrap step 10).

Playbooks

Playbooks are plain data: JSON documents following the Open Playbook Format (OPF), version 0.6 (the one format this repo reads; its pinned schema is playbooks/opf/playbook.schema-0.6.json, the same file as the playbook.schema-0.6.json in contract-opf/opf and the playbook-engine spec/ directory). playbooks/schema.json is the older v1 playbook schema, kept for the bundled sample. playbooks/registry.json is the single source of truth mapping a playbook_id to its playbook, anchor-map, section-config, and fixtures paths — adding a contract type never requires a code change.

  • Default: synthetic-nda-sample ("Synthetic NDA Sample") — a clearly fictional, brand-free NDA playbook, bundled with the image and installed and activated automatically on a Docker Compose deployment's first boot (see First run: the shipped playbook), so no real negotiating position is ever exposed by accident. Remove or replace it with your own playbook once you're ready.
  • Published playbooks: contract-opf/playbooks — published, corpus-derived OPF playbooks, offered as pedagogical examples of the format under that repository's PLAYBOOK-POLICY.md (attorney review is mandatory; they are not legal advice).
  • Bring your own: author a playbook + anchor map (optional, for a precision profile) + section config + fixtures directory, then add one playbooks/registry.json entry.

Architecture, in brief

  • Frontend. React SPA on AWS Amplify Hosting. Cognito user pool federated to Google as the only IdP, hosted-domain restricted.
  • Backend. Python (FastAPI) container on AWS App Runner serving the API only. That image runs Python 3.12 (backend/Dockerfile); everything else, including the Docker Compose image, is on 3.13. Builds go through CI (CodeBuild or equivalent): tests and scans run, a signed container image is pushed to ECR, and App Runner is pinned to an immutable image digest that is promoted deliberately. A merge to main never auto-mutates production legal behavior.
  • Review pipeline. Reviews run asynchronously: on AWS the API starts a Step Functions execution directly (extract → primary review → adversarial review → leakage scan → redline → persist; its review stage is still a mock there), and on Docker Compose an in-process worker runs the real chain. The UI polls for the result. There is no SQS buffer on the entry path. Idempotency comes from a submission record, one spend reservation per review_id, deterministic Step Functions execution names, and retry-safe "ensure execution started" semantics.
  • LLM. A two-pass (reviewer + adversarial critic) design. The AWS design calls Anthropic Claude via Amazon Bedrock (single-region, us-east-1, native model ID — no cross-region inference profile); the Docker Compose target, which is the one that runs real reviews today, calls models through OpenRouter and needs an OpenRouter API key (see The OpenRouter key). The model is governed by an explicit model-policy matrix: primary model, critic model, embedding model, optional fallback, region, request contract, evaluation run, and cost assumptions. v1 pins the same model (Opus 4.8) on both passes (issue #148): the critic, which has the last word (ADR 0001, issue #136), is the stronger model, and the reviewer runs on it too so the two passes share one prompt cache — unless a future evaluated policy says otherwise.
  • Retrieval: dormant by decision. No review retrieves precedent today. The deterministic keyword/rule detectors for hard-rejection terms do run; the designed semantic layer (a Bedrock Knowledge Base over S3 Vectors) and corpus activation are parked, and only the ingestion half exists (POST /api/corpus, which produces a draft snapshot nothing reads). docs/rag-dormant.md has the status and what reviving it would take.
  • Storage. S3 (uploads, redlines, corpus, audit; governance object lock on corpus and audit; admin-configurable document retention plus storage-level legal hold). DynamoDB (users, playbooks, playbook versions, reviews, audit log, cost ledger). Immutable audit rows contain only non-substantive audit facts; model rationales and critic deltas live only in retention-governed confidential storage.
  • Infrastructure as code. AWS CDK (TypeScript). Everything is cdk deploy.
  • Redlining. scripts/redline_block_apply.py is a small, dependency-free OOXML tracked-changes writer we own outright, built entirely on the Python standard library (zipfile + xml.etree.ElementTree); it edits the uploaded package in place against a proven block transcript, with the non-document.xml parts it attaches (footnotes, header/footer, styles) in scripts/docx_parts.py. There is no backend/vendor/ directory in this repo; see ARCHITECTURE.md → Redlining.
  • Review prompt and playbook structure. Prompts are assembled in code by scripts/primary_review_pass.py (system prompt = review guidance + binary-decision overlay + playbook JSON, per a fixed manifest), not stored as a prompts/ directory. The review guidance was informed by anthropics/claude-for-legal's contract-review skill as reference material, with our own overlay for the binary-decision output format. Active releases are a governed bundle: playbook hash, prompt hash, standard-form hash, model-policy hash, corpus snapshot, evaluation run, and legal approval. Precedent citations are internal-only; generated external footnotes cite the contract position, not prior counterparties.

See ARCHITECTURE.md for the full picture.

Repository layout

contract-toaster/
├── README.md
├── ARCHITECTURE.md
├── RUNBOOK.md
├── LICENSE
├── NOTICE
├── TRADEMARKS.md                 # name-use grant for the "Contract Toaster" mark
├── playbooks/                    # OPF-native playbooks: schema, registry, and playbook data
│   ├── schema.json               # older v1 playbook schema (the bundled sample's format); OPF 0.6 lives in playbooks/opf/
│   ├── registry.json             # playbook_id -> artifact paths (add one entry per new agreement type)
│   └── samples/
│       └── synthetic-nda-sample-v1.0.0.json  # default (bundled) playbook -- the worked example in this README
├── standard-forms/               # canonical standard-form .docx files and derived anchor maps (precision profile, opt-in)
│   └── README.md                 # directory guide and build instructions
├── scripts/                      # the review pipeline: pure, tested modules (no separate prompts/ or vendor/ tree)
│   ├── docs-lint.py              # documentation lint
│   ├── primary_review_pass.py    # prompt assembly + primary review pass (code-assembled, not a prompts/ directory)
│   └── redline_block_apply.py    # dependency-free OOXML tracked-changes writer we own outright
├── infra/                        # AWS CDK (TypeScript)
├── backend/                      # Python service for App Runner + pipeline tasks
│   ├── Dockerfile
│   ├── requirements.txt
│   └── src/
├── frontend/                     # React SPA for Amplify
├── deploy/                       # Docker Compose deployment target
└── docs/

Documentation source of truth

The source docs are this README, ARCHITECTURE.md, RUNBOOK.md, and the files under docs/. Generated review packets are not authoritative; regenerate them from these files if needed.

Local development

The toolchain is pinned to Python 3.13 (root .python-version and pyproject.toml); the Docker Compose backend image and the python-version: pins in CI declare the same one, and tests/test_python_version_parity_63.py fails if those declarations diverge. The fastest way to see the app running is the self-contained Docker Compose stack — no local Python and no AWS account required:

gh repo clone contract-opf/contract-toaster
cd contract-toaster
cp deploy/dts/.env.example deploy/dts/.env
# edit deploy/dts/.env: DEMO_TOKEN_SECRET=<output of `openssl rand -hex 32`>
# (the backend refuses to start without a strong one), and optionally
# OPENROUTER_API_KEY — or leave it blank and set the key in the admin UI
docker compose -f deploy/dts/docker-compose.yml --env-file deploy/dts/.env up --build

A review needs an OpenRouter key. This stack runs the real review chain and has no mock fallback: with no key, every review ends in ERROR. Set OPENROUTER_API_KEY in .env, or sign in as admin and set it on the Models tab (it overrides the variable). Reviews are billed to that key.

The demo logins are seeded once, on a deployment's first boot, and on every Docker Compose deployment, including one Coolify publishes to the internet. They do not come back after you delete them. On anything reachable by others: sign in as admin, change its password, add your own admin on the Users tab, sign in as that admin, and delete admin and user (details).

First run: the shipped playbook

Contract Toaster is an empty shell: it knows nothing about any agreement type until a playbook is installed and active. With no active playbook, submitting a review is refused with 503 no active playbook rather than silently falling back to some built-in default — there isn't one.

So that a fresh install is runnable out of the box, the Docker Compose bootstrap (deploy/dts/bootstrap.py) installs and activates the playbook the image ships with, synthetic-nda-sample ("Synthetic NDA Sample"), the first time it runs against empty tables. Bring the stack up, set the OpenRouter key, and upload a .docx — the review runs end to end, no first-run click required. (The AWS target does not run this bootstrap; there an admin uploads and activates the playbook — see RUNBOOK.md, bootstrap step 10.)

That install is not a special path: it calls the same record_playbook_version_upload / activate_playbook_version functions an admin-uploaded version goes through, so the shipped playbook lands as an ordinary version row with an ordinary ACTIVE status. The seed is install-once — once the playbook has version rows, or once an admin has removed it, re-running the bootstrap does nothing, so a container restart can never overwrite your changes or resurrect something you deleted.

The shipped playbook is a fictional, brand-free NDA playbook — a worked example, not legal content, and not anyone's real negotiating position. It carries an admin-editable note explaining what it is and where to find more.

Using your own playbook

The sample is a normal playbook, not a special case, so it supports the full lifecycle through the admin API: upload a new version, activate it, roll back, rename it, or remove it outright (removing the last one returns you to the empty-shell state).

  • More playbooks: contract-opf/playbooks — published OPF playbooks (pedagogical examples; see that repository's PLAYBOOK-POLICY.md).
  • Author your own: write an OPF 0.6 JSON document against playbooks/opf/playbook.schema-0.6.json (an engine-compiled playbook from playbook-engine already is one), then add one entry to playbooks/registry.json. No code change is required to add an agreement type — see docs/playbook-governance.md for versioning and approval.

Full instructions, what's mocked, and what to expect are in docs/REVIEW-GUIDE.md and deploy/dts/README.md.

Deploying to AWS

This repo is a work in progress; the AWS deployment (App Runner, Amplify Hosting, Cognito, Step Functions) is not yet reachable end-to-end from a fresh clone, and its review stage is still a mock — see docs/REVIEW-GUIDE.md for the current status and blocking issues.

The CDK app refuses to synthesize without six deploy-identity values: githubOwner, alarmsEmail, appDomain, adminEmail, hostedDomain and allowedEmailDomains (the email domains Cognito sign-ins may come from, which the stack hands the backend as ALLOWED_EMAIL_DOMAINS). Put them in infra/cdk.context.json (copy infra/cdk.context.example.json; the real file is git-ignored) or pass each as --context key=value. Also replace the dummy account IDs under dev / prod in infra/cdk.json. See infra/README.md.

# Prerequisites
brew install node awscli gh python@3.13   # Python 3.13 — see .python-version
npm install -g aws-cdk

# Infrastructure (CDK)
cd infra
npm install
cp cdk.context.example.json cdk.context.json   # then fill in the six values
cdk synth --context env=dev                    # validate (offline)
cdk diff --context env=dev                     # preview changes
cdk deploy --context env=dev                   # deploy (add --context profile=minimal for a demo footprint)

# Backend (run locally against deployed AWS resources)
cd ../backend
python3.13 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn src.main:app --reload

# Frontend (run locally against deployed Cognito)
cd ../frontend
npm install
npm run dev

Full setup details live in RUNBOOK.md.

Contributing

Start with CONTRIBUTING.md — setup that has been run from a clean clone, the four things that will bite you, and where to start in the backlog.

  • All work is tracked in GitHub issues in this repository.

  • Every change goes through a pull request, and CI runs on both branches and PRs. Note that main is not currently protected — the organisation's plan does not offer it, so nothing server-side stops a direct push. The interim control (issue #64) is local:

    bash scripts/setup-hooks.sh   # once per clone: opt in to .githooks/
    git checkout -b phase-N/short-description
    # ... commit ...
    bash scripts/land.sh          # gates, then push + `gh pr create --fill`

    scripts/land.sh refuses to run on main, runs ruff check … && mypy, npm test, scripts/check.sh and tests/lint-brand-free.py, and pushes nothing if any of them is red. .githooks/pre-push refuses git push origin main outright — override a deliberate direct push with LAND_TO_MAIN=1. The hooks are opt-in: nothing installs them for you.

  • The two gates are bash scripts/check.sh (the Python suite, docs-lint and the detectors) and bash scripts/check-frontend.sh (typecheck, build, vitest and the design-system audits). Add SKIP_INFRA=1 to the first unless you touched infra/. While you are iterating on one file, narrow it with a quoted glob — bash scripts/check.sh --only 'tests/test_review_api_84.py' — which runs the same loop over just the matches and exits 4, never green, if the glob matches nothing. A full run is still what lands a change. CONTRIBUTING.md has the rest.

  • Lint and types (issue #65). ruff + mypy for Python and eslint + tsc for TypeScript run in .github/workflows/lint.yml on every push and pull request, as a fast pre-step inside scripts/check.sh and scripts/check-frontend.sh, and as scripts/land.sh's first gate:

    pip install -r requirements-dev.txt          # ruff + mypy are pinned there
    ruff check backend scripts tests infra/lambda
    mypy                                         # config: [tool.mypy] in pyproject.toml
    cd frontend && npm run lint && npm run typecheck

    These are baseline-then-ratchet gates: green means no NEW violations, not no violations. The tree as of #65 carries # noqa comments (ruff), eslint-disable-next-line comments (eslint), and per-module [[tool.mypy.overrides]] ignore_errors = true entries mirrored in mypy-baseline.txt (mypy). tests/test_lint_gates_65.py fails if that mypy list grows, or if it still names a module mypy now passes — so the baseline can only shrink. Pay debt down by DELETING an annotation, never by adding one to new code.

    The formatters (ruff format, prettier) are configured but deliberately not enforced yet — a whole-tree reformat collides with the in-flight audit series. Issue #90 turns them on afterwards.

    Optional pre-commit hooks for both live in .pre-commit-config.yaml (pre-commit install).

  • Anything that changes playbooks/ or the review prompts (assembled in code by scripts/primary_review_pass.py and its sibling pass modules; there is no prompts/ directory) requires legal review.

  • Conventional commits: feat:, fix:, chore:, docs:, refactor:, test:.

License

  • Code (this repository, except where noted below) is licensed under the Apache License 2.0.
  • Playbook content — negotiation positions and related content under playbooks/ — is licensed under CC BY 4.0 where marked in the file or an accompanying README.
  • "Contract Toaster" is a trademark of Athletes' Performance, Inc.; you may use the name to refer to this project — see TRADEMARKS.md for the grant, and NOTICE for attribution requirements.

About

LLM-assisted review of Educational Affiliation Agreements (EIAA) against a codified playbook — ACCEPT or tracked-changes redline with footnoted rationales. Apache-2.0.

Resources

Contributing

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages