Dropbox folder watcher with write-back (Shlok Chauhan)

extensions/theshloksschauhan/dropbox-folder-watcher/.gitignore

@@ -0,0 +1,36 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+env/
+
+# Environment / Secrets
+.env
+*.env.local
+*.db
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Node (frontend)
+node_modules/
+frontend/build/
+frontend/dist/
+
+# Docker
+postgres_data/
+
+# Alembic
+# (we DO commit migrations, but not generated __pycache__)
+alembic/__pycache__/

extensions/theshloksschauhan/dropbox-folder-watcher/ARCHITECTURE.md

@@ -0,0 +1,46 @@
+# Architecture: SuperDocs Dropbox Watcher
+
+This document details the architectural decisions and patterns used to implement the SuperDocs Dropbox Folder Watcher, fulfilling the exact specifications in the original design document.
+
+## 1. Core Principles
+- **No Lost State**: Every state transition is written to PostgreSQL inside a transaction. If the worker crashes mid-run, it safely resumes.
+- **Fail-Safe Preview**: "Preview Mode" is guaranteed to cost $0 via a structural chokepoint. Billable operations are blocked at the lowest SDK-wrapper level, not left to application logic.
+- **Dependency Injection**: External API clients (Dropbox and SuperDocs) are abstracted using Python `typing.Protocol`. All tests run against offline fakes.
+
+## 2. Component Pipeline
+
+### The Watcher (Discovery)
+- Scans configured Dropbox folders using list_folder.
+- Passes every file through a strict filtering pipeline:
+  1. Is it our own output? (Anti-loop DB check) -> Ignore
+  2. Have we already processed this exact revision? -> Ignore
+  3. Is the file stable? (Content-hash debounce) -> Wait
+- Creates a `Job` in `DISCOVERED` state, transitions to `QUEUED`.
+
+### The Worker (Execution)
+- Atomically claims a `QUEUED` job (prevents double-processing across workers).
+- Phase 1: SuperDocs **upload + chat**, then **stops at `REVIEW_PENDING`**.
+- A human (or machine via `POST /api/jobs/{id}/approve`) must approve.
+- Phase 2: SuperDocs **approve + export**, Dropbox write-back, `known_outputs` registry.
+
+### The API (Visibility & Webhooks)
+- FastAPI endpoints providing health metrics, folder configuration management, and full Job audit trails.
+- Receives Dropbox webhooks (HMAC-SHA256 verified) to trigger fast-path polling (implemented as hybrid webhook/polling).
+
+## 3. Specific Solutions to Known Pitfalls
+
+### The "Double JSON Parse" Bug
+SuperDocs returns proposed changes as a JSON string inside a JSON response. The client explicitly implements a defensive `parse_proposed_changes` function that detects strings and parses a second time, preventing empty diffs.
+
+### Anti-Loop Protection (Layered)
+Naming conventions are fragile. This system uses three layers:
+1. **Naming Pattern**: Soft check (`.superdocs.` marker in filename).
+2. **Path Registry**: Exact output path match in DB.
+3. **Hash Registry**: Exact content-hash match in DB.
+Only the latter two are authoritative.
+
+### Stability (Half-Written Files)
+Dropbox will trigger events before large files are fully uploaded.
+- The `StabilityService` maintains a rolling window of observations.
+- It requires the `content_hash` to remain unchanged for a configurable `debounce_seconds` window.
+- Size checks are ignored in favor of the much safer `content_hash` equality.

extensions/theshloksschauhan/dropbox-folder-watcher/NOT_DOING.md

@@ -0,0 +1,21 @@
+# Deliberate cuts for the S2 Dropbox folder watcher (Task 2.1)
+
+## Cut: Multi-tenant auth and Dropbox OAuth per client
+**Why:** S2 band; a shared admin token and single Dropbox app token is enough to prove the loop. Full OAuth per client folder would dominate the build.
+**Impact:** Client isolation is enforced by folder path scoping in config, not by Dropbox shared-link permissions.
+
+## Cut: Item-level approve/reject of individual proposed changes
+**Why:** SuperDocs approve is document-level in the four-call contract. Partial approval would need custom diff splitting.
+**Impact:** Human gate is job-level: approve all proposed changes or reject the job.
+
+## Cut: Real-time webhook-driven scan
+**Why:** Polling + debounce already handles half-written files; webhook endpoint verifies signatures but fast-path scan deferred.
+**Impact:** Slightly higher latency on detection; no correctness loss.
+
+## Cut: MCP server for Task 2
+**Why:** REST API exposes approve/reject/list for machine driving; MCP reserved for Task 1 private repo.
+**Impact:** Behavior #4 for the watcher is via REST, not MCP.
+
+## Cut: PostgreSQL-only tests for Task 2
+**Why:** SQLite in-memory keeps `pytest` fast and keyless; production uses PostgreSQL via Docker.
+**Impact:** UUID/pg-specific edge cases tested in integration manually.

extensions/theshloksschauhan/dropbox-folder-watcher/PROGRESS.md

@@ -0,0 +1,16 @@
+# Assumptions & Progress
+
+## Assumptions
+- Assigned S2 Dropbox folder watcher with write-back (Task 2.1). Task 1 lives in the private `doctask-*` repo.
+- PostgreSQL for durable job state; SQLite in-memory for keyless tests.
+- Polling is the source of truth; webhooks trigger an opportunistic scan.
+- Client isolation is folder-root path scoping, not Dropbox OAuth ACLs.
+
+## Progress
+- Human gate: worker stops at `REVIEW_PENDING`; export/write-back only after approve.
+- Preview toggle is process-wide and blocks billable SuperDocs calls.
+- Operation budget enforced before new jobs are created.
+- Webhook POST attempts a folder scan (falls back to daemon poll).
+- Folder nomination rejected if the path is outside the client root.
+- Dead Vite sidebar removed; treatments are the three assigned modes.
+- Task 4 draft: `SUBMISSION.md`.

extensions/theshloksschauhan/dropbox-folder-watcher/README.md

@@ -0,0 +1,47 @@
+# SuperDocs Dropbox Folder Watcher
+
+Studio / freelancer Dropbox watcher: a client drops a file, SuperDocs treats it, a human gates the diff, the export is filed beside the original. Built for the SuperDocs engineer task (assigned S2 build).
+
+Credit: built by **Shlok Chauhan** for the SuperDocs task.
+
+![Architecture](docs/task4-architecture.png)
+
+## What SuperDocs features it uses
+
+REST: upload document, send edit instruction (chat), approve proposed changes, export. Proposed-change content is double-parsed. Preview mode never calls billable upload/chat.
+
+## What it does
+
+- Nominate folders and treatments: **normalize**, **summarize** (companion brief), **respond** (standard response).
+- Debounces Dropbox sync so half-written files are not processed.
+- Never re-triggers on its own output (name marker + path registry + content hash).
+- Client folders cannot be nominated outside that client's Dropbox root.
+- Preview mode is a structural no-spend chokepoint (no billable SuperDocs calls).
+- Hourly operation budget per folder.
+- Console shows what ran, proposed changes, and what needs a human. Approve and reject are first-class API operations (machine-drivable).
+
+## Quick start
+
+```bash
+cp backend/.env.template backend/.env
+docker-compose up -d --build
+docker-compose exec api alembic upgrade head
+```
+
+- Console: http://localhost:5173
+- API: http://localhost:8001
+
+Fill `DROPBOX_ACCESS_TOKEN` and `SUPERDOCS_API_KEY` for a live loop. Without them, tests still pass and the console can seed a drop for UI review.
+
+## Tests (no live key)
+
+```bash
+cd backend
+pytest
+```
+
+## Formats / domain
+
+Studio documents in a Dropbox shared folder: `.docx`, `.pdf`, and other extensions you allow on the folder config. Second-run proof: a different file in the same nominated folder, not a replay of the first bytes.
+
+Cuts and why: see [NOT_DOING.md](NOT_DOING.md). Architecture: [ARCHITECTURE.md](ARCHITECTURE.md). Submission pack: [SUBMISSION.md](SUBMISSION.md).

extensions/theshloksschauhan/dropbox-folder-watcher/SUBMISSION.md

@@ -0,0 +1,45 @@
+# One-page write-up + architecture (Task 4)
+
+Built for a small studio that already lives in Dropbox. A client drops a file into a nominated folder; SuperDocs applies a treatment (normalize / companion brief / standard response); a human reviews proposed changes in a console; the approved export is filed beside the original with a `.superdocs.` name. Own outputs never re-enter the loop. Preview mode spends nothing.
+
+## Measured (offline, no live keys)
+
+- Backend tests: `pytest` in `backend/` — crash-resume, concurrent claim, anti-loop, debounce, preview $0 spend, hourly budget, path isolation, human gate (prepare does not write back; reject does not write back).
+- Clone → run: `docker-compose up -d --build` then `docker-compose exec api alembic upgrade head`. Console at `http://localhost:5173`.
+- Stopping rules: content-hash debounce, three-layer anti-loop, `operation_budget_per_hour`, global preview toggle.
+
+## Trade-offs
+
+Polling + optional webhook scan instead of OAuth-per-client. Isolation is path-scoped to each client's Dropbox root, not Dropbox shared-link ACLs (documented in `NOT_DOING.md`). Job-level approve/reject, not per-hunk, because SuperDocs approve is document-level.
+
+## Limitations (honest)
+
+Without Dropbox/SuperDocs tokens, discovery is seeded from the console. Preview jobs never produce a real SuperDocs diff. The assigned build is this watcher; the Task 1 pile-analyst lives in the private `doctask-*` repo.
+
+## Architecture
+
+```mermaid
+flowchart LR
+  Dropbox -->|poll or webhook| Watcher
+  Watcher -->|stable + not own output + budget OK| Job[(PostgreSQL jobs)]
+  Job --> Worker
+  Worker -->|upload + chat| SuperDocs
+  Worker -->|REVIEW_PENDING| Console[React console]
+  Console -->|approve or reject| API
+  API -->|approve + export| SuperDocs
+  API -->|write-back .superdocs.| Dropbox
+```
+
+## Four form answers (draft)
+
+1. **What broke?** First request in a SuperDocs session can stall; proposed-change payloads are JSON strings that need a second parse; large-doc runs sit silent for minutes (still processing). Dropbox fires events on half-written files. Console preview used to be UI-only until wired to a process-level flag.
+
+2. **One morning number:** share of `REVIEW_PENDING` jobs resolved within 24 hours. It is the product: edits that never get gated do not ship.
+
+3. **Next five, in order:** (1) per-client Dropbox OAuth, (2) webhook fast-path as default, (3) item-level reject of individual hunks, (4) Slack ping on review, (5) cost dashboard per folder. Drop: in-app mock-drop once live Dropbox is the demo path. Fix immediately: empty diffs if the second JSON parse regresses.
+
+4. **Self-running ops:** watchers on docs.superdocs.app + GitHub issues; an agent files bugs from the in-app button; CI runs keyless tests; a daily digest of failed jobs and ops spend; humans only approve production schema changes, pricing, and outreach. What breaks first: silent model-quality drift, not infrastructure.
+
+## Video checklist (record this)
+
+1. Watch a folder (two clients if possible). 2. Drop a file. 3. Console shows real proposed changes. 4. Reject one job, approve another. 5. Output appears beside source, named `*.superdocs.*`. 6. Touch the output — no new job. 7. Preview on — no spend, no write-back.

extensions/theshloksschauhan/dropbox-folder-watcher/TASK.md

@@ -0,0 +1,6 @@
+# Working Agreements & Rules
+1. **Source of Truth**: The SuperDocs Engineer Task Document.
+2. **Migrations**: Always write an Alembic migration alongside any SQLAlchemy model change.
+3. **Core Services**: Never touch `superdocs_client.py`'s preview chokepoint without flagging it for manual review.
+4. **Resilience**: State is durable in PostgreSQL. Never rely on in-memory state for job tracking.
+5. **No Scope Creep**: Stick to the mandatory requirements for the S2 Dropbox watcher. Defer extras to `NOT_DOING.md`.

extensions/theshloksschauhan/dropbox-folder-watcher/TASK4_FOUR_QUESTIONS.md

@@ -0,0 +1,38 @@
+# Four questions (submission form)
+
+## 1. What broke?
+
+Using SuperDocs: the first instruction in a fresh session can hang or fail until things warm up (as the task email warned). Proposed-change payloads arrive as a JSON string inside JSON; missing the second parse yields empty diffs with every field `undefined`. Large documents sit with no progress for minutes — still processing, not a crash. Exports do not cost operations, which is easy to miss when budgeting a loop.
+
+Using Dropbox: events fire on half-written uploads; a content-hash debounce is required. Write-back of our own file would retrigger the loop without a path + hash registry.
+
+Using our own first build: the worker originally auto-approved in the same run, which made the console gate fake. Preview was a UI toggle that did not block spend. Those are fixed: the worker stops at `REVIEW_PENDING`; preview is a chokepoint in the SuperDocs client.
+
+I have not filed these through the in-app bug button yet; I will, because reported bugs score.
+
+## 2. If you ran SuperDocs, what one number would you watch every morning?
+
+**Share of proposed edits that a human resolves (approve or reject) within 24 hours.**
+
+Generating diffs is not the product. The product is edits that land, gated, in the real file. If that number falls, either the model is noisy, review is too expensive, or the loop is writing without a person. I would pair it with a secondary: operations spent per *committed* export, so spend without write-back shows up.
+
+## 3. Five features next, in order — what I would drop — what I would fix now
+
+1. Per-client Dropbox OAuth (real shared-folder permissions).
+2. Webhook-first scan with polling as backup.
+3. Item-level reject of individual proposed hunks, if SuperDocs ever exposes it; until then, clearer job-level diffs.
+4. Slack/email when a job hits `REVIEW_PENDING`.
+5. Per-folder ops dashboard (budget used vs cap).
+
+**Drop:** the in-console “seed sample drop” once a live Dropbox demo is the default path.
+
+**Fix immediately:** any regression of the double JSON parse; empty review cards destroy trust. Also: say “still processing” in the console when SuperDocs is silent for >30s.
+
+## 4. How would day-to-day engineering and GTM run themselves?
+
+Loops, not headcount:
+
+- **Product:** every in-app bug report becomes a ticket with “did / expected / got / blocked / file.” An agent triages duplicates against docs.superdocs.app. Humans only re-open severity.
+- **Engineering:** CI runs keyless tests on every PR. Staging has a golden corpus. A nightly job posts failed runs, p95 latency, and ops spend. Schema and billing changes stay human-gated.
+- **GTM:** a watched folder of public case studies; SuperDocs drafts a one-page note; a human approves before anything is sent. No candidate or employee ever emails prospects on the company’s behalf unless that is their job.
+- **What breaks first:** silent quality drift (diffs look fine, lawyers disagree), not CPU. The check is sampled human review of committed exports, not more dashboards.

extensions/theshloksschauhan/dropbox-folder-watcher/TASK4_ONE_PAGE.md

@@ -0,0 +1,27 @@
+# Studio Dropbox Watcher — one page
+
+**Built by:** Shlok Chauhan  
+**For:** SuperDocs engineer task, assigned build (S2) — Dropbox folder watcher with write-back  
+**Who it serves:** A small studio or freelancer collective whose whole business already lives in Dropbox.
+
+## What it does
+
+A client drops a file into a nominated shared folder. The watcher waits until Dropbox has finished writing it, ignores anything we already wrote back, and sends the file through SuperDocs (upload → edit instruction → human review → approve → export). The finished file is stored **beside** the original with a stable name: `{basename}.superdocs.{treatment}{ext}`. Treatments are normalize-to-template, companion brief, or standard response.
+
+## Results you can measure
+
+- **75** pytest cases, **~2s**, **no live API key**.
+- Claims under test: half-written files wait; own outputs never re-queue; two workers cannot claim the same job; preview mode spends **zero** SuperDocs operations; reject writes **nothing** back; crash after review resumes without a second upload.
+- Clone to running: `docker-compose up -d --build` then `alembic upgrade head`. Console: `http://localhost:5173`.
+
+## Why these trade-offs
+
+Polling is the source of truth because Dropbox webhooks only say “something changed.” A generated access token is enough to prove the loop at S2; per-client OAuth would dominate the build. Approve/reject is **job-level** because SuperDocs’ four-call contract approves a document, not a hunk. Client isolation is **path scoping** under each client’s Dropbox root, not Dropbox shared-link ACLs.
+
+## Honest limits
+
+Without tokens the console can still seed a job for UI review. Preview mode never calls billable SuperDocs endpoints, so it will not show a live diff. The first SuperDocs request in a cold session can stall; that is their product, not this watcher. This build is an integration **on** SuperDocs, not a clone of SuperDocs.
+
+## SuperDocs surfaces used
+
+Upload, chat (edit instruction, including the double JSON parse on proposed changes), approve, export. Optional webhook receiver; daemon poll covers discovery.

extensions/theshloksschauhan/dropbox-folder-watcher/Task_3_Use_Cases.md

@@ -0,0 +1,37 @@
+# Task 3 — Where SuperDocs sells
+
+I have not contacted anyone. These are buyers I would *name* on a form, not a lead list.
+
+Knowing nobody at these companies is the normal answer here.
+
+### 1. Boutique architecture / interior studios (Dropbox as the OS)
+A client drops a messy brief into a shared Dropbox folder; SuperDocs returns a studio-template proposal or a one-page companion brief beside it. **Buyers:** small architecture and interior practices that already bill from Dropbox, not from a DMS. **Companies:** local studio collectives (e.g. a 8–20 person practice), WeWork-style freelance clusters, and design shops like Gensler *regional* studios that still share client folders. **Who I know:** nobody.
+
+### 2. Vendor contract + amendment piles
+Legal ops needs one register of payment terms, liability caps, and renewal dates that stay cited to clause and file. **Buyers:** procurement legal at mid-market SaaS. **Companies:** Freshworks, Zoho, Postman. **Who I know:** nobody.
+
+### 3. Clinical protocol vs. lab manual consistency
+A protocol amendment must not silently disagree with the lab manual. SuperDocs flags the mismatch with a citation; a person gates the finding. **Buyers:** CRO medical writing. **Companies:** IQVIA, Syneos, PPD. **Who I know:** nobody. Certification is a roadmap conversation, not a claim.
+
+### 4. Commercial lease abstraction
+100-page leases → 2-page abstract (rent, break, CAM). An amendment that contradicts the original is a conflict, not an overwrite. **Buyers:** asset managers. **Companies:** CBRE, JLL, Cushman & Wakefield. **Who I know:** nobody.
+
+### 5. Insurance claim vs. policy
+Claim pack vs. the governing policy; covered / not covered with page pins. Human reviews before anything is sent to a claimant. **Buyers:** TPA claims examiners. **Companies:** Sedgwick, Gallagher Bassett. **Who I know:** nobody.
+
+### 6. Construction weekly pack
+Site notes + invoices + change orders → one status brief. A change order that breaks the baseline is surfaced. **Buyers:** PMOs at GCs. **Companies:** Turner, Skanska, L&T Construction. **Who I know:** nobody.
+
+### 7. Bank policy vs. new bulletin
+A FINRA/SEBI circular vs. the internal manual: which paragraphs aged out. **Buyers:** compliance change-management. **Companies:** HDFC Bank, ICICI, a US broker-dealer’s compliance team. **Who I know:** nobody.
+
+### 8. RFP response factory
+A 200-page RFP becomes a checklist; answers are drafted from prior winning text, still reviewed. **Buyers:** bid teams. **Companies:** Infosys, Tata Consultancy Services, IBM India bid desks. **Who I know:** nobody.
+
+### 9. PE / law-firm diligence folders
+Employment agreements land in a VDR-like Dropbox; change-of-control clauses get findings. **Buyers:** associates, not the archive itself (SuperDocs is not a DMS). **Companies:** Khaitan & Co, AZB, a PE ops team at a house like KKR. **Who I know:** nobody.
+
+### 10. Municipal permit packs
+Application vs. zoning note: honest “no findings” or exact setback misses. **Buyers:** a city’s permit analysts. **Companies / orgs:** a mid-size Indian municipal corporation’s building department; analogous to NYC DOB at a smaller scale. **Who I know:** nobody.
+
+The Dropbox watcher (Task 2) is the demo for **use case 1**. The others are where I would sell the same four-call loop next.