docs(adr): design Inkspan-based LLM email writing guidance - #1322
docs(adr): design Inkspan-based LLM email writing guidance#1322seonghobae wants to merge 20 commits into
Conversation
|
Warning Review limit reachedNext included review available in 58 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe PR adds ADR governance updates, a proposed Inkspan-based LLM email guidance design, an 18-task implementation plan, fast-mlsirm calibration evidence, and release and verification requirements. It does not add product implementation. ChangesEmail writing guidance
Estimated code review effort: 3 (Moderate) | ~25 minutes Merge Risk: 🟡 Moderate · up to This documentation-only PR defines the future email-guidance contract but still leaves unresolved policy-admission, rollback, evaluation-reproducibility, source-verification, and criterion-naming issues. No current production behavior changes, but these gaps could lead to unsafe or irreproducible implementation later, so merge should wait for correction or explicit owner acceptance. 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
PR governance metadata gate update for PR governance metadata gate is waiting on current-head requirements; see the latest check for pending reasons. |
|
@coderabbitai review Please review exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== HEAD =='
git rev-parse HEAD
git status --short
printf '%s\n' '== DIFF SUMMARY VS DEVELOP =='
git diff --stat develop...HEAD
git diff --name-status develop...HEAD
printf '%s\n' '== DOCUMENT FILES =='
fd -t f . docs | sort
printf '%s\n' '== CHANGED DOCUMENT HEADINGS =='
git diff --name-only develop...HEAD | while IFS= read -r f; do
case "$f" in
*.md)
printf '\n--- %s ---\n' "$f"
rg -n '^(#|##|###) ' "$f" || true
;;
esac
done
printf '%s\n' '== FOCUSED TERMS IN CHANGED DOCS =='
git diff --name-only develop...HEAD | while IFS= read -r f; do
case "$f" in
*.md)
printf '\n--- %s ---\n' "$f"
rg -n -i -C 2 'keyword|regex|fallback|candidate|judge|adjudicat|calibrat|evaluation_only|PII|privacy|retention|Inkspan|release|lockfile|verif|rollback|acceptance|stale|selector|send gate|undo|DIF|drift|reliability' "$f" || true
;;
esac
done
printf '%s\n' '== PR METADATA =='
gh pr view 1322 --repo ContextualWisdomLab/naruon --json number,title,headRefOid,baseRefName,files,url --jq '{number,title,headRefOid,baseRefName,url,files:[.files[].path]}'Length of output: 232437 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== ADR: WORKFLOW, PRIVACY, ACCEPTANCE, ROLLBACK =='
sed -n '21,197p' docs/adr/0001-inkspan-backed-llm-email-writing-guidance.md
printf '%s\n' '== DESIGN: API, ADMISSION, MEASUREMENT, PRIVACY, RELEASE =='
sed -n '112,180p' docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md
sed -n '223,371p' docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md
sed -n '373,558p' docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md
printf '%s\n' '== PLAN: DEPENDENCY, PRIVACY, POLICY, ACCEPTANCE, RELEASE =='
sed -n '39,160p' docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md
sed -n '275,395p' docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md
sed -n '546,705p' docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md
printf '%s\n' '== CROSS-DOCUMENT TERM CONSISTENCY =='
for term in \
'evaluation_only' \
'approved policy' \
'immutable released' \
'rollback' \
'no raw' \
'send gate' \
'candidate reviewer' \
'independent judge' \
'adjudicat'; do
printf '\n-- %s --\n' "$term"
rg -n -i --glob '*.md' "$term" \
docs/adr/0001-inkspan-backed-llm-email-writing-guidance.md \
docs/doctoring/llm-email-writing-guidance-and-fast-mlsirm-judge.md \
docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md \
docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md \
|| true
done
printf '%s\n' '== COMPANION PR METADATA =='
gh pr view 248 --repo ContextualWisdomLab/inkspan \
--json number,state,title,headRefOid,baseRefName,mergedAt,url \
--jq '{number,state,title,headRefOid,baseRefName,mergedAt,url}' || true
printf '%s\n' '== FAST-MLSIRM PR METADATA =='
gh pr view 733 --repo ContextualWisdomLab/fast-mlsirm \
--json number,state,title,headRefOid,baseRefName,mergedAt,url \
--jq '{number,state,title,headRefOid,baseRefName,mergedAt,url}' || trueLength of output: 76893
I found one measurement-validity issue.
The reviewed documents otherwise define the requested boundaries clearly:
|
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Reconcile the Ready email-writing design onto current protected develop. Keep topic-intelligence ADRs 0001-0003 and accepted calendar ADR-0004. Renumber the email-writing decision to ADR-0005 so numbered identifiers stay unique. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
|
@coderabbitai review Please review the unchanged exact current head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
expected_head='bfc2df112136bb9fe358778d701e78bf9e78b685'
base='dd8d1519'
printf '%s\n' '== EXACT HEAD AND MERGE =='
git rev-parse HEAD
git show -s --format='commit=%H%nparents=%P%nsubject=%s' HEAD
git status --short
git merge-base --is-ancestor "$base" HEAD && echo "base_is_ancestor=yes" || echo "base_is_ancestor=no"
if [ "$(git rev-parse HEAD)" != "$expected_head" ]; then
echo "ERROR: checked-out HEAD does not match requested exact head" >&2
exit 1
fi
printf '%s\n' '== DIFF AGAINST LIVE DEVELOP COMMIT =='
git diff --stat "$base...HEAD"
git diff --name-status "$base...HEAD"
printf '%s\n' '== CHANGED MARKDOWN HEADINGS =='
git diff --name-only "$base...HEAD" | while IFS= read -r f; do
case "$f" in
*.md)
printf '\n--- %s ---\n' "$f"
rg -n '^(#{1,4}) ' "$f" || true
;;
esac
done
printf '%s\n' '== ADR INDEX AND IDENTIFIER CHECK =='
if [ -f docs/adr/README.md ]; then
sed -n '1,240p' docs/adr/README.md
fi
fd -t f -e md . docs/adr | sort | while IFS= read -r f; do
printf '\n--- %s ---\n' "$f"
sed -n '1,45p' "$f"
done
printf '%s\n' '== FOCUSED CONTRACTS IN REVIEW DOCUMENTS =='
for f in \
docs/adr/0005-inkspan-backed-llm-email-writing-guidance.md \
docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md \
docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md \
docs/doctoring/llm-email-writing-guidance-and-fast-mlsirm-judge.md
do
if [ -f "$f" ]; then
printf '\n--- %s ---\n' "$f"
rg -n -i -C 3 \
'keyword|regex|fallback|candidate|independent judge|adjudicat|policy artifact|calibrat|threshold|holdout|data split|evaluation protocol|publish|PII|privacy|retention|raw.content|telemetry|immutable|release|lockfile|rollback|acceptance|stale|selector|send' \
"$f" || true
else
echo "MISSING: $f"
fi
done
printf '%s\n' '== POLICY AND EVALUATION CONTEXT FROM IMPLEMENTATION PLAN =='
plan='docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md'
if [ -f "$plan" ]; then
rg -n '^(##|###) ' "$plan"
sed -n '530,680p' "$plan"
fi
printf '%s\n' '== PR METADATA =='
gh pr view 1322 --repo ContextualWisdomLab/naruon \
--json number,headRefOid,baseRefName,state,mergeable,reviewDecision,url \
--jq '{number,headRefOid,baseRefName,state,mergeable,reviewDecision,url}'Length of output: 124240 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== ADR IDENTIFIERS AND INDEX LINKS =='
printf 'ADR headers: '
rg -n '^# ADR-[0-9]{4}:' docs/adr --glob '*.md' | wc -l
printf 'Unique ADR header identifiers: '
rg '^# ADR-[0-9]{4}:' docs/adr --glob '*.md' \
| sed -E 's/^# (ADR-[0-9]{4}):.*/\1/' | sort -u | wc -l
printf 'Indexed ADR markdown links: '
rg -o '\[ADR-[0-9]{4}\]\([^)]*\.md\)' docs/adr/README.md | wc -l
printf 'Conflict markers in changed docs: '
git diff --name-only dd8d1519...HEAD | while IFS= read -r f; do
case "$f" in *.md) printf '%s\0' "$f" ;; esac
done | xargs -0 -r rg -n '^(<<<<<<<|=======|>>>>>>>)' || true
printf '%s\n' '== REQUIRED ADR-0005 SECTIONS =='
for heading in \
'Context' \
'Alternatives considered' \
'Decision' \
'Consequences' \
'Failure and recovery' \
'Security and privacy impact' \
'Accessibility' \
'Compatibility and migration' \
'Verification and acceptance evidence' \
'Research and standards traceability' \
'Rollback or supersession'
do
if rg -qx "## ${heading}" docs/adr/0005-inkspan-backed-llm-email-writing-guidance.md; then
printf 'present: %s\n' "$heading"
else
printf 'MISSING: %s\n' "$heading"
fi
done
printf '%s\n' '== ALL MEASUREMENT-PUBLICATION REFERENCES =='
rg -n -i -C 2 \
'pre.?specif|pre.?register|threshold|holdout|data split|evaluation protocol|publication job|policy artifact|dataset.*hash|hash.*dataset' \
docs/adr/0005-inkspan-backed-llm-email-writing-guidance.md \
docs/superpowers/specs/2026-08-12-inkspan-llm-email-writing-guidance-design.md \
docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md \
docs/doctoring/llm-email-writing-guidance-and-fast-mlsirm-judge.md
printf '%s\n' '== TASK 14 WITH LINE NUMBERS =='
nl -ba docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md | sed -n '546,578p'Length of output: 16518
I found one issue.
The requested ADR checks are consistent at this head:
The documents also retain the required semantic-authority boundary, immutable dependency gates, privacy controls, fail-closed review behavior, and rollback that preserves Inkspan authoring and the existing send path. This review does not provide the required qualifying non-author approval. You are interacting with an AI system. |
…ask4' into feat/llm-email-writing-orchestrator-task5 Retarget Task 5 onto live #1329 head 4570747 (merged onto live #1328 51fb5e8 / #1327 fb7c406 / #1322 bfc2df1 / develop@dd8d1519). Preserve the hardened contextual-orchestrator boundary. Do not restore write-capable Task 5 promotion/finalize workflows. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headbfc2df112136bb9fe358778d701e78bf9e78b685. -
Head SHA:
bfc2df112136bb9fe358778d701e78bf9e78b685 -
Workflow run: 32137363745
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Docs (5 files)"]
S1 --> I1["operator or user guidance"]
I1 --> R1["Review risk: Docs (5 files)"]
R1 --> V1["docs review"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Docs (5 files)"]
S1 --> I1["operator or user guidance"]
I1 --> R1["Review risk: Docs (5 files)"]
R1 --> V1["docs review"]
|
|
Current-head documentation fixes pushed at |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Stale review: all review-thread comments on this PR are resolved and the reviewer's cited commit predates the current head, which passes all non-metadata-gate required checks (verified via gh pr checks and the reviewThreads GraphQL query — 0 unresolved threads). Dismissing as superseded per AGENTS.md stale-review guidance.
opencode-agent
Bot
added
area: ui-ux
2 participants
ADR decision summary
Naruon will implement Grammarly-like email guidance as an LLM-native contextual review workflow, not as a keyword, regex, phrase-list, sender-domain, recipient-count, language-name, nearest-text, or positional classifier.
Provider failure, malformed output, uncertainty, Judge disagreement, expired or unsupported calibration policy, stale revision, or unsupported language/context produces abstention or review unavailability. There is no semantic keyword fallback. Editing and sending remain available.
Included
docs/superpowers/plans/2026-08-12-llm-email-writing-guidance-implementation.md.Runtime and measurement boundary
The candidate reviewer and independent Judge are separate roles and calls. A candidate is not admitted merely because its generator returned valid JSON or a high confidence value.
The Judge consumes explicit criterion descriptions and ordered categories through a released, immutable fast-mlsirm adapter. Runtime admission applies a versioned calibration policy. Mandatory preservation criteria—including fact, intent, actor, deadline, technical precision, actionability, and request strength—cannot be compensated for by a high average score.
The synchronous path consumes a pre-published policy. IRT fitting, multilevel/rater analysis, DIF, category-count ablation, reliability, and drift analysis run offline or nearline. An
evaluation_onlyartifact can exercise the pipeline but cannot emit production user-facing diagnostics.Cross-repository dependency boundary
Companion Inkspan design and implementation-plan PR:
ContextualWisdomLab/inkspan#248.Naruon will consume only immutable released artifacts with lockfile and package-verification evidence. Mutable branches, copied editor forks, source archives, git dependencies, and local paths are not production dependencies.
The first migration preserves the existing plain-text send contract. Inkspan provides authoring and deterministic plain-text projection. Rich HTML sending is a separate reviewed slice.
Evaluation and policy publication
Production policy publication requires human reference evidence and contrast sets that distinguish contextual semantics from lexical triggers, including paraphrases, quotations, code and identifiers, context-sensitive recipient scenarios, firm legitimate requests, technically unsuitable requests, Korean, English, mixed-language, CJK, hostile-Unicode, and negative controls.
Evaluation records issue/category precision and recall, span quality, replacement correctness, unsupported claims, preservation, calibration, human disagreement, DIF, temporal drift, latency, tokens, workflow steps, and cost. Scheduled live evaluation uses
NVIDIA_NIM_API_KEY; model calls never useCOPILOT_GITHUB_TOKEN.Scope and claim boundary
This is a design-and-implementation-plan PR. It changes no runtime code, database, package dependency, UI, model configuration, or release version. It does not claim that the feature, any language profile, or any model is validated or shipped. Runtime implementation must follow the committed TDD plan.
Customer next action
This PR records the design and TDD plan only. Customers should continue to write and send email with the current editor and send path. No writing-guidance feature, language profile, or model is available in product until a later implementation PR lands on protected
developwith independent review and exact-head product CI.Exact-head verification
Previous head (stale body advertised
2f7b5780; live PR head before this merge):Current head:
Live protected
developmerged as the second parent:Normal merge commit (not squash, not rebase, not force-push):
b4953c1510478157ae226304c4919e1344ae3b1ddd8d15191338b841f9e6f3a06507c6a5643b95d0ADR-index conflict resolution:
0004-status-weighted-calendar-conflicts.mdfrom#1367);Local validation counts on
bfc2df112136bb9fe358778d701e78bf9e78b685:git diff --checkon merge-resolution files: okdevelop: 5 documentation files, 0 runtime/database/dependency/UI/model/version filesPredecessor evidence does not transfer. Checks, reviews, and security evidence recorded for
2f7b5780,b4953c15, or any earlier head are non-passing for this head.Merge gate
Merge requires an independent non-author APPROVE on exact head
bfc2df112136bb9fe358778d701e78bf9e78b685plus exact-head product CI. This update does not approve, merge, squash, empty-requeue, or force-cancel.Review focus
Summary by CodeRabbit